🖨️ Printing Instructions: Press Ctrl/Cmd + P and select "Save as PDF".

Prim's, Kruskal's, and the Cut Property

Learning Goals

Understand the Minimum Spanning Tree problem.
Master Kruskal's algorithm with Union-Find.
Master Prim's algorithm with priority queues.
Learn the Cut Property and correctness proofs.
Understand the Union-Find data structure in detail.

Problem Definition

Spanning Tree

Given a connected, weighted graph $G = (V, E)$.
A Spanning Tree $T$ is a subset of $E$ that:
1. Touches all vertices $V$.
2. Is Connected.
3. Is Acyclic (Tree).
A spanning tree always has exactly $|V| - 1$ edges.
A graph can have many spanning trees.

Minimum Spanning Tree (MST)

Cost of tree = sum of edge weights.
Goal: Find a spanning tree with minimum total cost.
Applications: Network design (Cable TV, Roads, Circuitry).
Key idea: Connect everyone as cheaply as possible.

Uniqueness

If all edge weights are distinct, the MST is unique.
If weights are not distinct, there may be multiple MSTs — but all share the same total cost.
Prerequisite: The graph must be connected. Otherwise, no spanning tree exists (we get a Minimum Spanning Forest).

The Cut Property

Definition of a Cut

A Cut $(S, V \setminus S)$ partitions vertices into two non-empty sets.
An edge crosses the cut if one endpoint is in $S$ and the other in $V \setminus S$.
A cut respects a set of edges $A$ if no edge in $A$ crosses the cut.

The Cut Property (Crucial)

Theorem: For any cut, the minimum-weight edge crossing the cut belongs to some MST.
Intuition: To connect $S$ and $V \setminus S$, we MUST pick at least one crossing edge. Be greedy — pick the cheapest.
This single property justifies both Prim's and Kruskal's algorithms.

Proof of Cut Property

Suppose MST $T$ does not contain the min-weight crossing edge $e$.
Adding $e$ to $T$ creates a cycle.
This cycle must cross the cut again via some other edge $e'$.
Since $w(e) \le w(e')$, replacing $e'$ with $e$ gives a tree with weight $\le w(T)$.
If $w(e) < w(e')$, the new tree is strictly cheaper — contradicting that $T$ is an MST.

Generic Greedy MST

Generic Algorithm

$A = \emptyset$.
While $A$ is not a spanning tree:
Find a cut that respects $A$ (no edge in $A$ crosses it).
Add the light edge (min-weight crossing edge) to $A$.
Return $A$.
Both Prim's and Kruskal's are instantiations of this framework.

Two Instantiations

1. Prim's Algorithm: Cut separates "In Tree" from "Not In Tree". Grows one tree outward.
2. Kruskal's Algorithm: Cut separates components in a forest. Merges components together.

Kruskal's Algorithm

Strategy

Sort all edges by weight (ascending).
Iterate through edges in order.
If edge $(u, v)$ connects two different components → Add it.
If $u$ and $v$ are already connected → Discard it (would create a cycle).
Stop when we have $|V| - 1$ edges.

Kruskal(G)

A = empty
for each vertex v:
    Make-Set(v)
Sort edges E by weight ascending
for each edge (u, v) in sorted E:
    if Find-Set(u) != Find-Set(v):
        A = A ∪ {(u, v)}
        Union(u, v)
return A

Disjoint Set Union (DSU / Union-Find)

Data structure to track which component each vertex belongs to.
Make-Set(x): Create singleton set $\{x\}$.
Find-Set(x): Return representative of the set containing $x$.
Union(x, y): Merge the sets containing $x$ and $y$.
Optimizations: Path Compression + Union by Rank.
With both optimizations, $m$ operations run in $O(m \cdot \alpha(n))$ — nearly $O(m)$.

Union-Find Deep Dive

Union-Find: The Big Picture

Problem: As Kruskal adds edges, how do we quickly check if two vertices are already connected?
Answer: Union-Find — a data structure that tracks groups (connected components).
Think of it like tracking friend groups:
Make-Set: Each person starts as their own group.
Find: "What group is person $x$ in?"
Union: "Merge the groups containing $x$ and $y$."
Kruskal calls Find before every edge, and Union every time an edge is accepted.

Representing Sets as Trees

Each set (component) is stored as a rooted tree.
Every element $x$ has a parent pointer: `parent[x]`.
The root of the tree is the representative of the set: `parent[root] = root`.
Example with set $\{A, B, C\}$:
A is the root: `parent[A] = A`
B's parent is A: `parent[B] = A`
C's parent is A: `parent[C] = A`
All elements in the same tree belong to the same set.

Make-Set and Naive Find

Make-Set(x): Simply set `parent[x] = x` and `rank[x] = 0`. Creates a single-node tree.
Find-Set(x) — naive version: Follow parent pointers up until you reach the root.
`while parent[x] ≠ x: x = parent[x]`
Return `x` (the root = representative).
Problem: Without optimization, trees can become tall chains.
Union(A,B), Union(B,C), Union(C,D) → chain: D → C → B → A.
Find(D) traverses 3 pointers — in general, $O(n)$ per operation!
We need two tricks to fix this...

Trick 1: Path Compression

Idea: During Find(x), make every node on the path point directly to the root.
Before Find(D) on chain D → C → B → A:
`parent = {A:A, B:A, C:B, D:C}`
After Find(D) with path compression:
`parent = {A:A, B:A, C:A, D:A}`
The tree becomes almost completely flat — future Finds are nearly instant!
Implementation: one line of recursion:
`Find(x): if parent[x] ≠ x then parent[x] = Find(parent[x]). Return parent[x].`

Trick 2: Union by Rank

Idea: When merging two trees, always attach the shorter tree under the taller one.
Each node has a rank — an upper bound on the height of its subtree.
Union(x, y): Find roots $r_x$ and $r_y$.
If `rank[r_x] < rank[r_y]`: attach $r_x$ under $r_y$.
If `rank[r_x] > rank[r_y]`: attach $r_y$ under $r_x$.
If `rank[r_x] = rank[r_y]`: pick either as root, increment its rank by 1.
This prevents chains from forming in the first place.
Without path compression, Union by Rank alone guarantees $O(\log n)$ per Find.

Both Tricks Together

Path Compression + Union by Rank = near-constant time per operation.
Amortized cost: $O(\alpha(n))$ per operation.
$\alpha(n)$ is the inverse Ackermann function:
$\alpha(n) \le 4$ for any $n$ up to $2^{2^{2^{65536}}}$.
In practice: treat it as $O(1)$.
For Kruskal's algorithm with $E$ edges:
$2E$ Find operations + up to $V-1$ Union operations.
Total DSU cost: $O(E \cdot \alpha(V)) \approx O(E)$.

Union-Find Implementation

Make-Set(x):
    parent[x] = x
    rank[x] = 0

Find-Set(x):              // with path compression
    if parent[x] ≠ x:
        parent[x] = Find-Set(parent[x])
    return parent[x]

Union(x, y):              // with union by rank
    rx = Find-Set(x)
    ry = Find-Set(y)
    if rx = ry: return     // already same set
    if rank[rx] < rank[ry]:
        parent[rx] = ry
    else if rank[rx] > rank[ry]:
        parent[ry] = rx
    else:
        parent[ry] = rx
        rank[rx] = rank[rx] + 1

Union-Find: Step-by-Step

Start: Make-Set for A, B, C, D. `parent = [A,B,C,D]`, `rank = [0,0,0,0]`.
Union(A, B): roots A, B. Equal rank → attach B under A. `rank[A] = 1`.
`parent = [A,A,C,D]`, `rank = [1,0,0,0]`.
Union(C, D): roots C, D. Equal rank → attach D under C. `rank[C] = 1`.
`parent = [A,A,C,C]`, `rank = [1,0,1,0]`.
Union(B, D): Find(B) = A, Find(D) = C. Equal rank → attach C under A. `rank[A] = 2`.
`parent = [A,A,A,C]`, `rank = [2,0,1,0]`.
Find(D): path D → C → A. Path compression: `parent[D] = A`, `parent[C] = A`.
`parent = [A,A,A,A]` — tree is now flat!

Interactive: Union-Find

🚀 Interactive Demo: union_find_demo.html

Kruskal's Analysis

Sorting edges: $O(E \log E)$.
DSU operations: $O(E) \cdot \alpha(V) \approx O(E)$.
Total: $O(E \log E) = O(E \log V)$.
(Since $E \le V^2$, we have $\log E \le 2 \log V = O(\log V)$.)

Kruskal Trace

Edges: (A,B,1), (C,D,2), (A,C,4), (B,C,5).
Step 1: Pick (A,B,1). Components: {A,B}, {C}, {D}.
Step 2: Pick (C,D,2). Components: {A,B}, {C,D}.
Step 3: Pick (A,C,4). Merges {A,B} and {C,D}. Component: {A,B,C,D}.
Step 4: (B,C,5) — same component. Discard.
MST edges: (A,B), (C,D), (A,C). Total cost: 1 + 2 + 4 = 7.

Interactive: Kruskal's Algorithm

🚀 Interactive Demo: kruskal_demo.html

Prim's Algorithm

Strategy

Start from an arbitrary root vertex $r$.
Maintain set $S$ of vertices already in the tree.
At each step, add the minimum-weight edge connecting $S$ to $V \setminus S$.
Repeat until all vertices are in $S$.
Very similar in structure to Dijkstra's algorithm.

Prim(G, r)

for each u in V:
    key[u] = ∞
    π[u] = NIL
key[r] = 0
Q = V   // Min-priority queue keyed by key[]
while Q is not empty:
    u = Extract-Min(Q)
    for each v in Adj[u]:
        if v ∈ Q and w(u, v) < key[v]:
            π[v] = u
            key[v] = w(u, v)   // Decrease-Key

Prim's Key Concepts

key[v]: Minimum weight of any edge connecting $v$ to the current tree.
π[v]: The vertex in the tree that causes $v$'s key value (parent in MST).
When $u$ is extracted, edge $(\pi[u], u)$ is added to the MST.
Neighbors of $u$ may get cheaper connections — update their keys (relaxation).

Prim's Analysis

Priority queue operations:
$|V|$ Extract-Min operations.
Up to $|E|$ Decrease-Key operations.
With Binary Heap: $O((V + E) \log V) = O(E \log V)$.
With Fibonacci Heap: $O(E + V \log V)$.
Identical complexity profile to Dijkstra's algorithm.

Prim Trace

Graph: A—B (weight 1), A—C (weight 4), B—C (weight 2). Start at A.
Init Q: [(A, 0), (B, ∞), (C, ∞)].
Pop A. Update neighbors: key[B] = 1, key[C] = 4.
Q: [(B, 1), (C, 4)].
Pop B. Neighbor C has w(B,C) = 2 < key[C] = 4. Update key[C] = 2, π[C] = B.
Q: [(C, 2)].
Pop C. Q empty. Done.
MST edges: (A, B) and (B, C). Total cost: 1 + 2 = 3.

Interactive: Prim's Algorithm

🚀 Interactive Demo: prim_demo.html

Comparison

Kruskal vs Prim

Kruskal's Algorithm
Best for sparse graphs ($E \approx V$).
Easy to implement with Union-Find.
Logic: Globally sort edges, merge components.
Prim's Algorithm
Best for dense graphs ($E \approx V^2$); with adjacency matrix: $O(V^2)$.
Logic: Locally grow a single tree.
Both run in $O(E \log V)$ with standard data structures.

Advanced Topics

Reverse-Delete Algorithm

Start with all edges (the full graph).
Sort edges in descending order of weight.
For each edge: if removing it does not disconnect the graph, remove it.
Result is an MST. (Reverse greedy — dual of Kruskal's.)
Slower in practice: connectivity checks are expensive.

Borůvka's Algorithm

Oldest MST algorithm (1926).
For each component, find the cheapest edge leaving it.
Add all such edges simultaneously. Merge components.
Repeat until one component remains.
Each phase halves the number of components → $O(\log V)$ phases.
Total: $O(E \log V)$. Highly parallelizable.

Common Pitfalls

Watch Out For...

Forgetting that MST requires a connected graph. If disconnected, you get a Minimum Spanning Forest.
Confusing Prim's with Dijkstra's: Prim uses edge weight as key, Dijkstra uses total distance.
In Kruskal's, forgetting to check if endpoints are in different components before adding an edge.
Assuming MST gives shortest paths between vertices — it does not.
Assuming a unique MST when edge weights are not distinct.
In Union-Find, forgetting path compression — without it, Find can be $O(n)$.
In Union by Rank, confusing rank with size — rank is an upper bound on height, not the number of elements.

All Interactive Demos

🚀 Interactive Demo: union_find_demo.html

🚀 Interactive Demo: kruskal_demo.html

🚀 Interactive Demo: prim_demo.html

Summary

Lecture Summary

The Cut Property is the theoretical foundation: the lightest crossing edge belongs to the MST.
Kruskal's: Sort edges globally, merge components with Union-Find. $O(E \log V)$.
Prim's: Grow tree locally with a priority queue. $O(E \log V)$ or $O(E + V \log V)$ with Fibonacci heap.
Union-Find uses path compression + union by rank for nearly $O(1)$ per operation.
MST ≠ Shortest Paths. Prim's ≠ Dijkstra's.
MST has exactly $|V| - 1$ edges and is unique when all weights are distinct.

Supplementary Resources

🚀 Interactive Demo: handout.html