Learning Goals

• Master the Bellman-Ford algorithm for graphs with negative edges.
• Detect negative-weight cycles.
• Understand the All-Pairs Shortest Paths (APSP) problem.
• Master the Floyd-Warshall algorithm.
• Compare Dijkstra, Bellman-Ford, and Floyd-Warshall.

Motivation

• Dijkstra fails with negative edges (finalizes vertices too early).
• Bellman-Ford solves SSSP even with negative edge weights.
• Can also detect negative-weight cycles.
• Tradeoff: slower than Dijkstra — $O(VE)$ vs $O(E \log V)$.

Interactive Demo

🚀 Interactive Demo: bellman_ford_demo.html

Key Idea

• Relax all edges $|V|-1$ times.
• After $k$ iterations, we have correct shortest paths using at most $k$ edges.
• Any shortest path has at most $|V|-1$ edges (no cycles in shortest paths).
• So after $|V|-1$ iterations, all shortest paths are found.
• No priority queue needed — just iterate over all edges repeatedly.

Bellman-Ford(G, s)

Init-Single-Source(G, s) // d[s]=0, others inf
for i = 1 to |V| - 1
  for each edge (u, v) in E
    Relax(u, v, w)
// Negative cycle detection:
for each edge (u, v) in E
  if d[v] > d[u] + w(u, v)
    return "Negative cycle exists"
return d[], pi[]

Example Trace

• Graph: $s \to A$ (4), $s \to B$ (5), $A \to B$ ($-3$), $B \to C$ (2).
• Init: $d[s]=0$, $d[A]=\infty$, $d[B]=\infty$, $d[C]=\infty$.
• Iteration 1: Relax $s \to A$: $d[A]=4$. Relax $s \to B$: $d[B]=5$. Relax $A \to B$: $4+(-3)=1 < 5$, $d[B]=1$. Relax $B \to C$: $1+2=3$, $d[C]=3$.
• Iteration 2: Check all edges again — no further improvements.
• Result: $d[A]=4$, $d[B]=1$, $d[C]=3$. Dijkstra would have gotten $d[B]=5$ wrong!

Complexity

• Outer loop: $|V|-1$ iterations.
• Inner loop: Iterate over all $|E|$ edges.
• Time: $O(VE)$.
• Space: $O(V)$ for $d[]$ and $\pi[]$ arrays.
• Slower than Dijkstra but handles negative edges.

How It Works

• After $|V|-1$ iterations, all shortest paths (if they exist) are found.
• Run one more pass over all edges.
• If any edge can still be relaxed: a negative cycle is reachable from $s$.
• Why? If no negative cycle exists, $|V|-1$ relaxations suffice. If relaxation still helps, distances can decrease forever.

Negative Cycle Implications

• If a negative cycle is reachable from $s$, shortest path is $-\infty$ for affected vertices.
• Bellman-Ford correctly reports this situation.
• Real-world example: currency arbitrage — cycle of exchange rates yielding profit.
• Dijkstra cannot detect negative cycles at all.

The APSP Problem

• Given weighted graph $G=(V, E)$.
• Compute shortest path distance between every pair of vertices.
• Output: $n \times n$ matrix $D$ where $D_{ij} = \delta(i, j)$.

Naive Approaches

• Run Dijkstra from every vertex: $O(V \cdot E \log V)$. Only works for non-negative edges.
• Run Bellman-Ford from every vertex: $O(V^2 E)$. Dense: $O(V^4)$. Too slow.
• Can we do better? Yes — Floyd-Warshall achieves $O(V^3)$.

Interactive Demo

🚀 Interactive Demo: floyd_warshall_demo.html

DP Formulation

• Number vertices $1, 2, \ldots, n$.
• Let $d_{ij}^{(k)}$ = shortest path from $i$ to $j$ using only vertices $\{1, \ldots, k\}$ as intermediates.
• Base case ($k=0$): $d_{ij}^{(0)} = w(i,j)$ (direct edge weight, or $\infty$).
• Recurrence: Consider whether vertex $k$ is on the shortest path:
• $$d_{ij}^{(k)} = \min(d_{ij}^{(k-1)},\; d_{ik}^{(k-1)} + d_{kj}^{(k-1)})$$

Understanding the Recurrence

• Case 1: Shortest path $i \to j$ does NOT use vertex $k$. Cost: $d_{ij}^{(k-1)}$.
• Case 2: Shortest path $i \to j$ DOES go through $k$. Cost: $d_{ik}^{(k-1)} + d_{kj}^{(k-1)}$.
• We take the minimum of both cases.
• After $k=n$, all vertices are allowed as intermediates — we have the answer.

Floyd-Warshall(W)

n = |V|
D = W  // Initialize with edge weights
for k = 1 to n
  for i = 1 to n
    for j = 1 to n
      D[i][j] = min(D[i][j], D[i][k] + D[k][j])
return D

Space Optimization

• We dropped the superscript $(k)$ — just one matrix $D$.
• When computing for pivot $k$: $D[i][k]$ and $D[k][j]$ don't change in iteration $k$.
• Why? Because $D[k][k] = 0$, so $\min(D[k][j], D[k][k]+D[k][j]) = D[k][j]$.
• Safe to overwrite in place. Space: $O(V^2)$.

Example Graph

• 3 vertices. Edges: $1 \to 2$ (3), $2 \to 1$ (8), $2 \to 3$ (2), $3 \to 1$ (5).
• Initial $D$: $[0, 3, \infty]$, $[8, 0, 2]$, $[5, \infty, 0]$.

Pivot Through Each Vertex

• $k=1$: Can $3 \to 2$ go through 1? $D[3][1]+D[1][2] = 5+3 = 8 < \infty$. Update!
• $k=2$: Can $1 \to 3$ go through 2? $D[1][2]+D[2][3] = 3+2 = 5 < \infty$. Update!
• $k=3$: Can $1 \to 2$ go through 3? $D[1][3]+D[3][2] = 5+8 = 13 > 3$. No change.
• Final $D$: $[0, 3, 5]$, $[8, 0, 2]$, $[5, 8, 0]$.

Floyd-Warshall Properties

• Time: $O(V^3)$ — three nested loops.
• Space: $O(V^2)$ — the distance matrix.
• Handles negative edges: Yes! (Unlike Dijkstra.)
• Negative cycle detection: Check if $D[i][i] < 0$ for any $i$ after algorithm completes.
• Simple to implement: ~5 lines of core code.

Predecessor Matrix

• Maintain matrix $\Pi$ where $\Pi[i][j]$ = predecessor of $j$ on shortest path from $i$.
• Initialize: $\Pi[i][j] = i$ if edge $(i,j)$ exists, else NIL.
• Update: When $D[i][j]$ is improved via $k$, set $\Pi[i][j] = \Pi[k][j]$.
• To print path $i \to j$: recursively follow $\Pi$ backwards.

Warshall's Algorithm

• Variant: determine if a path exists between any two vertices (reachability).
• Use boolean matrix instead of distance matrix.
• Recurrence: $T[i][j] = T[i][j] \lor (T[i][k] \land T[k][j])$.
• Same $O(V^3)$ structure as Floyd-Warshall.

Johnson's Algorithm (Overview)

• For sparse graphs, $O(VE \log V) < O(V^3)$.
• Step 1: Add dummy vertex $s'$ with zero-weight edges to all vertices.
• Step 2: Run Bellman-Ford from $s'$ to get potential function $h(v)$.
• Step 3: Reweight edges: $\hat{w}(u,v) = w(u,v) + h(u) - h(v) \geq 0$.
• Step 4: Run Dijkstra from every vertex on reweighted graph.
• Elegant combination of Bellman-Ford and Dijkstra.

Shortest Paths as Dynamic Programming

• Both Bellman-Ford and Floyd-Warshall are DP algorithms.
• Bellman-Ford: DP on the number of edges in the path.
• Floyd-Warshall: DP on the set of intermediate vertices.
• This is a preview of Dynamic Programming — our next topic!
• Key DP properties: optimal substructure (subpaths of shortest paths are shortest) and overlapping subproblems (many pairs share intermediate paths).

Watch Out For...

• Using Dijkstra when there are negative edges — use Bellman-Ford instead.
• Wrong loop order in Floyd-Warshall — $k$ (pivot) must be the outermost loop.
• Initializing diagonal wrong — $D[i][i] = 0$, not $\infty$.
• Forgetting to check for negative cycles — run the extra pass in Bellman-Ford.
• Using Floyd-Warshall on sparse graphs — Johnson's algorithm may be faster.

All Interactive Demos

🚀 Interactive Demo: bellman_ford_demo.html
🚀 Interactive Demo: floyd_warshall_demo.html

Lecture Summary

• Bellman-Ford: Solves SSSP with negative edges in $O(VE)$ by relaxing all edges $|V|-1$ times. One extra pass detects negative cycles.
• Floyd-Warshall: Solves APSP in $O(V^3)$ using DP on intermediate vertices. Handles negatives, detects negative cycles via $D[i][i]<0$.
• Johnson's: Combines Bellman-Ford reweighting with Dijkstra for sparse APSP in $O(VE \log V)$.
• Algorithm Choice: BFS for unweighted, Dijkstra for non-negative SSSP, Bellman-Ford for negative SSSP, Floyd-Warshall or Johnson's for APSP.
• Both Bellman-Ford and Floyd-Warshall are DP algorithms — a bridge to our next topic.

Supplementary Resources

🚀 Interactive Demo: handout.html