🖨️ Printing Instructions: Press Ctrl/Cmd + P and select "Save as PDF".

DAGs, Topological Sort, SCCs and Midterm Review

Learning Goals

Understand DAGs and their properties.
Master Topological Sort algorithms (DFS and Kahn's).
Learn about Strongly Connected Components.
Review for Midterm exam.

Directed Acyclic Graphs

DAG Definition

A Directed Graph with NO directed cycles.
Represents dependency relationships.
Example: Course prerequisites.
Example: Compilation dependencies (Makefile).
Example: Getting dressed (Socks before Shoes).

Topological Sort

Problem Statement

Input: DAG $G$.
Output: A linear ordering of vertices $v_1, v_2, \ldots, v_n$ such that:
For every edge $(u, v)$, $u$ appears before $v$ in the ordering.
If the graph has a cycle, no such ordering exists.

Example: Getting Dressed

Dependencies (edges):
Undershorts → Pants → Shoes.
Socks → Shoes.
Shirt → Belt → Jacket.
Valid Order: Socks, Undershorts, Pants, Shoes, Shirt, Belt, Jacket.
Note: Multiple valid orderings exist.

Algorithm 1: DFS-Based

Key Idea

Recall DFS finish times $f[u]$.
When we finish a node $u$, all its descendants have already finished.
If edge $u \to v$ exists, we call DFS($v$) first. $v$ finishes before $u$.
So $f[v] < f[u]$.
Higher finish time = Earlier in topological order.
Output vertices in decreasing order of finish time.

Topological-Sort(G)

1. Call DFS(G) to compute finish times f[v]
2. As each vertex finishes, prepend it to a linked list
3. Return the linked list

Complexity

DFS takes $O(V+E)$.
No explicit sorting needed.
Just prepend to a linked list as each vertex finishes.
Total time: $O(V+E)$.

Correctness Proof

Consider any edge $(u, v)$. We must show $f[v] < f[u]$.
When $(u,v)$ is explored, $u$ is Gray.
Case 1: $v$ is White → $v$ is a descendant of $u$ → $v$ finishes before $u$ → $f[v] < f[u]$. ✓
Case 2: $v$ is Black → $v$ already finished → $f[v] < f[u]$. ✓
Case 3: $v$ is Gray → Back edge → Cycle! Contradicts DAG assumption. ✗

Algorithm 2: Kahn's Algorithm

Source Removal Idea

1. Compute in-degree for all vertices.
2. Enqueue all vertices with in-degree 0 (sources).
3. While queue is not empty:
Dequeue $u$. Add $u$ to sorted list.
For each neighbor $v$ of $u$: decrement in-degree($v$).
If in-degree($v$) becomes 0, enqueue $v$.
4. If sorted list has fewer than $n$ vertices, a cycle exists.

Analysis of Kahn's

Computing in-degrees: $O(V+E)$.
Queue operations: each vertex enqueued/dequeued at most once.
Edge processing: each edge examined once during decrements.
Total: $O(V+E)$.

DFS vs. Kahn's: Which to Use?

Both run in $O(V+E)$.
DFS-based is elegant if you already have DFS infrastructure.
Kahn's is intuitive and naturally detects cycles (output size $< n$ means cycle).
Kahn's can also be adapted to find all valid topological orderings.

Interactive Demo: Topological Sort

🚀 Interactive Demo: topological_sort_demo.html

Strongly Connected Components

Definition

SCC: A maximal set of vertices $C \subseteq V$ such that for every $u, v \in C$, there is a path $u \leadsto v$ and $v \leadsto u$.
Every vertex belongs to exactly one SCC.
If you shrink each SCC to a single super-node, the resulting Component Graph is a DAG.

Kosaraju's Algorithm

1. Run DFS($G$) to compute finish times.
2. Compute $G^T$ (transpose: reverse all edges).
3. Run DFS($G^T$), processing vertices in decreasing finish time from step 1.
4. Each DFS tree in the second pass is one SCC.
Complexity: $O(V+E)$.

Interactive Demo: Strongly Connected Components

🚀 Interactive Demo: scc_demo.html

🚀 Interactive Demo: dfs_scc_demo.html

Midterm Review

Topics Covered

Asymptotic Analysis.
Recurrences.
Sorting.
Heaps.
Greedy Algorithms.
Graphs.

Review Q1: Big-O

Order the following by growth rate:
$n$, $\sqrt{n}$, $n \log n$, $n^2$, $100$
Answer: $100 \prec \sqrt{n} \prec n \prec n \log n \prec n^2$.

Review Q2: Recurrence

$T(n) = 3T(n/3) + n$.
Master Theorem: $a=3, b=3$. Watershed: $n^{\log_b a} = n^1$.
$f(n) = n = \Theta(n^{\log_b a})$.
Case 2 (tie).
Result: $\Theta(n \log n)$.

Review Q3: Loop Invariant

For Insertion Sort:
Invariant: At the start of iteration $j$, $A[0..j-1]$ contains the original first $j$ elements in sorted order.
Initialization: True for $j=1$ (one element is trivially sorted).
Maintenance: The insert step places $A[j]$ in its correct position.
Termination: When $j = n$, the entire array is sorted.

Review Q4: Sorting Stability

Which sorting algorithms are stable?
Merge Sort — Yes.
Insertion Sort — Yes.
Quicksort — No (in-place version).
Heapsort — No.

Review Q5: Heaps

Where is the 2nd largest element in a max-heap?
It must be a child of the root.
Using 1-based indexing: index 2 or 3.
Can it be at index 4? No — it would have a parent that is not the root and thus not the max.

Review Q6: Graph Traversal

BFS explores by distance (level by level).
DFS explores by depth (as deep as possible first).
On a tree: DFS corresponds to pre-order traversal.
On a tree: BFS corresponds to level-order traversal.

Review Q7: Quicksort Worst Case

When does it happen?
Already sorted or reverse-sorted input (with first/last element as pivot).
Worst-case time: $O(n^2)$.
Fix: Use randomized pivot selection → expected $O(n \log n)$.

Review Q8: Comparison Sort Lower Bound

Why is comparison-based sorting $\Omega(n \log n)$?
Decision tree must have $\geq n!$ leaves → height $\geq \log(n!) = \Omega(n \log n)$.
Does this apply to Radix Sort?
No — Radix Sort is not comparison-based, so the lower bound does not apply.

Review Q9: Graph Density

Dense graph: $|E| \approx |V|^2$.
Sparse graph: $|E| \approx |V|$.
Adjacency matrix is better for dense graphs.
Adjacency list is better for sparse graphs.

Review Q10: Pseudocode Debugging

Does this find-max code work? max = 0; for each x in A: if x > max, max = x.
Flaw: If all elements are negative, the function incorrectly returns 0.
Fix: Initialize max = $-\infty$ or max = $A[0]$.

Review Q11: DAGs

Does every DAG have a source (in-degree 0)? Yes.
Does every DAG have a unique topological ordering?
No. Example: vertices $A, B$ with no edges → both $A,B$ and $B,A$ are valid.

Review Q12: Master Theorem Gap

$T(n) = 2T(n/2) + n \log n$.
Standard Master Theorem does not apply ($f(n) = n \log n$ is not polynomially larger than $n$).
Use the recursion tree method: $\log n$ levels, cost per level grows logarithmically.
Result: $\Theta(n \log^2 n)$.

Review Q13: Heap Build

Is building a heap $O(n \log n)$ or $O(n)$?
Bottom-up Build-Heap (Floyd's method) is $O(n)$.
Top-down (repeated insertion) is $O(n \log n)$.

Review Q14: BFS with a Stack

If we replace the queue in BFS with a stack, what do we get?
DFS — the LIFO ordering causes depth-first exploration.
This is essentially an iterative implementation of DFS.

Exam Strategies

Show your work — partial credit matters.
Check boundary conditions (empty input, single element, etc.).
State the technique before diving into details (Master Theorem, loop invariant, etc.).
Don't panic. Good luck!

Common Pitfalls & Anti-Patterns

Watch Out For...

Running topological sort on a graph with cycles (undefined behavior).
Forgetting that topological order is not unique for most DAGs.
Confusing DFS finish time order (decreasing) with discovery time order.
Not detecting cycles when using Kahn's (check if output size $< n$).

Interactive Practice

Activity 1: Topological Sort (DFS)

❓ Problem:
DAG: $A \to B$, $A \to C$, $B \to D$, $C \to D$. Give a valid topological order using DFS.
Take 2 minutes to trace finish times.

Solution 1

💡 Answer:
DFS from A: Visit B → D (D finishes, B finishes), Visit C (C finishes), A finishes.
Finish times: $f[D]=1, f[B]=2, f[C]=3, f[A]=4$.
Order by decreasing finish: A, C, B, D.
(A, B, C, D is also a valid topological order.)

Activity 2: Kahn's Algorithm

❓ Problem:
Same DAG: $A \to B$, $A \to C$, $B \to D$, $C \to D$. Trace Kahn's algorithm.
Take 2 minutes to trace.

Solution 2

💡 Answer:
In-degrees: A:0, B:1, C:1, D:2. Queue = [A].
Dequeue A. Output: [A]. Decrement B→0, C→0. Queue = [B, C].
Dequeue B. Output: [A, B]. Decrement D→1. Queue = [C].
Dequeue C. Output: [A, B, C]. Decrement D→0. Queue = [D].
Dequeue D. Output: [A, B, C, D]. Done.

Activity 3: Cycle Detection

❓ Problem:
Graph: $A \to B$, $B \to C$, $C \to A$, $A \to D$. Is there a valid topological order?
Take 1 minute to analyze.

Solution 3

💡 Answer:
No! Cycle $A \to B \to C \to A$ exists.
DFS: Would discover a back edge (Gray → Gray).
Kahn's: Output size would be $< 4$ (only D has in-degree 0 after processing).

Activity 4: SCC Identification

❓ Problem:
Graph: $1 \to 2$, $2 \to 3$, $3 \to 1$, $3 \to 4$, $4 \to 5$, $5 \to 4$. How many SCCs?
Take 2 minutes to identify.

Solution 4

💡 Answer:
SCC 1: $\{1, 2, 3\}$ — cycle $1 \to 2 \to 3 \to 1$.
SCC 2: $\{4, 5\}$ — cycle $4 \to 5 \to 4$.
Total: 2 SCCs.
Component graph: $\{1,2,3\} \to \{4,5\}$ — a DAG.

Lecture Summary

DAGs model dependencies; no directed cycles allowed.
Topological Sort orders vertices so all edges go left-to-right.
DFS-based: Output in decreasing finish time. $O(V+E)$.
Kahn's Algorithm: Repeatedly remove sources. $O(V+E)$.
SCCs partition a directed graph; the component graph is always a DAG.

All Interactive Demos

🚀 Interactive Demo: topological_sort_demo.html

🚀 Interactive Demo: scc_demo.html

🚀 Interactive Demo: dfs_scc_demo.html

Supplementary Resources

🚀 Interactive Demo: worked_examples.html

🚀 Interactive Demo: reading_practice.html

🚀 Interactive Demo: handout.html