🖨️ Printing Instructions: Press Ctrl/Cmd + P and select "Save as PDF".

Divide & Conquer and Recurrences

Learning Goals

The D&C Paradigm: Divide, Conquer, Combine.
Merge Sort Analysis.
Recursion Trees.

The Divide & Conquer Paradigm

🎬 See It First: Merge Sort

Interactive Demo: merge_sort_complete.html

Watch the full divide-and-conquer process before we explain the theory.

Strategy

1. Divide: Break problem into subproblems of the same type.
2. Conquer: Solve subproblems recursively.
If small enough, solve directly (Base Case).
3. Combine: Merge sub-solutions into the final solution.

Example: Merge Sort

Divide: Split array $A[p..r]$ into two halves.
Conquer: Recursively sort both halves.
Combine: Merge two sorted lists into one.
Key: The work is mostly in the Combine step.

Merge Sort Algorithm

MergeSort(A, p, r)
  if p < r
    q = floor((p+r)/2)
    MergeSort(A, p, q)
    MergeSort(A, q+1, r)
    Merge(A, p, q, r)

Analysis of Merge Sort

Let $T(n)$ be the runtime.
Divide: $D(n) = O(1)$ (calculate midpoint).
Conquer: $2 T(n/2)$ (two subproblems).
Combine: $C(n) = \Theta(n)$ (linear merge).
Total: $T(n) = 2T(n/2) + \Theta(n)$.

Solving Recurrences

What is a Recurrence?

An equation defining a function in terms of itself.
Example: Fibonacci $F(n) = F(n-1) + F(n-2)$.
Algorithm Example: $T(n) = 2T(n/2) + n$.

Three Methods

1. Recursion Tree Method: Visualize the cost per level.
2. Substitution Method: Guess and Verify (Induction).
3. Master Theorem: "Cookbook" for standard forms.

Method 1: Recursion Tree

🎬 Try It First: Build a Recursion Tree

Interactive Demo: recursion_tree_demo.html

Build and explore recursion trees interactively before we analyze them.

Visualizing Merge Sort

Root: Input size $n$. Cost $cn$.
Level 1: 2 nodes size $n/2$. Cost $2 \cdot (cn/2) = cn$.
Level 2: 4 nodes size $n/4$. Cost $4 \cdot (cn/4) = cn$.
It seems every level costs $cn$.

Tree Height

We divide $n$ by 2 at each step.
When does $n / 2^h = 1$?
When $h = \log_2 n$.
So there are $\log n + 1$ levels.

Total Cost

Total = (Cost per Level) $\times$ (Number of Levels).
Total = $cn \times (\log n + 1)$.
$$T(n) = cn \log n + cn = \Theta(n \log n)$$

Example 2: Bad Split

$T(n) = T(n/3) + T(2n/3) + n$.
Tree is not balanced.
Shortest path to leaf: $\log_3 n$.
Longest path to leaf: $\log_{3/2} n$.
But sum at each level is still $n$.
So still $\Theta(n \log n)$.

Method 2: Substitution

🎬 See It First: Substitution Proof

Interactive Demo: substitution_method_demo.html

Watch the proof unfold line-by-line like a whiteboard.

Steps

1. Guess the form of the solution (e.g., $O(n \log n)$).
2. Use induction to find constants and prove it.
We assume $T(k) \le c k \log k$ for $k < n$.

Substitution Proof

$T(n) = 2T(n/2) + n$.
Assume $T(n/2) \le c (n/2) \log (n/2)$.
$T(n) \le 2 [ c (n/2) \log(n/2) ] + n$.
$T(n) \le cn (\log n - 1) + n$.

Substitution Proof (Continued)

$T(n) \le cn \log n - cn + n$.
We want $T(n) \le cn \log n$.
Need $-cn + n \le 0$.
$n \le cn \implies c \ge 1$.
It works!

Method 3: The Master Theorem

🎬 See Why It Works

Interactive Demo: master_theorem_visual.html

Interactive Demo: master_theorem_demo.html

Interactive Demo: master_theorem_proof.html

Interactive Demo: master_theorem_proof_math.html

Visualize WHY each case works, then try the calculator.

Standard Form

$$T(n) = a T(n/b) + f(n)$$
$a \ge 1$: Number of subproblems.
$b > 1$: Shrink factor.
$f(n)$: Cost of divide/combine.

Compare f(n) vs Watersheld

We compare $f(n)$ with $n^{\log_b a}$.
$n^{\log_b a}$ is the number of leaves in the recursion tree.
Basically: Who wins? Top of tree (root) or Bottom (leaves)?

Case 1: Leaves Win

If $f(n) = O(n^{\log_b a - \epsilon})$ for some $\epsilon > 0$.
Then $T(n) = \Theta(n^{\log_b a})$.
Meaning: The work at leaves dominates.

Case 2: Tie

If $f(n) = \Theta(n^{\log_b a})$.
Then $T(n) = \Theta(n^{\log_b a} \log n)$.
Meaning: Every level has same cost. Multiply by height.

Case 3: Root Wins

If $f(n) = \Omega(n^{\log_b a + \epsilon})$.
AND regularity condition holds ($a f(n/b) \le c f(n)$).
Then $T(n) = \Theta(f(n))$.
Meaning: The work at the root dominates everything.

Master Theorem Examples

Ex 1: Merge Sort

$T(n) = 2T(n/2) + n$.
$a=2, b=2$.
Watershed: $n^{\log_2 2} = n^1 = n$.
$f(n) = n$.
Compare $n$ vs $n$. It is a TIE.
Case 2: $T(n) = \Theta(n \log n)$.

Ex 2: Binary Search

$T(n) = T(n/2) + 1$.
$a=1, b=2$.
Watershed: $n^{\log_2 1} = n^0 = 1$.
$f(n) = 1$.
Compare $1$ vs $1$. TIE.
Case 2: $T(n) = \Theta(1 \cdot \log n) = \Theta(\log n)$.

Ex 3: Matrix Multiplication (Strassen)

Original: 8 multiplications. $T(n) = 8T(n/2) + n^2$.
$n^{\log_2 8} = n^3$. $f(n)=n^2$. Case 1 (Leaves win). $\Theta(n^3)$.
Strassen: 7 multiplications. $T(n) = 7T(n/2) + n^2$.
$n^{\log_2 7} \approx n^{2.81}$.
$f(n) = n^2 = O(n^{2.81})$. Case 1.
Result: $\Theta(n^{2.81})$.

Ex 4: Root Heavy

$T(n) = 3T(n/4) + n \log n$.
$a=3, b=4$. Watershed $n^{\log_4 3} \approx n^{0.79}$.
$f(n) = n \log n$.
$n^{0.79}$ vs $n \log n$. $f(n)$ is much bigger.
Case 3 (Root wins).
$T(n) = \Theta(n \log n)$.

When Master Theorem Fail

Gaps

The cases require polynomial difference.
Consider $T(n) = 2T(n/2) + n \log n$.
Watershed $n$. $f(n) = n \log n$.
Ratio is $\log n$. This is NOT $n^\epsilon$.
Falls into the gap between Case 2 and 3.
Must use recursion tree (Answer: $\Theta(n \log^2 n)$).

Other Examples

Maximum Subarray Problem

Problem: Find contiguous subarray with largest sum.
Naive: $\Theta(n^2)$.
D&C approach:
Max is in Left half, Right half, or Crossing middle.
$T(n) = 2T(n/2) + \Theta(n)$.
Result: $\Theta(n \log n)$.

Integer Multiplication (Karatsuba)

Multiply two $n$-digit numbers.
Grade school: $\Theta(n^2)$.
Karatsuba: 3 mults instead of 4.
$T(n) = 3T(n/2) + \Theta(n)$.
Result: $\Theta(n^{\log_2 3}) \approx \Theta(n^{1.585})$.

Practice Problems

Practice 1

$T(n) = 4T(n/2) + n$.
a=4, b=2. $n^{\log_2 4} = n^2$.
$f(n)=n$.
Case 1.
$\Theta(n^2)$.

Practice 2

$T(n) = 4T(n/2) + n^2$.
a=4, b=2. $n^2$.
$f(n)=n^2$.
Case 2 (Tie).
$\Theta(n^2 \log n)$.

Practice 3

$T(n) = 4T(n/2) + n^3$.
a=4, b=2. $n^2$.
$f(n)=n^3$.
Case 3 (Root wins).
$\Theta(n^3)$.

Common Pitfalls & Anti-Patterns

Watch Out For...

Forgetting the 'Combine' step cost.
Assuming all recursions are O(n log n).

Interactive Practice

Activity 1: Merge Sort Recurrence

❓ Problem:
Solve $T(n) = 2T(n/2) + n$.
Take 2 minutes to solve this.

Solution 1

💡 Answer:
$O(n \log n)$.

Activity 2: Binary Search Recurrence

❓ Problem:
Solve $T(n) = T(n/2) + 1$.
Take 2 minutes to solve this.

Solution 2

💡 Answer:
$O(\log n)$.

Activity 3: Bad Split

❓ Problem:
Solve $T(n) = T(n-1) + n$ (e.g. Selection Sort).
Take 2 minutes to solve this.

Solution 3

💡 Answer:
$O(n^2)$.

Activity 4: Ternary Search

❓ Problem:
Split into 3 parts, look in 1. $T(n) = T(n/3) + 1$.
Take 2 minutes to solve this.

Solution 4

💡 Answer:
$O(\log_3 n) = O(\log n)$.

Activity 5: Max Element D&C

❓ Problem:
Find max of array. $T(n)=2T(n/2)+1$.
Take 2 minutes to solve this.

Solution 5

💡 Answer:
$O(n)$. No improvement over iterative.

Activity 6: Stooge Sort

❓ Problem:
Weird sort. $T(n) = 3T(2n/3) + 1$.
Take 2 minutes to solve this.

Solution 6

💡 Answer:
Apply Master Theorem (Case 1).

Interactive: Merge Sort

Interactive Demo: merge_sort_complete.html

Complete visualization with tree view, step-by-step merge, and pseudocode.

Interactive: Master Theorem

Interactive Demo: master_theorem_demo.html

Interactive Demo: master_theorem_visual.html

Interactive Demo: master_theorem_proof.html

Interactive Demo: master_theorem_proof_math.html

Calculator + Visual proof showing WHY each case works.

Interactive: Recursion Tree

Interactive Demo: recursion_tree_demo.html

Build and analyze recursion trees step by step.

Interactive: Substitution Method

Interactive Demo: substitution_method_demo.html

Whiteboard-style line-by-line proof walkthrough.

All Interactive Demos

📊 Algorithm Visualizations:

Interactive Demo: merge_sort_complete.html

Interactive Demo: recursion_tree_demo.html

📐 Proof & Analysis Tools:

Interactive Demo: substitution_method_demo.html

Interactive Demo: master_theorem_demo.html

Interactive Demo: master_theorem_visual.html

Interactive Demo: master_theorem_proof.html

Interactive Demo: master_theorem_proof_math.html

Supplementary Resources

Interactive Demo: worked_examples.html

Interactive Demo: reading_practice.html

Interactive Demo: handout.html