🖨️ Printing Instructions: Press Ctrl/Cmd + P and select "Save as PDF".

The Greedy Paradigm

Learning Goals

Understand the Greedy paradigm.
Learn Greedy Choice Property and Optimal Substructure.
Master Activity Selection problem.
Analyze when greedy algorithms produce optimal solutions.

Introduction to Greedy Strategy

The Core Idea

Optimization Problems.
Goal: Find the Best Solution.
Greedy Strategy: At each step, make the locally optimal choice.
Hope: Local optimums lead to Global optimum.
Analogy: Hiking a mountain by always taking the steepest step up.

Pros and Cons

Pros:
Simple to design.
Fast (usually linear or log-linear).
Cons:
Often incorrect!
Hard to prove correct.

Activity Selection Problem

Interactive: Activity Selection

🚀 Interactive Demo: activity_selection_demo.html

Try the greedy algorithm yourself before we dive into theory.

Problem Statement

Given a set of $n$ activities.
Each has start time $s_i$ and finish time $f_i$.
Two activities are compatible if intervals $[s_i, f_i)$ do not overlap.
Goal: Select Maximum Number of compatible activities.

Example

A1: [1, 4)
A2: [3, 5)
A3: [0, 6)
A4: [5, 7)
A5: [3, 9)
A6: [5, 9)
A7: [6, 10)
A8: [8, 11)

Possible Greedy Choices

1. Shortest Duration first?
Counter-example: One short task [4,5] blocks two long ones [0,5] and [4,10].
2. Earliest Start Time first?
Counter-example: Task [0, 100] starts first but kills everything else.
3. Fewest Overlaps?
Also fails.

The Correct Greedy Choice

Earliest Finish Time First.
Intuition: Finishing early leaves maximum resources for subsequent tasks.
Sort activities by $f_i$.

Greedy-Activity-Selector(s, f)

Sort by finish times f[1] <= f[2] <= ... <= f[n]
A = {Activity 1}
k = 1  // Last selected index
for m = 2 to n
  if s[m] >= f[k]
    A = A U {Activity m}
    k = m
return A

Trace

Sorted: A1[1,4], A2[3,5], A4[5,7], A3[0,6]...
Pick A1 (Ends 4).
Check A2 (Starts 3): Conflict (3 < 4). Skip.
Check A4 (Starts 5): Compatible (5 >= 4). Pick A4.
Last finish is now 7.

Proof of Correctness

Greedy Choice Property

Theorem: There exists an optimal solution that includes the greedy choice (Activity 1).
Proof (Exchange Argument):
Let $S^*$ be an optimal solution.
Let $a_k$ be the first activity in $S^*$ (ordered by finish time).

Exchange Step

If $a_k = a_1$, great.
If $a_k \neq a_1$: Replace $a_k$ with $a_1$.
Is passing $a_1$ valid?
Yes, $f(a_1) \le f(a_k)$ (Greedy property).
So $a_1$ finishes even earlier, so it won't overlap with rest of $S^*$.
New solution has same size. Also optimal.

Optimal Substructure

Once we pick $a_1$, we preserve the subproblem:
Find max activities in remaining time $[f_1, \infty)$.
Combining greedy choice + optimal sub-solution yields global optimal.

Knapsack Problems

Interactive: Knapsack Comparison

🚀 Interactive Demo: knapsack_demo.html

See why greedy works for fractional but fails for 0/1.

0/1 Knapsack

Items with weight $w_i$ and value $v_i$.
Max capacity $W$.
Must take whole item or leave it.
Greedy by Value/Weight? FAILS.
Example: W=50. Item A: $60, 10kg ($6/kg). Item B: $100, 20kg ($5/kg). Item C: $120, 30kg ($4/kg).
Greedy takes A. Remaining cap 40. Can take B. Total $160.
Optimal: Take B and C. Total $220.

Fractional Knapsack

Can take fractions of items (e.g., gold dust).
Greedy by Value/Weight? WORKS.
Take as much as possible of highest density item.
Fill the bag.

Fractional Knapsack Greedy

Calculate v[i]/w[i] for all i
Sort by density descending
Total = 0
for i = 1 to n
  if w[i] <= Capacity
    Take all of i
    Capacity -= w[i]
    Total += v[i]
  else
    Take Fraction (Capacity/w[i]) of i
    Total += Fraction * v[i]
    break

Coin Change Problem

Interactive: Coin Change

🚀 Interactive Demo: coin_change_demo.html

Compare greedy vs optimal on different coin systems.

Problem

Make change for amount $N$ using fewest coins.
Denominations: $D = {d_1, d_2, ...}$.
Assuming $d_1 = 1$ (pennies) always exists.

US Currency

Coins: 25, 10, 5, 1.
Greedy works!
Example: 30 cents.
Greedy: 25, 5. (2 coins).
Optimal.

General Systems

Coins: 1, 3, 4.
Target: 6.
Greedy: Take 4. Remaining 2. Take 1, 1. Total 3 coins.
Optimal: Take 3, 3. Total 2 coins.
Greedy Fails.

Canonical Coin Systems

A coin system is 'canonical' if Greedy always works.
US coins, Powers of 2, Factorials are canonical.
For general coins, we need Dynamic Programming.

Design Principles

Properties Required

1. Greedy Choice Property: A global optimum can be arrived at by selecting a local optimum.
2. Optimal Substructure: An optimal solution to the problem contains optimal solutions to subproblems.

Contrast with DP

DP also needs Optimal Substructure.
But DP considers ALL choices (recursively).
Greedy considers ONLY ONE choice.

Example: Minimum Spanning Tree

Preview

Prim's Algorithm and Kruskal's Algorithm.
Both are Greedy.
Both work.
Why? Matroid Theory explains this (Advanced topic).

Practice Problems

Problem 1

Suppose activities have weights (importance).
Does Greedy (finish time) work?
No.
Need DP (Weighted Interval Scheduling).

Problem 2

Gas Station Problem.
Travel from A to B. Tank capacity C.
Minimize stops.
Greedy Strategy: Go as far as possible, then stop at last station reachable.
Correct? Yes.

Common Pitfalls & Anti-Patterns

Watch Out For...

Assuming greedy always works without proof.
Confusing 0/1 Knapsack (needs DP) with Fractional Knapsack (greedy works).
Forgetting to sort by the correct criterion.
Not proving greedy choice property before implementing.

Interactive Practice

Activity 1: Greedy Choice

❓ Problem:
For Activity Selection, why does 'shortest duration first' fail?
Take 2 minutes to think of a counter-example.

Solution 1

💡 Answer:
A short activity in the middle can block two longer non-overlapping ones.
Example: [0,5], [4,5], [4,10]. Picking [4,5] blocks both others.

Activity 2: Knapsack Analysis

❓ Problem:
W=50, Items: A(10kg, $60), B(20kg, $100), C(30kg, $120).
What does greedy by $/kg select for 0/1 Knapsack? Is it optimal?
Take 2 minutes to solve this.

Solution 2

💡 Answer:
Greedy: A ($6/kg) + B ($5/kg) = $160, weight 30kg.
Optimal: B + C = $220, weight 50kg.
Greedy fails because taking A wastes capacity.

Activity 3: Coin Change

❓ Problem:
Coins: {1, 3, 4}. Make change for 6.
What does greedy give? What's optimal?
Take 2 minutes to solve this.

Solution 3

💡 Answer:
Greedy: 4 + 1 + 1 = 3 coins.
Optimal: 3 + 3 = 2 coins.
This coin system is not canonical.

Lecture Summary

Greedy Strategy: Make locally optimal choices at each step.
Greedy Choice Property: Must prove a global optimum includes the greedy choice.
Optimal Substructure: Problem has optimal solutions to subproblems.
Activity Selection: Earliest finish time first works.
Knapsack: Greedy works for fractional, fails for 0/1.
Coin Change: Greedy works for canonical systems only.

All Interactive Demos

📊 Algorithm Visualizations:

🚀 Interactive Demo: activity_selection_demo.html

🚀 Interactive Demo: knapsack_demo.html

🚀 Interactive Demo: coin_change_demo.html

Click any demo above to practice interactively.

Supplementary Resources

🚀 Interactive Demo: worked_examples.html

🚀 Interactive Demo: reading_practice.html

🚀 Interactive Demo: handout.html