🖨️ Printing Instructions: Press Ctrl/Cmd + P and select "Save as PDF".
1
Course Review
Mastering Algorithms, Analysis, and Complexity
2
Learning Outcomes Review
- 1. Model algorithmic solutions and compare them.
- 2. Formulate time order analysis and assess improvements.
- 3. Apply Greedy approaches and assess accuracy.
- 4. Apply Divide and Conquer and analyze correctness.
- 5. Apply Dynamic Programming and establish correctness.
- 6. Describe classical algorithms (Sorting, Graphs).
- 7. Describe P and NP complexity classes.
3
1. Foundations & Analysis (LO 1, 2)
4
Asymptotic Notation
- O-notation (Upper Bound): $f(n) = O(g(n))$ if $f(n) \le c \cdot g(n)$ for large $n$.
- Ω-notation (Lower Bound): $f(n) = \Omega(g(n))$ if $f(n) \ge c \cdot g(n)$.
- Θ-notation (Tight Bound): Both upper and lower bounds match.
- Crucial Goal: Compare algorithm growth rates independent of hardware.
5
Recurrences & Master Theorem
- Standard D&C Recurrence: $T(n) = aT(n/b) + f(n)$.
- Master Theorem Cases:
- 1. $f(n) = O(n^{\log_b a - \epsilon}) \implies T(n) = \Theta(n^{\log_b a})$.
- 2. $f(n) = \Theta(n^{\log_b a}) \implies T(n) = \Theta(n^{\log_b a} \lg n)$.
- 3. $f(n) = \Omega(n^{\log_b a + \epsilon}) \implies T(n) = \Theta(f(n))$.
- Always check regular conditions for Case 3.
6
2. Divide and Conquer (LO 4, 6)
7
Key Paradigms
- Step 1: Divide into smaller subproblems.
- Step 2: Conquer via recursive calls.
- Step 3: Combine results.
- Classical Examples:
- Merge Sort ($n \lg n$), Quicksort ($n^2$ worst, $n \lg n$ avg), Binary Search ($\lg n$).
8
3. Sorting & Order Statistics (LO 6)
9
Sorting Landscape
- Comparison-Based: Lower bound $\Omega(n \lg n)$.
- - Heapsort: In-place, $O(n \lg n)$ worst-case.
- - Quicksort: Fast in practice, randomized avoids worst-case.
- Linear Time Sorts: Non-comparison-based (integers).
- - Counting Sort ($O(k+n)$), Radix Sort ($O(d(n+k))$).
- Decision Trees: Proof of the $n \lg n$ lower bound.
10
4. Greedy Algorithms (LO 3, 6)
11
The Greedy Strategy
- Make the locally optimal choice at each step.
- Two Properties Needed:
- 1. Greedy Choice Property: Local optimal leads to global optimal.
- 2. Optimal Substructure: Solution contains optimal solutions to subproblems.
- Key Examples:
- Activity Selection, Huffman Encoding, MST (Prim/Kruskal).
12
5. Dynamic Programming (LO 5, 6)
13
DP vs. Greedy
- DP solves problems with Overlapping Subproblems.
- Stores results in a table (Memoization or Tabulation).
- 4-Step Recipe:
- 1. Characterize optimal solution structure.
- 2. Define value of optimal solution recursively.
- 3. Compute values (bottom-up/top-down).
- 4. Reconstruct solution from computed info.
14
Classic DP Problems
- Rod Cutting: $r_n = \max(p_i + r_{n-i})$.
- 0/1 Knapsack: Use table $c[i, w]$.
- Longest Common Subsequence (LCS).
- Floyd-Warshall: All-pairs shortest path $O(V^3)$.
15
6. Graph Algorithms (LO 6)
16
Traversals & MST
- BFS: Queue-based, finds shortest path in unweighted graphs.
- DFS: Stack/Rec-based, finds components, topological sort.
- MST: Kruskal (Edge-based, DSU), Prim (Node-based, Priority Queue).
- Cut Property ensures edge inclusion.
17
Shortest Paths
- Dijkstra: Greedy, $O(E \lg V)$, non-negative weights only.
- Bellman-Ford: DP-ish, $O(VE)$, handles negative weights, detects cycles.
- Floyd-Warshall: All-pairs shortest path $O(V^3)$
18
7. Complexity Theory (LO 7)
19
P, NP, and Reductions
- Class P: Solvable in polynomial time.
- Class NP: Verifiable in polynomial time.
- Class NP-Complete: Hardest problems in NP. $L \in NPC$ if $L \in NP$ and every $L' \in NP$ reduces to $L$.
- Reductions ($A \le_p B$): If B is solvable in poly-time, then A is too.
21
Which Paradigm to Use?
- D&C: Independent subproblems (Sorting).
- DP: Overlapping subproblems, optimization (LCS).
- Greedy: No need to look ahead, local best is enough (MST).
- NPC: If problem is equivalent to SAT, use heuristics or approximations.
22
Course Wrap-Up
- Algorithms are the core of computer science.
- Tradeoffs: Time vs. Space vs. Correctness (Approximations).
- Always start by modeling the problem formally.
- Good luck on the final exam!
23
Supplementary Review Resources