Learning Goals

• Understand Quicksort algorithm.
• Analyze best, worst, and average case complexity.
• Learn Randomized Quicksort and QuickSelect.
• Understand the $\Omega(n \log n)$ lower bound for comparison sorts.

The Quicksort Philosophy

• Divide and Conquer applied differently from Merge Sort.
• Merge Sort: Divide is easy (split in middle). Combine is hard (merge).
• Quicksort: Divide is hard (partition). Combine is easy (nothing).

Algorithm Overview

• 1. Partition: Rearrange array so small elements are on left, large on right.
• 2. Recurse: Sort left part. Sort right part.
• 3. Done: No merge needed because elements are already where they belong relative to the pivot.
Interactive Demo: quick_sort_demo.html

Quicksort(A, p, r)

if p < r
  q = Partition(A, p, r)
  Quicksort(A, p, q)
  Quicksort(A, q+1, r)

Goal of Partition

• Input: Array $A[p..r]$.
• Select a Pivot element (say $x$).
• Rearrange A so that:
• $A[p..q] \le x$
• $A[q+1..r] \ge x$
• (Pivot $x$ is not necessarily at $A[q]$)
• Return index $q$.

Lomuto Partition

• Simpler to understand/implement than Hoare.
• Pick last element $A[r]$ as pivot.
• Index $i$ marks the boundary of 'small' elements.
• Scan $j$ from $p$ to $r-1$. If $A[j] \le x$, move it to the 'small' side.

Lomuto Partition Logic

x = A[r]  // Pivot
i = p - 1
for j = p to r - 1
  if (A[j] <= x) {
    i = i + 1
    swap A[i] with A[j]
  }
swap A[i+1] with A[r]
return i + 1

Hoare Partition

• Often pick first element $A[p]$ as pivot.
• Indices $i$ and $j$ scan from outside in.
• Swap elements that are on the wrong side.
• Repeat until indices cross.
Interactive Demo: quicksort_hoare_partition.html

Hoare Partition Logic

x = A[p]  // Pivot
i = p - 1
j = r + 1
while (i < j){
  while (A[i] < x) i += 1;
  while (A[j] > x) j -= 1;
  if (i < j) {
    swap A[i] with A[j];
    i += 1;
    j -= 1;
  }
}
return j;

Example Trace

• Array: [4, 8, 7, 1, 3, 5, 2, 6]. Pivot = 4.
• j dec to 2 (Val 2 <= 4). i inc to 4 (Val 4 >= 4). Swap.
• Array: [2, 8, 7, 1, 3, 5, 4, 6].
• j dec to 4 (Val 4 <= 4). i inc to 8 (Val 8 >= 4). Swap.
• Array: [2, 4, 7, 1, 3, 5, 8, 6].
• j dec to 3 (Val 3 <= 4). i inc to 7 (Val 7 >= 4). Swap.
• Array: [2, 4, 3, 1, 7, 5, 8, 6].
• Continuing...
• j dec to 1 (Val 1 <= 4). i inc to 7 (Val 7 >= 4). Indices crossed!
• Return j=3 (index of 1).
• Split: [2, 4, 3, 1] <= 4 and [7, 5, 8, 6] >= 4.

Worst Case

• What if pivot is always the Min or Max?
• One side has 0 elements. Other side has $n-1$.
• $T(n) = T(n-1) + T(0) + \Theta(n)$.
• Sum of arithmetic series: $\Theta(n^2)$.
• Occurs when list is ALREADY sorted (with last element pivot).

Best Case

• Pivot is always the Median.
• Split is 50/50.
• $T(n) = 2T(n/2) + \Theta(n)$.
• Same as Merge Sort: $\Theta(n \log n)$.

Balanced Partitions

• What if split is 10% / 90%?
• $T(n) = T(n/10) + T(9n/10) + n$.
• Tree depth is $\log_{10/9} n$ (roughly $22 \log n$).
• Still $\Theta(n \log n)$!
• Any constant proportionality yields $O(n \log n)$.

Why Randomize?

• We cannot control the input order (malicious inputs exist).
• We CAN control the pivot choice.
• Pick a random element in $A[p..r]$ and swap with $A[r]$. Use it as pivot.
• No input can trigger worst-case behavior consistently.

Expected Runtime

• Running time is a Random Variable.
• Expected Runtime $E[T(n)] = O(n \log n)$.
• Proof involves indicator random variables and Harmonic series.
• Probability of bad split is low.

Quicksort vs Merge Sort

• Merge Sort: Guaranteed $O(n \log n)$. Stable. Requires $O(n)$ aux space.
• Quicksort: $O(n^2)$ worst case. Not Stable. Sorts in-place ($O(\log n)$ stack).
• In practice: Quicksort is faster (better cache locality, smaller constants).

The Selection Problem

• Input: Array A, number k.
• Goal: Find the $k$-th smallest element.
• Naive: Sort A, return $A[k]$. Time $O(n \log n)$.
• Can we do $O(n)$?

QuickSelect Algorithm

• Use Partition!
• Pivot lands at index $q$.
• If $q == k$: Found it!
• If $q > k$: Answer is in Left side. Recurse Left.
• If $q < k$: Answer is in Right side. Recurse Right.
• Only recurse on ONE side.

QuickSelect Analysis

• Average Case: $T(n) = T(n/2) + n$.
• Sum: $n + n/2 + n/4 + ... = 2n = O(n)$.
• We found the median in linear time!
• Worst Case: $T(n) = T(n-1) + n = O(n^2)$.

The Limit

• We have $O(n \log n)$ algorithms.
• Can we sort in $O(n)$?
• Only if we make assumptions (Counting Sort, Radix Sort).
• For COMPARISON SORTS, we cannot beat $n \log n$.

Decision Tree Model

• View algorithm as a binary tree.
• Each internal node is a comparison $a_i : a_j$.
• Left child: $a_i < a_j$. Right child: $a_i > a_j$.
• Leaves: Permutations of the input (Output).

Tree Properties

• For $n$ elements, there are $n!$ possible permutations.
• The tree must have at least $n!$ leaves.
• A binary tree of height $h$ has at most $2^h$ leaves.

The Lower Bound Proof

• $2^h \ge n!$
• $h \ge \log_2(n!)$
• Recall $\log(n!) = \Theta(n \log n)$.
• So height (worst case runtime) $h \ge \Omega(n \log n)$.

Beating the Limit

• If we don't compare elements, the lower bound doesn't apply.
• Counting Sort: Assume integers in range $1..k$.
• Use array index to count occurrences.
• $O(n + k)$.

Radix Sort

• Sort digit by digit (Least Significant first).
• Use stable Counting Sort for each digit.
• Time: $O(d(n+k))$ where $d$ is number of digits.

Activity 1: Master Case 1

• ❓ Problem:
• $T(n) = 9T(n/3) + n$.
• Take 2 minutes to solve this.

Solution 1

• 💡 Answer:
• $a=9, b=3, \log_b a = 2$. $f(n)=n = O(n^{2-\epsilon})$. $T(n)=\Theta(n^2)$.

Activity 2: Master Case 2

• ❓ Problem:
• $T(n) = T(2n/3) + 1$.
• Take 2 minutes to solve this.

Solution 2

• 💡 Answer:
• $a=1, b=3/2, \log_b a = 0$. $f(n)=1 = \Theta(n^0)$. $T(n)=\Theta(\log n)$.

Activity 3: Master Case 3

• ❓ Problem:
• $T(n) = 3T(n/4) + n \log n$.
• Take 2 minutes to solve this.

Solution 3

• 💡 Answer:
• $\log_4 3 < 1$. $f(n)$ dominates. $T(n) = \Theta(n \log n)$.

Activity 4: Substitution

• ❓ Problem:
• Guess $O(n)$ for $T(n)=2T(n/2)+n$.
• Take 2 minutes to solve this.

Solution 4

• 💡 Answer:
• Fails. Attempt $cn \log n$. Succeeds.

Activity 5: Uneven Split

• ❓ Problem:
• $T(n) = T(n/3) + T(2n/3) + n$.
• Take 2 minutes to solve this.

Solution 5

• 💡 Answer:
• Recursion tree. Depth is $\log_{3/2} n$. Cost per level $n$. $O(n \log n)$.

Activity 6: Non-Master

• ❓ Problem:
• $T(n) = 2^n T(n/2) + n^n$.
• Take 2 minutes to solve this.

Solution 6

• 💡 Answer:
• Master Theorem doesn't apply (constant 'a' required).

Interactive: Quicksort Partition

Interactive Demo: quicksort_lomuto_partition.html
Interactive Demo: quicksort_hoare_partition.html

Supplementary Resources

Interactive Demo: worked_examples.html
Interactive Demo: reading_practice.html
Interactive Demo: handout.html