🖨️ Printing Instructions: Press Ctrl/Cmd + P and select "Save as PDF".

Heaps, Heapsort, and Priority Queues

Learning Goals

Understand heap property (min-heap and max-heap).
Master array representation of heaps.
Learn Heapify and BuildHeap operations.
Analyze Heapsort and Priority Queue operations.

Background Review: Trees & Complexity

What is a Tree?

A Tree is a hierarchical data structure.
Consists of nodes connected by edges.
Has a single root node at the top.
Each node can have zero or more children.
Nodes with no children are called leaves.
No cycles: exactly one path between any two nodes.

Tree Terminology

Root: The topmost node (has no parent).
Parent: The node directly above a given node.
Child: A node directly below a given node.
Siblings: Nodes sharing the same parent.
Leaf: A node with no children.
Depth: Distance (edges) from root to a node.
Height: Maximum depth among all nodes.

What is a Binary Tree?

A Binary Tree is a tree where each node has at most 2 children.
Children are called left child and right child.
Very common in CS: search trees, expression trees, heaps.
Simple navigation: go left or go right.
Many operations are $O(height)$ — we want short trees!

Types of Binary Trees

Full Binary Tree: Every node has 0 or 2 children (never just 1).
Perfect Binary Tree: All levels completely filled.
Complete Binary Tree: Filled level by level, left to right.
In a complete tree, the last level may be incomplete but has no gaps.
Complete trees are essential for heaps!

Why Complete Binary Trees Matter

Guaranteed balanced: Height is always $\lfloor \log_2 n \rfloor$.
No wasted space when stored in an array.
Shape is predictable — only one possible shape for $n$ nodes.
Example: 10 nodes → height 3 (levels 0,1,2,3).
This makes heaps very efficient!

Quick Review: Big-O Notation

Describes how runtime grows with input size $n$.
$O(1)$: Constant — same time regardless of $n$.
$O(\log n)$: Logarithmic — very fast (halving each step).
$O(n)$: Linear — proportional to input size.
$O(n \log n)$: Log-linear — efficient sorting algorithms.
$O(n^2)$: Quadratic — nested loops, gets slow fast.

Why $\log n$ Appears Often

When we halve the problem size each step: $n \to n/2 \to n/4 \to \ldots \to 1$.
Number of halvings to reach 1 is $\log_2 n$.
Tree height of $n$ nodes (complete binary tree) = $\lfloor \log_2 n \rfloor$.
Operations that traverse root-to-leaf are $O(\log n)$.
$n = 1000 \Rightarrow \log_2 n \approx 10$. Very efficient!

The Heap Data Structure

What is a Heap?

A Heap is a generic term for a data structure that implements a Priority Queue.
We focus on the Binary Heap.
It is a Complete Binary Tree.
Satisfies the Heap Property.

Binary Tree Concepts

Complete: Filled from top to bottom, left to right. No gaps.
Height is $\lfloor \log_2 n \rfloor$.
Key advantage: Can be stored in an Array.

Array Representation (1-indexed)

Textbook convention: Root at index 1.
Left Child of $i$: $2i$
Right Child of $i$: $2i + 1$
Parent of $i$: $\lfloor i/2 \rfloor$
Index 0 is unused (wasted space).
Advantage: Clean formulas, easy bit operations.

Array Representation (0-indexed)

Programming convention: Root at index 0.
Left Child of $i$: $2i + 1$
Right Child of $i$: $2i + 2$
Parent of $i$: $\lfloor (i-1)/2 \rfloor$
No wasted space.
This is what you'll use in actual code (C, Java, Python).

Why Two Conventions?

1-indexed: Cleaner math, used in textbooks (CLRS).
- Bit tricks: left = $i << 1$, right = $(i << 1) | 1$, parent = $i >> 1$
0-indexed: Matches real programming languages.
- No wasted array slot.
Both are correct! Just be consistent.
We'll use 1-indexed for teaching, note differences for coding.

Array Representation: Visual Example

Tree: Root=16, Left=14, Right=10, etc.
1-indexed: [_, 16, 14, 10, 8, 7, 9, 3, 2, 4, 1]
0-indexed: [16, 14, 10, 8, 7, 9, 3, 2, 4, 1]
Node 14: 1-indexed at 2, 0-indexed at 1.
Same tree, different index arithmetic!
Simple arithmetic replaces pointer chasing.

Index Formulas: Quick Reference

| Operation | 1-indexed | 0-indexed |
| Left Child | $2i$ | $2i + 1$ |
| Right Child | $2i + 1$ | $2i + 2$ |
| Parent | $\lfloor i/2 \rfloor$ | $\lfloor (i-1)/2 \rfloor$ |
| Is Leaf? | $i > \lfloor n/2 \rfloor$ | $i \geq \lfloor n/2 \rfloor$ |

Heap Property

Max-Heap Property: For every node $i$, $A[parent(i)] \ge A[i]$.
Largest element is at the Root.
Min-Heap Property: $A[parent(i)] \le A[i]$.
Smallest element is at the Root.

Heap vs. Sorted Array

Heap is partially ordered, not fully sorted.
We only know parent ≥ children (max-heap).
We do NOT know left child vs. right child ordering.
This weaker constraint allows $O(\log n)$ insert/delete.
Sorted array: $O(n)$ insert, $O(1)$ find-max.

Maintaining the Heap

Quick Review: Recursion

A function that calls itself to solve smaller subproblems.
Needs a base case to stop (e.g., reaching a leaf).
Recursive case: solve smaller version, combine results.
Heapify is recursive: fix current node, then fix affected subtree.
Alternative: can also be written iteratively with a loop.

Max-Heapify

Assume Left and Right subtrees are heaps.
But Root might violate the property (too small).
Goal: "Float down" the root to its correct place.
Time: $O(\log n)$ (Height of tree).

Max-Heapify: Intuition

Compare node with its two children.
Find the largest among node, left child, right child.
If node is not largest, swap with the largest child.
Recursively fix the subtree where we swapped.
Stop when node is larger than both children (or is a leaf).

Max-Heapify(A, i) — 1-indexed

l = 2*i          // left child
r = 2*i + 1      // right child
largest = i
if l <= heap_size and A[l] > A[largest]
  largest = l
if r <= heap_size and A[r] > A[largest]
  largest = r
if largest != i
  swap A[i] with A[largest]
  Max-Heapify(A, largest)

Max-Heapify(A, i) — 0-indexed

l = 2*i + 1      // left child
r = 2*i + 2      // right child
largest = i
if l < heap_size and A[l] > A[largest]
  largest = l
if r < heap_size and A[r] > A[largest]
  largest = r
if largest != i
  swap A[i] with A[largest]
  Max-Heapify(A, largest)

Heapify Trace

Node 4. Children 14, 10.
14 is > 4. Swap.
Now 4 is at index 2 (Left child).
Children of new pos (index 2)? Say 8, 7.
8 > 4. Swap.
Leaf reached. Done.

Building a Heap

Build-Max-Heap

Given an unsorted array.
Convert it into a Max-Heap.
Strategy: Call Heapify from bottom up.
Leaves are already heaps (size 1). Start at parent of last leaf.

Why Bottom-Up?

Heapify assumes children are already heaps.
Leaves (no children) trivially satisfy heap property.
Process internal nodes from bottom to top.
When we reach a node, its subtrees are already fixed!
Start index: $\lfloor n/2 \rfloor$ (last internal node).

Build-Max-Heap(A) — 1-indexed

heap_size = A.length
for i = floor(A.length / 2) down to 1
  Max-Heapify(A, i)

Build-Max-Heap(A) — 0-indexed

heap_size = A.length
for i = floor(A.length / 2) - 1 down to 0
  Max-Heapify(A, i)

Complexity of Build-Heap

Simple bound: $n$ calls times $O(\log n) = O(n \log n)$.
Correct bound: Linear Time $O(n)$.
Why? Most nodes are near leaves (height 0).
Few nodes are near root (log n work).

Proof of Linear Build Time

Cost = $\sum_{h=0}^{\log n} \lceil \frac{n}{2^{h+1}} \rceil O(h)$.
Sum of $\frac{h}{2^h}$ converges to constant 2.
Total cost $O(n)$.

Heapsort Algorithm

Sorting: The Goal

Given $n$ elements, arrange them in order.
Comparison-based sorting: lower bound $\Omega(n \log n)$.
Heapsort achieves this bound: $O(n \log n)$ worst case.
Key insight: Max element is always at root of max-heap.

Heapsort Strategy

1. Build Max Heap.
2. Swap Root (Max) with Last Element.
3. Decrement Heap Size (Max is "frozen" in sorted position).
4. Fix the new root (Heapify).
5. Repeat.

Heapsort(A) — 1-indexed

Build-Max-Heap(A)
for i = A.length down to 2
  swap A[1] with A[i]
  heap_size = heap_size - 1
  Max-Heapify(A, 1)

Heapsort(A) — 0-indexed

Build-Max-Heap(A)
for i = A.length - 1 down to 1
  swap A[0] with A[i]
  heap_size = heap_size - 1
  Max-Heapify(A, 0)

Heapsort Analysis

Build Heap: $O(n)$.
Loop runs $n-1$ times.
Inside loop: Swap $O(1)$, Heapify $O(\log n)$.
Total: $O(n + n \log n) = O(n \log n)$.

Comparison

Like Merge Sort: Guaranteed $O(n \log n)$.
Like Insertion Sort: In-Place.
Not Stable.
Generally slower than Quicksort (poor locality, jumping indices).

Priority Queues

Review: What is a Queue?

A Queue is a First-In-First-Out (FIFO) data structure.
Like a line at a store: first in line is served first.
Enqueue: Add element to the back.
Dequeue: Remove element from the front.
Fair, but what if some items are more urgent?

Motivation: Why Priority Queues?

Sometimes we process by importance, not arrival order.
Hospital ER: Critical patients first, not first-come-first-served.
CPU scheduling: High-priority processes run first.
Airline boarding: First class before economy.
We need quick access to the "most important" item.

Priority Queue: The Concept

Each element has an associated priority (a key).
Highest priority element is served first.
Not FIFO — order depends on priority values.
Can implement with: unsorted list, sorted list, or heap.
Heap gives best balance: $O(\log n)$ insert and extract.

Implementation Comparison

Unsorted Array: Insert $O(1)$, Extract-Max $O(n)$.
Sorted Array: Insert $O(n)$, Extract-Max $O(1)$.
Heap: Insert $O(\log n)$, Extract-Max $O(\log n)$.
Heap is optimal when both operations are frequent!
This is why heaps are the standard PQ implementation.

The Interface

A Data Structure maintaining a set of elements with keys.
Insert(S, x): Add element x.
Maximum(S): Return max.
Extract-Max(S): Remove and return max.
Increase-Key(S, x, k): Update key of x to k.

Implementing with Heap

Maximum: Return A[1]. $O(1)$.
Extract-Max: Save A[1], swap with last, discard last, Heapify root. $O(\log n)$.
Insert: Add to end, "Float Up" (decrease key logic reversed) towards root. $O(\log n)$.

Insert: Float Up

Add new element at the end of the array (next available spot).
This maintains the complete tree shape.
New element might be larger than its parent → violates heap property.
Float up: Compare with parent, swap if larger, repeat.
Stop when parent is larger or we reach the root.

Increase-Key Logic

Change value at index $i$.
If new value > parent, swap with parent.
Repeat until property restored.
This is the inverse of Heapify.

Applications

Job Scheduling (OS).
Event Simulation (Next event time).
Dijkstra's Algorithm (Shortest Path).
Prim's Algorithm (MST).
Huffman Coding.

Summary: Key Takeaways

What We Learned

Heap = Complete binary tree + heap property.
Array representation: two conventions (0-indexed vs 1-indexed).
Heapify: Fix one violation in $O(\log n)$.
Build-Heap: Create heap from array in $O(n)$.
Heapsort: $O(n \log n)$ in-place sorting.
Priority Queue: Efficient insert/extract with heaps.

Practice Problems

Problem 1

Where is the smallest element in a Max-Heap?
It must be at a Leaf.
Which leaves? Any of them.
To find it, we must scan all leaves: $O(n)$.

Problem 2

Is a sorted array a Heap?
Ascending: [1, 2, 3]. Min-Heap? Yes. Max-Heap? No.
Descending: [3, 2, 1]. Max-Heap? Yes.

Problem 3

Height of a node.
Leaves have height 0.
Root has height $\lfloor \log n \rfloor$.

Problem 4: K-ary Heaps

What if use 3 children?
Tree is shallower: $\log_3 n$.
Push down (Heapify) is slower: 3 comparisons per level.
Good for applications with many insertions?

Interactive: Binary Heap

Interactive Demo: heap_demo.html

Supplementary Resources

Interactive Demo: worked_examples.html

Interactive Demo: reading_practice.html

Interactive Demo: handout.html