🖨️ Printing Instructions: Press Ctrl/Cmd + P and select "Save as PDF".

Correctness & Loop Invariants

Learning Goals

Understand Pre/Post conditions.
Master Loop Invariants.
Prove Insertion Sort correctness.

Introduction to Correctness

Does it work?

We learned how to analyze SPEED (Big-O).
But a fast algorithm that gives the WRONG answer is useless.
How do we PROVE an algorithm always works?
Testing is not enough. (Dijkstra: "Testing shows the presence, not the absence of bugs").

Formal Verification

We treat an algorithm like a mathematical theorem.
Input: Assumptions (Preconditions).
Output: Conclusion (Postconditions).
Algorithm: The Proof.

Preconditions vs Postconditions

Precondition: What must be true BEFORE calling the function.
Example: "Array A must be sorted."
Postcondition: What will be true AFTER the function returns.
Example: "Returns index of x or -1."

Contracts

If the Caller satisfies the Precondition...
The Callee (Function) promises to satisfy the Postcondition.
If Precondition is violated (e.g., passing null), behavior is undefined.

Loop Invariants

The Loop Invariant Concept

Interactive Demo: loop_invariant_intro.html

A statement $P$ that is true at the start of every iteration of the loop.
It acts like an induction hypothesis.
Critical for proving loops correct.

Three Properties

1. Initialization: It is true before the first iteration.
2. Maintenance: If it is true before iteration $i$, it remains true before iteration $i+1$.
3. Termination: When the loop ends, the invariant gives us the correct answer.

Example 1: Summing an Array

function Sum(A[0..n-1])
  sum = 0
  i = 0
  while i < n:
    sum = sum + A[i]
    i = i + 1
  return sum

Defining the Invariant

Interactive Demo: sum_invariant_demo.html

What have we done so far at step $i$?
Invariant: At line 4 (while loop check), variable `sum` holds the sum of the subarray $A[0..i-1]$.

Proof: Initialization

Before first loop:
$i=0$. `sum` = 0.
Subarray $A[0..-1]$ is empty.
Sum of empty set is 0.
Statement is TRUE.

Proof: Maintenance

Assume invariant holds for $i$ (`sum` = $\sum_{k=0}^{i-1} A[k]$).
Body of loop:
`sum_new` = `sum` + $A[i]$.
`i_new` = $i + 1$.
After body: `sum_new` = $(\sum_{k=0}^{i-1} A[k]) + A[i]$ = $\sum_{k=0}^{i} A[k]$.
This matches $A[0 .. i_{new}-1]$.
Statement is TRUE.

Proof: Termination

Loop ends when: $i < n$ is false. So $i = n$.
Invariant says: `sum` holds sum of $A[0..n-1]$.
Conclusion: This is exactly the sum of the entire array.
Algorithm is CORRECT.

Example 2: Finding Maximum

function FindMax(A[0..n-1])
  max_val = -infinity
  for i = 0 to n-1:
    if A[i] > max_val:
      max_val = A[i]
  return max_val

Invariant for Max

Interactive Demo: max_invariant_demo.html

Invariant: At the start of iteration $i$, `max_val` contains the maximum value in $A[0..i-1]$.

Initialization

$i=0$. Subarray empty.
Max of empty set is $-\infty$ (identity element).
True.

Maintenance

Assume `max_val` is max of $A[0..i-1]$.
We inspect $A[i]$.
If $A[i] > max\_val$, we update.
New `max_val` is $\max(old\_max, A[i])$.
This is exactly the max of $A[0..i]$.
True.

Termination

Variable $i$ goes up to $n$.
Invariant says `max_val` is max of $A[0..n-1]$.
Returns correct answer.

Example 3: Insertion Sort

Insertion Sort Concept

Like sorting cards in your hand.
Left hand holds sorted cards.
Right hand picks next card and inserts it into correct spot.

for j = 1 to n-1:
  key = A[j]
  i = j - 1
  while i >= 0 and A[i] > key:
    A[i+1] = A[i]
    i = i - 1
  A[i+1] = key

Outer Loop Invariant

Invariant: At start of iteration $j$, the subarray $A[0..j-1]$ consists of the elements originally in $A[0..j-1]$ but in sorted order.

Initialization

$j=1$. Subarray is $A[0..0]$ (Single element).
A single element is always sorted.
True.

Maintenance Step (Tricky)

We need to show that inserting $A[j]$ preserves the sorted property.
The Inner While Loop shifts elements larger than `key` to the right.
It opens a hole at correct position $i+1$.
We place `key` there.
Now $A[0..j]$ is sorted.

Termination

Loop ends when $j=n$.
Invariant says $A[0..n-1]$ is sorted.
Done!

Interactive: Insertion Sort

Interactive Demo: insertion_sort_invariant.html

Inner Loop Correctness

Proving the Inner Loop

The inner loop also has an invariant!
Invariant: Elements $A[i+1..j]$ are greater than `key`.
And they are sorted relative to each other (shifted versions of original sorted array).

Correctness of Recursion

Mathematical Induction

To prove recursive function $F(n)$ works:
1. Base Case: Prove $F(base)$ works.
2. Inductive Step: Assume $F(k)$ works for $k < n$. Prove $F(n)$ works using that assumption.

Example: Factorial

python:

def fact(n):
  if n == 0: return 1
  return n * fact(n-1)

Base: $n=0$. Returns 1. Correct ($0! = 1$).
Step: Assume `fact(n-1)` returns $(n-1)!$.
Function returns $n \cdot (n-1)! = n!$. Correct.

Common Bugs

Off-by-one Errors

Often caused by wrong initialization or termination logic.
Invariant analysis helps catch these.
Does loop go to $n$ or $n-1$?
Check Invariant termination.

Infinite Loops

We must prove the loop makes progress.
Example: $i$ increases by 1.
Example: Binary Search window size decreases.

Practice: Find Bug 1

function BadSum(A, n)
  sum = 0
  i = 1          // Bug here?
  while i < n:
    sum = sum + A[i]
    i = i + 1
  return sum

Analysis of Bug 1

Init: $i=1$. `sum`=0.
Invariant says sum should be $A[0..0]$? No.
This skips $A[0]$.
Correct initialization: $i=0$.

Practice: Find Bug 2

function BinarySearch(A, x)
  ... 
  if x < A[mid]:
     high = mid   // Bug here?
  else:
     low = mid

Analysis of Bug 2

If $low = mid$ and $high = low + 1$,
$mid = low$.
If we go to else branch ($low = mid$),
Values don't change! Infinite loop.
Correct: $low = mid + 1$.

Writing Your Own Proofs

Template for Assignment

1. State the Invariant clearly.
2. Init: Show it holds at start.
3. Maintenance: Show it is preserved.
4. Termination: Show what it implies at end.
Don't just wave hands!

Common Pitfalls & Anti-Patterns

Watch Out For...

Forgetting to prove 'Termination' properties.
Stating invariant that isn't useful for the goal.

Interactive Practice

Activity 1: Sum Array

❓ Problem:
Invariant for code summing array A.
Take 2 minutes to solve this.

Solution 1

💡 Answer:
At start of loop i, 'sum' equals sum of A[0..i-1].

Activity 2: Max Finding

❓ Problem:
Invariant for Finding Max.
Take 2 minutes to solve this.

Solution 2

💡 Answer:
'max' holds largest value in A[0..i-1].

Activity 3: Selection Sort

❓ Problem:
State Invariant.
Take 2 minutes to solve this.

Solution 3

💡 Answer:
Subarray A[0..i-1] contains the i smallest elements in sorted order.

Activity 4: Binary Search

❓ Problem:
State Invariant.
Take 2 minutes to solve this.

Solution 4

💡 Answer:
If target exists, it is in A[low..high].

Interactive: Binary Search

Interactive Demo: binary_search_invariant.html

Activity 5: Termination

❓ Problem:
Why does Binary Search terminate?
Take 2 minutes to solve this.

Solution 5

💡 Answer:
Range size (high-low) decreases strictly monotonically.

Activity 6: Bubble Sort

❓ Problem:
State Invariant (Outer loop).
Take 2 minutes to solve this.

Solution 6

💡 Answer:
Subarray A[n-i..n] contains the i largest elements in sorted order.

All Interactive Demos

📊 Basic Invariants:

Interactive Demo: sum_invariant_demo.html

Interactive Demo: max_invariant_demo.html

📏 Searching:

Interactive Demo: binary_search_invariant.html

🔢 Sorting:

Interactive Demo: insertion_sort_invariant.html

Interactive Demo: selection_sort_invariant.html

Interactive Demo: bubble_sort_invariant.html

Supplementary Resources

Interactive Demo: worked_examples.html

Interactive Demo: reading_practice.html

Interactive Demo: handout.html