Week 10

Problem-Solving via Software

1/41

Programming: solving problems by building software

The process:

1. define the problem to be solved
2. develop a method for solving it
3. express the method as a software system
4. check that it really solves the problem
5. use the software to solve instances of the problem

... Problem-Solving via Software

2/41

Computer science has its own jargon for these steps:

1. specification: define the problem
2. design: develop a method to solve it
3. implementation: develop software
4. testing: check the software
5. deployment: use the software

• assists with the "how do I get started" problem
• is the next step in becoming an effective programmer

A Programmer's Toolkit

3/41

What a programmer needs to know:

• nuts-and-bolts:
  • types: numeric, character, boolean, values+operations
  • structures: hetero/homogeneous, linear, branched
  • data: memory, variables, scope, lifetime, assignment
  • control: function-calls, sequence, selection, iteration
  • abstraction: functions, modules/libraries, interfaces
• how to exploit the nuts-and-bolts:
  • a range of problem-solving strategies
  • ability to reason clearly and logically

Problem-Solving Strategies

4/41

Some questions which often perplex novice programmers:

• how do I design my program?
• where do the algorithms come from?
• how can I get started on a problem, given a spec?

• a small number of strategies cover many problems
• you hardly ever (never?) need to invent new algorithms
• a given algorithm can be applied in many different contexts

• it helps to remember a good-sized set of useful algorithms
• there's no substitute for experience (helps you to "recognise" the problem)

... Problem-Solving Strategies

5/41

We'll consider five basic strategies:

1. solution by evolution
2. divide and conquer
3. generate and test
4. simulation
5. approximation techniques

• determine which strategy might be best for solving it
• as noted above, experience improves ability to do this

Solution by Evolution

6/41

Many problems you come across

• will have similarities to ones you've seen/solved before
• perhaps differing only in the kind of data involved

Example:

• sorting by decreasing order, or sorting by date
• simply modify any sorting algorithm you've seen

... Solution by Evolution

7/41

Assumptions in "solution by evolution":

• you've already seen a solution to a related/similar problem
• you understand the essence of the solution   (abstract view)
• you can "fill in the details" specific to your new problem

The effort is spent on developing problem-specific operations

Exercise: Sorting

8/41

Take the supplied program for sorting numbers (taken from function median(), week 6)

• modify it so that it sorts in descending order
• modify it so that it can sort strings (ascending)
• modify it so that it can sort dates dd/mm/yyyy (ascending)

void sort(int array[], int n) {
        int i, min, marker;
        for (marker = 0; marker < n; marker++) {
		// find minimum between marker and n
                min = marker;
                for (i = marker+1; i < n; i++) {
                        if (array[i] < array[min]) {
				min = i;
			}
		}
		// swap element 'min' and 'marker'
                int tmp = array[marker];
                array[marker] = array[min];
		array[min] = tmp;
        }
}

Sorting Algorithms

Sorting Algorithms

10/41

We will discuss three sorting algorithms

• Selection Sort
• Insertion Sort
• Quick Sort

Selection Sort

11/41

This is the sorting method we just saw.

Basic algorithm (based on arrays):

• Starting with the first element in a list
  • if this is the last element in the list, you have finished
  • find the minimum in the list, and swap with the first element
  • redefine the list as starting with the next element
  • do next iteration

... Selection Sort

12/41

void selSort(int array[], int n) {  // an 'in place' sort so ...
	int i, min, marker;         // ... the array is 'overwritten'

	for (marker = 0; marker < n; marker++) {
		min = marker;       // assume element 'marker' is min
		for (i = marker+1; i < n; i++) {
			if (array[i] < array[min]) {
				min = i;  // found better min at 'i'
			}
		}
		int tmp;
		tmp = array[marker];
		array[marker] = array[min]; // swap elements 'min' ...
		array[min] = tmp;           // ... and 'marker'
	}
}

... Selection Sort

13/41

3  6  5  2  4  1

1  6  5  2  4  3

1  2  5  6  4  3

1  2  3  6  4  5

1  2  3  4  6  5

1  2  3  4  5  6

Exercise: Performance of Selection Sort

14/41

Measure performance of selection sort using the UNIX time command

• 1,000 numbers
• 10,000 numbers
• 100,000 numbers

• randomly generated
• already ordered
• reverse ordered (numbers in descending order)

... Exercise: Performance of Selection Sort

15/41

Selection sort is typical of a class of quadratic algorithms

• these have quadratic performance
  • quadratic means to the power of 2
  • execution time is proportional to the square of the input N
    • double N, quadruple the time
    • triple N, increase the time by factor 9
    • increase N by a factor of 10, it takes 10² = 100 times as long to execute
    • increase N by a factor of 100, it takes 100² = 10000 times as long to execute
    • etc

Selection Sort on Linked Lists

16/41

Sort algorithms are usually implemented using arrays:

• allow random access
  • every element can be accessed immediately
• are implemented efficiently/concisley

• linked lists
• even stacks (really!?)

... Selection Sort on Linked Lists

17/41

In Selection Sort

• outside loop:
  • marker  starts off at head of list
  • and steps through until the end
• inner loop:
  • starts off at  marker->next
  • and goes to the end of the list
• to swap elements: re-direct pointers

Insertion Sort

18/41

Often used by card players.

Basic algorithm:

• iteratively remove an element from the input data
  • insert that element at the correct position in the already sorted list
  • until no elements are left in the input list
• note: after k iterations, a sorted list of length k has been produced

... Insertion Sort

19/41

3  6  5  2  4  1

3  6  5  2  4  1

3  5  6  2  4  1

2  3  5  6  4  1

2  3  4  5  6  1

1  2  3  4  5  6

Exercise: Insertion Sort

20/41

Implement insertion sort

void insertionSort(int array[], int n)

Measure performance of insertion sort using the UNIX time command

• 1,000 numbers
• 10,000 numbers
• 100,000 numbers

• randomly generated
• already ordered
• reverse ordered (numbers in descending order)

... Exercise: Insertion Sort

21/41

If  array[i] >= array[i-1]

• then the inner loop does nothing
• and no element is moved

• to know an element is in the right position
  • needs 1 comparison each time
• total number of comparisons = (n-1)
• total number of moves = 0
• best case: linear performance

... Exercise: Insertion Sort

22/41

If  array[i] < array[0]

• then the inner loop goes all the way down to 0
• and moves up every element below  i

• total number of moves in worst case =
  • (n-1) moves for element n
  • + (n-2) moves for element n-1
  • + (n-3) moves for element n-2
  • + ...
  • + 2 moves for element 3
  • + 1 move for element 2
  • + 0 moves for element 1

Total number of comparisons is the same. Insertion sort is quadratic

Sorting with Stacks

23/41

The quadratic sorts can also be implemented using stacks

• not random-access; restricted to pushing and popping
• this restriction makes implementing more difficult

... Sorting with Stacks

24/41

Assume we have

• two stacks, A and B
• n unsorted numbers in stack A

1. pop the n numbers from A and push to B
   • but not the maximum number
2. push the maximum number onto A
3. pop the n-1 numbers from B and push them onto A
4. let n = n-1 and repeat steps 1, 2 and 3 until n = 1
5. A is now sorted (and B is of course empty)

... Sorting with Stacks

25/41

Bubble Sort

26/41

The simplest(?!) - but least efficient - of all sorting algorithms.

Video: Obama on Sorting Algorithms

Basic algorithm:

• repeatedly traverse the list:
  • compare each pair of nodes, and swap them if they are out of order
• if no swaps occur in a traversal, the alogrithm terminates
  • called early exit
• in each traversal, one or more element is "bubbled" into the right place
  • worst case: n passes required to sort n elements

... Bubble Sort

27/41

void bubbleSort(int array[], int n) {
	int i, j;

	int earlyexit = 0;
	for (i = n-1; i > 0 && !earlyexit; i--) {
		earlyexit = 1;                         // assume no swaps needed    
		for (j = 1; j <= i; j++) { 
			if (array[j-1] > array[j]){
				int tmp = array[j];    // swap elements j and j-1
				array[j] = array[j-1];
				array[j-1] = tmp;
				earlyexit = 0;         // swap was required
			}
		}
	}
}

Exercise: Performance of Bubble Sort

28/41

Measure performance of insertion sort using the UNIX time command

• 1,000 numbers
• 10,000 numbers
• 100,000 numbers

• randomly generated
• already ordered
• reverse ordered (numbers in descending order)

Quicksort

Problem-Solving Strategies

30/41

Five basic problem-solving strategies:

1. solution by evolution: adapt a known method
2. divide and conquer: solve partial problem, then extend
3. generate and test
4. simulation
5. approximation

Divide and Conquer

31/41

In scenarios where:

• the problem/data can be partitioned
• so that we solve a smaller sub-problem
• and then extend the solution to solve the whole problem

Requires a systematic way of "reducing" the problem/data

• e.g. consider one less element, or two halves of the elements

• e.g. no elements (empty data) or one element (often trivial)

... Divide and Conquer

32/41

Comments on divide and conquer:

• often lends itself to implementation via recursion
• is clearly related to proof-by-induction

Exercise: Towers of Hanoi

33/41

• move all the disks to the rightmost peg
• only one disk may be moved at a time
• no disk may ever be placed on top of a small disk

• hanoi(char from, char via, char to, int n)
• that outputs all moves to solve Towers of Hanoi with n disks

Quicksort

34/41

Quicksort applies divide and conquer to sorting:

• Divide
  • pick a pivot element
  • move all elements smaller than the pivot to its left
  • move all elements greater than the pivot to its right
• Conquer
  • sort the elements on the left
  • sort the elements on the right

... Quicksort

35/41

Divide ...

int partition(int array[], int low, int high) {
	int pivot_item = array[low];
	int left = low, right = high;
	while (left < right) {
		// find leftmost element > pivot_item 
		while (array[left] <= pivot_item) {
			left++;
		}
		// find rightmost element < pivot_item 
		while (array[right] > pivot_item) {
			right--;
		}
		if (left < right) {
			int tmp = array[left]; // swap left and right       
			array[left] = array[right];
			array[right] = tmp;
		}
	}
	array[low] = array[right]; // right is final position for pivot
	array[right] = pivot_item;
	return right;
}

... Quicksort

36/41

... and Conquer!

void quicksort(int array[], int low, int high) {
	int pivot;
	if (high > low) {       // termination condition low >= high  
		pivot = partition(array, low, high);
		quicksort(array, low, pivot-1);
		quicksort(array, pivot+1, high);
	}
}

... Quicksort

37/41

3  6  5  2  4  1

3  1  5  2  4  6

3  1  2  5  4  6

2  1   | 3 |   6  4  5

1  2   | 3 |   6  4  5

1  2   | 3 |   5  4   | 6 |

1  2   | 3 |   4  5   | 6 |

Exercise: Performance of Quicksort

38/41

Measure performance of quicksort using the UNIX time command

• 1,000 numbers
• 10,000 numbers
• 100,000 numbers

• randomly generated
• already ordered
• reverse ordered (numbers in descending order)

... Exercise: Performance of Quicksort

39/41

• average execution time is proportional to N * log(N) of the input N
  • increase N by a factor of 10
    • it takes around 10 * log₂10 = 33 times as long to execute
  • increase N by a factor of 100
    • it takes around 100 * log₂100 = 664 times as long to execute
  • etc

• worst case is still quadratic (but rare in practice)

Comparison of Sorting Algorithms

40/41

Tips for Next Week's Lab

41/41

Sorting with different data structures

• Implement selection sort on a linked list
  • use method described earlier (slide 17)

• Implement insertion sort on a linked list (challenging!)
  • develop method on paper before you start coding

• Implement selection sort on stacks (slide 24-25)
  • requires a function to pop n elements from a stack A and push onto a stack B except the maximum
    • elements need not be pushed in the exact same order in which they are popped

• Demonstrate that you have completed Stage 2 of the Assignment

Produced: 12 Oct 2016