Langton’s Ant is a turmite governed by simple rules whose outcome is both unpredictable and intresting. The path taken by the ant generates some surprising shapes, never appearing when you would expect them to, but a seemingly random moments. This article describes the rules behind Langton’s Ant, shows some of the images formed and provides a Python programme to simulate the ant.

Langton’s Ant

Chris Langton is a biologist, and the founder of the field of artificial life. One of his simplest and most intresting creations is Langton’s Ant. It is quite important theoretically, but it also has some intresting practical applications.

The ant’s world consists of a grid, possibly infinite, but limited in our example:

The ant always moves through this grid one step at a time. It’s direction and it’s effect on the grid is defined by 3 simple rules:

1. If the ant is on a black square, it turns right 90 degrees.
2. If the ant is on a white square, it turns left 90 degrees.
3. When the ant leaves a square, it inverts colour.

It is fairly obvious that the ant’s movement will leave a coloured trail. When asked, most people would guess that either no patterns show or that some basic symmetric images appear. What actually happens is this.

For the first 50 or so steps, it just seems to move around randomly, and then at step 52, you get a heart:

For the next 300 or so steps, the ant seems to draw random shapes, ofter erasing what it had drawn before. At about step 383 you get something that looks like this:

Next you get what’s technically known as a mess. The ant’s movement degenerates into chaos. No more patterns are observed:

By now, you are probably thinking: “Well yes. Simple rules, so at first you get simple patterns, but as the board gets more and more complex, the simple rules can’t handle it, so the ant moves randomly”. Well that’s right, up until about step 10000; that is when you start getting highways.

The highways the ant draws are intresting shapes. You can see one at the bottom of the next image. It is the diagonal bit going down and right. To build then, the ant seems to move in circles, the last row forcing it to draw the next. Thus, the ant continues to draw until something gets in its way. In this simulation, because the screen wraps around, the ant eventually gets back to the mess, where it starts moving randomly again.

What’s intresting is that, after some time, the ant starts building highways again. Actually, there is no known way to stop it. The only hindrance here is that the board is finite so it has a tendency to fill up. Anyway, here’s what the board looks like at step 97049:

The Programme

The following is a simple programme written in Python and using PyGame.

The variables that control the display and the board are at the top of the programme. They are heavily commented so there shouldn’t be any problems.

Here’s the code in Python (langton.py):

#************************************************
# Rules of the game
#	1. If the ant is on a black square, it turns
#		right 90 and moves forward one unit
#	2. If the ant is on a white square, it turns
# 		left 90 and moves forward one unit
#	3. When the ant leaves a square, it inverts
#		colour
#
# SEE: http://mathworld.wolfram.com/LangtonsAnt.html
#************************************************

import sys, pygame
from pygame.locals import *
import time

dirs = (
		(-1, 0),
		(0, 1),
		(1, 0),
		(0, -1)
		)

cellSize = 12 # size in pixels of the board (4 pixels are used to draw the grid)
numCells = 64 # length of the side of the board
background = 0, 0, 0 # background colour; black here
foreground = 23, 23, 23 # foreground colour; the grid's colour; dark gray here
textcol = 177, 177, 177 # the colour of the step display in the upper left of the screen
antwalk = 44, 88, 44 # the ant's trail; greenish here
antant = 222, 44, 44 # the ant's colour; red here
fps = 1.0 / 40 # time between steps; 1.0 / 40 means 40 steps per second

def main():
	pygame.init()

	size = width, height = numCells * cellSize, numCells * cellSize

	pygame.display.set_caption("Langton's Ant")

	screen = pygame.display.set_mode(size) # Screen is now an object representing the window in which we paint
	screen.fill(background)
	pygame.display.flip() # IMPORTANT: No changes are displayed until this function gets called

	for i in xrange(1, numCells):
		pygame.draw.line(screen, foreground, (i * cellSize, 1), (i * cellSize, numCells * cellSize), 2)
		pygame.draw.line(screen, foreground, (1, i * cellSize), (numCells * cellSize, i * cellSize), 2)
	pygame.display.flip() # IMPORTANT: No changes are displayed until this function gets called

	font = pygame.font.Font(None, 36)

	antx, anty = numCells / 2, numCells / 2
	antdir = 0
	board = [[False] * numCells for e in xrange(numCells)]

	step = 1
	pause = False
	while True:
		for event in pygame.event.get():
				if event.type == QUIT:
					return
				elif event.type == KEYUP:
					if event.key == 32: # If space pressed, pause or unpause
						pause = not pause
					elif event.key == 115:
						pygame.image.save(screen, "Step%d.tga" % (step))

		if pause:
			time.sleep(fps)
			continue

		text = font.render("%d " % (step), True, textcol, background)
		screen.blit(text, (10, 10))

		if board[antx][anty]:
			board[antx][anty] = False # See rule 3
			screen.fill(background, pygame.Rect(antx * cellSize + 1, anty * cellSize + 1, cellSize - 2, cellSize - 2))
			antdir = (antdir + 1) % 4 # See rule 1
		else:
			board[antx][anty] = True # See rule 3
			screen.fill(antwalk, pygame.Rect(antx * cellSize + 1, anty * cellSize + 1, cellSize - 2, cellSize - 2))
			antdir = (antdir + 3) % 4 # See rule 2

		antx = (antx + dirs[antdir][0]) % numCells
		anty = (anty + dirs[antdir][1]) % numCells

		# The current square (i.e. the ant) is painted a different colour
		screen.fill(antant, pygame.Rect(antx * cellSize + 1, anty * cellSize + 1, cellSize -2, cellSize -2))

		pygame.display.flip() # IMPORTANT: No changes are displayed until this function gets called

		step += 1
		time.sleep(fps)

if __name__ == "__main__":
	main()

It is intresting to note that although you know all the rules that govern the ant’s universe, you cannot predict its movement without resorting to simulation. This just goes to show that knowing the rules of the components at the lowest level might not help you understand the system as a whole.

That’s it. Have fun with the code. Always open to comments.

In this article I describe a way of modifying Dijkstra’s Alogrithm in order to find all the shortest path from a source to a node.

This article assumes you know how Dijkstra’s Algorithm works. If you don’t, see my previous post or the Wikipedia article.

The Problem

You know how to use Dijkstra’s algorithm to find the length of the shortest path to a node. You’ve even figured out how to record the path to each node. But you what you really need are all the shortest paths leading to a node.

The Idea

I can help, but to be honest, this is obvious.

In order to record the path to each node, I used an array to record which node comes before each other node in the shortest path. That is to say: prev[i] was the node that comes just before node i in the shortest path from the source to node i.

To record all the shortest paths that lead to a node, I just turned prev into a matrix with the following meaning: prev[i][0] is the number of nodes that could come before node i on a path of minimum length; prev[i][1..] are the nodes that could come before node i on path of minimum length.

The Programme

Here’s the code in C (dijkstraAll.c):

#include <stdio.h>

#define GRAPHSIZE 2048
#define INFINITY GRAPHSIZE*GRAPHSIZE
#define MAX(a, b) ((a > b) ? (a) : (b))

int e; /* The number of nonzero edges in the graph */
int n; /* The number of nodes in the graph */
long dist[GRAPHSIZE][GRAPHSIZE]; /* dist[i][j] is the distance between node i and j; or 0 if there is no direct connection */
long d[GRAPHSIZE]; /* d[i] is the length of the shortest path between the source (s) and node i */
int prev[GRAPHSIZE][GRAPHSIZE + 1]; /* prev[i] holds the nodes that could comes right before i in the shortest path from the source to i;
		prev[i][0] is the number of nodes and prev[i][1..] are the nodes */

void printD() {
	int i;

	printf("Distances:\n");
	for (i = 1; i <= n; ++i)
		printf("%10d", i);
	printf("\n");
	for (i = 1; i <= n; ++i) {
		printf("%10ld", d[i]);
	}
	printf("\n");
}

/*
 * Prints the shortest path from the source to dest.
 *
 * dijkstra(int) MUST be run at least once BEFORE
 * this is called
 */
void printPath(int dest, int depth) {
	int i, j;

	printf("-%d\n", dest);
	for (i = 1; i <= prev[dest][0]; ++i) {
		for (j = 0; j <= depth; ++j)
			printf(" |");
		printPath(prev[dest][i], depth + 1);
	}
}

void dijkstra(int s) {
	int i, k, mini;
	int visited[GRAPHSIZE];

	for (i = 1; i <= n; ++i) {
		d[i] = INFINITY;
		prev[i][0] = 0; /* no path has yet been found to i */
		visited[i] = 0; /* the i-th element has not yet been visited */
	}

	d[s] = 0;

	for (k = 1; k <= n; ++k) {
		mini = -1;
		for (i = 1; i <= n; ++i)
			if (!visited[i] && ((mini == -1) || (d[i] < d[mini])))
				mini = i;

		visited[mini] = 1;

		for (i = 1; i <= n; ++i)
			if (dist[mini][i]) {
				if (d[mini] + dist[mini][i] < d[i]) { /* a shorter path has been found */
					d[i] = d[mini] + dist[mini][i];
					prev[i][0] = 1;
					prev[i][1] = mini;
				} else if (d[mini] + dist[mini][i] == d[i]) { /* a path of the same length has been found */
					++prev[i][0];
					prev[i][prev[i][0]] = mini;
				}
			}
	}
}

int main(int argc, char *argv[]) {
	int i, j;
	int u, v, w;

	FILE *fin = fopen("dist.txt", "r");
	fscanf(fin, "%d", &e);
	for (i = 0; i < e; ++i)
		for (j = 0; j < e; ++j)
			dist[i][j] = 0;
	n = -1;
	for (i = 0; i < e; ++i) {
		fscanf(fin, "%d%d%d", &u, &v, &w);
		dist[u][v] = w;
		n = MAX(u, MAX(v, n));
	}
	fclose(fin);

	dijkstra(1);

	printD();

	printf("\n");
	for (i = 1; i <= n; ++i) {
		printf("Path to %d:\n", i);
		printPath(i, 0);
		printf("\n");
	}

	return 0;
}

And here’s an input file: dist.txt.
10
1 2 5
1 4 3
2 3 1
2 4 3
3 5 6
4 2 2
4 3 9
4 5 2
5 1 7
5 3 4


The input file describes this graph:

As you can see, there are two paths from node 1 to node 3: 1 -> 2 -> 3 and 1 -> 4 -> 2 -> 3 both of length 6.

Now, what does the programme output?
Distances:
1 2 3 4 5
0 5 6 3 5

Path to 5:
-5
|-4
| |-1


It first outputs the distances, and… yes! They’re correct.

Next, it prints those ASCII art drawings. They not drawings. They’re trees with the destination as root and the leafs as the source. To read a path from such a tree, start at a leaf (always 1) and go left, reading the first numbers you can see above.

Let’s find the paths to node 3. There are two leafs, so there are two paths of minimal length. The first one is 1 -> 4 -> 2 -> 3. The second one is 1 -> 2 -> 3. Check on the graph.

That’s it. If you’re up to a challenge, implement prev as an array of linked lists.

Good luck. Always open to comments.

In this article, I describe a simple (adds less than 1min of work) way to speed up Dijkstra’s Algorithm for finding the single source shortest path to every node in a graph.

In a previous post I described the simple O(n²) implementation of the algorithm. Here, I focus on a method that will probably speed up the algorithm.

Why Bother

The previous implementation of the algorithm ran in O(n²) time, where n is the number of nodes in the graph. This means that for a graph of, say 100 nodes, it would do about 100 * 100 = 100000 calculations. Considering that computers nowadays are said to be able to do about 100000000 (a hundred million) calculations per second, we’re fine, and the programme will finish in well under a second. But what if we have a graph with 100000 nodes? This might take 100 seconds to run. Now we’re in trouble. We need a faster algorithm.

The two most common ways to speed up Dijkstra’s Algorithm are to implement the finding of the closest node not yet visited as priority queues. Usually heaps or Fibonacci Heaps are used for this purpose (Fibonacci Heaps were actually invented for this).

Heaps are somewhat difficult to implement and Fibonacci Heaps are horror to implement. Incidentally, there’s a very easy of speeding it up.

Just Use Queues

The idea is to simply use queues instead of priority queues. This way provides nowhere near the same level of speedup (the algorithm is still O(n²)), but it makes it run faster, on average, by a factor of 4.

Some bad news: a carefully crafted graph could slow this algorithm down to O(n³). As a rule, graphs in real life are never like this, and, as the method isn’t widely known, test sets for contests are not written to catch this optimisation.

Now for the good news: it’s shockingly easy to write. Compare the old dijkstra1() with the new dijkstra2().

void dijkstra1(int s) {
	int i, k, mini;
	int visited[GRAPHSIZE];

	for (i = 1; i <= n; ++i) {
		d1[i] = INFINITY;
		visited[i] = 0; /* the i-th element has not yet been visited */
	}

	d1[s] = 0;

	for (k = 1; k <= n; ++k) {
		mini = -1;
		for (i = 1; i <= n; ++i)
			if (!visited[i] && ((mini == -1) || (d1[i] < d1[mini])))
				mini = i;

		visited[mini] = 1;

		for (i = 1; i <= n; ++i)
			if (dist[mini][i])
				if (d1[mini] + dist[mini][i] < d1[i])
					d1[i] = d1[mini] + dist[mini][i];
	}
}

void dijkstra2(int s) {
	int queue[GRAPHSIZE];
	char inQueue[GRAPHSIZE];
	int begq = 0,
	    endq = 0;
	int i, mini;
	int visited[GRAPHSIZE];

	for (i = 1; i <= n; ++i) {
		d2[i] = INFINITY;
		visited[i] = 0; /* the i-th element has not yet been visited */
		inQueue[i] = 0;
	}

	d2[s] = 0;
	queue[endq] = s;
	endq = (endq + 1) % GRAPHSIZE;

	while (begq != endq) {
		mini = queue[begq];
		begq = (begq + 1) % GRAPHSIZE;
		inQueue[mini] = 0;

		visited[mini] = 1;

		for (i = 1; i <= n; ++i)
			if (dist[mini][i])
				if (d2[mini] + dist[mini][i] < d2[i]) {
					d2[i] = d2[mini] + dist[mini][i];
					if (!inQueue[i]) {
						queue[endq] = i;
						endq = (endq + 1) % GRAPHSIZE;
						inQueue[i] = 1;
					}
				}
	}
}

What’s changed? First, we define several new variables. These, together, make up the queue:

int queue[GRAPHSIZE];
char inQueue[GRAPHSIZE];
int begq = 0,
    endq = 0;

Next, during the initialisation part of the function, we mark all nodes as not being in the queue.

for (i = 1; i <= n; ++i) {
	/* OTHER INITIALISATIONS (look at the programme) */
	inQueue[i] = 0;
}

Now, add the source node to the queue.

queue[endq] = s;
endq = (endq + 1) % GRAPHSIZE;

What does this do? The first line add s to the end of the queue. The second line moves the end of the queue one step to the right (I’ll explain a few paragraphs down). The modulo operation here is not really necessary, but I like to be consistent.

At this point we’ll start looping. When do we stop? The idea here is that a node is in the queue when its neighbours need to be updated (i.e. when a new shortest path might be found leading to them). So, we stop when the queue is empty. Note that this occurs when begq == endq and not when !(begq < endq). So, while (begq < endq) is incorrect because, in one case, begq will be greater then endq.

What was the first thing we did in the loop? We were supposed to find the closest node not yet visited. Now, we merely take the first node from the queue.

mini = queue[begq];
begq = (begq + 1) % GRAPHSIZE;
inQueue[mini] = 0;

Here, the first element is pop’d out of the queue, the head of the queue is moved one step to the right and the element is marked as not being in the queue. The problem with queues in general, and this one in particular is that the part of them that actually hold the elements tends to move around. Here, every insert moves the tail one step to the right and every pop moves the head one step to the right. Consider the following sequence of operations:

Moves 1 through 8 clearly show that, while the size of the information content of the queue changes erratically, it constantly moves to the right. What happened at 9? The queue got to the end of available memory and wrapped around to the beginning. This is the purpose of (begq + 1) % GRAPHSIZE and (endq + 1) % GRAPHSIZE. It turns 7, 8, 9, etc. into 1, 2, 3, etc. But won’t endq overrun begq? No, the use of inQueue guarantees that no element will be inserted in the queue more than once. And as the queue is of size GRAPHSIZE, no overrun is possible.

So far, so good. One last modification: when we update the distance to a node, we add it to the queue (if it’s not already in it).

for (i = 1; i <= n; ++i)
	if (dist[mini][i])
		if (d2[mini] + dist[mini][i] < d2[i]) {
			d2[i] = d2[mini] + dist[mini][i];
			if (!inQueue[i]) {
				queue[endq] = i;
				endq = (endq + 1) % GRAPHSIZE;
				inQueue[i] = 1;
			}
		}

Comparing the speed

When I first wrote this, I wanted to be able to check that it outputs correct results and I wanted to see how much faster it is. The following programme does both. The function cmpd() checks the output against that given by the simple implementation and the various clock() calls littered through the code time the two functions.

Here’s the code in C (dijkstra2.c):
Note: Source might be mangled by WordPress, consider downloading the file.

#include
#include 

#define GRAPHSIZE 2048
#define INFINITY GRAPHSIZE*GRAPHSIZE
#define MAX(a, b) ((a > b) ? (a) : (b))

int e; /* The number of nonzero edges in the graph */
int n; /* The number of nodes in the graph */
long dist[GRAPHSIZE][GRAPHSIZE]; /* dist[i][j] is the distance between node i and j; or 0 if there is no direct connection */
long d1[GRAPHSIZE], d2[GRAPHSIZE]; /* d[i] is the length of the shortest path between the source (s) and node i */

void dijkstra1(int s) {
	int i, k, mini;
	int visited[GRAPHSIZE];

	for (i = 1; i <= n; ++i) {
		d1[i] = INFINITY;
		visited[i] = 0; /* the i-th element has not yet been visited */
	}

	d1[s] = 0;

	for (k = 1; k <= n; ++k) {
		mini = -1;
		for (i = 1; i <= n; ++i)
			if (!visited[i] && ((mini == -1) || (d1[i] < d1[mini])))
				mini = i;

		visited[mini] = 1;

		for (i = 1; i <= n; ++i)
			if (dist[mini][i])
				if (d1[mini] + dist[mini][i] < d1[i])
					d1[i] = d1[mini] + dist[mini][i];
	}
}

void dijkstra2(int s) {
	int queue[GRAPHSIZE];
	char inQueue[GRAPHSIZE];
	int begq = 0,
	    endq = 0;
	int i, mini;
	int visited[GRAPHSIZE];

	for (i = 1; i <= n; ++i) {
		d2[i] = INFINITY;
		visited[i] = 0; /* the i-th element has not yet been visited */
		inQueue[i] = 0;
	}

	d2[s] = 0;
	queue[endq] = s;
	endq = (endq + 1) % GRAPHSIZE;

	while (begq != endq) {
		mini = queue[begq];
		begq = (begq + 1) % GRAPHSIZE;
		inQueue[mini] = 0;

		visited[mini] = 1;

		for (i = 1; i <= n; ++i)
			if (dist[mini][i])
				if (d2[mini] + dist[mini][i] < d2[i]) {
					d2[i] = d2[mini] + dist[mini][i];
					if (!inQueue[i]) {
						queue[endq] = i;
						endq = (endq + 1) % GRAPHSIZE;
						inQueue[i] = 1;
					}
				}
	}
}

int cmpd() {
	int i;

	for (i = 0; i &lt; n; ++i)
		if (d1[i] != d2[i])
			return 0;

	return 1;
}

int main(int argc, char *argv[]) {
	int i, j;
	int u, v, w;
	long t1 = 0,
	     t2 = 0;

	FILE *fin = fopen("dist2.txt", "r");
	fscanf(fin, "%d", &amp;e);
	for (i = 0; i &lt; e; ++i)
		for (j = 0; j &lt; e; ++j)
			dist[i][j] = 0;
	n = -1;
	for (i = 0; i &lt; e; ++i) {
		fscanf(fin, "%d%d%d", &amp;u, &amp;v, &amp;w);
		dist[u][v] = w;
		n = MAX(u, MAX(v, n));
	}
	fclose(fin);

	for (i = 1; i &lt;= n; ++i) {
		long aux = clock();
		dijkstra1(i);
		t1 += clock() - aux;
		aux = clock();
		dijkstra2(i);
		t2 += clock() - aux;
		if (i % 10 == 0) {
			printf("%d / %d\n", i, n);
			fflush(stdout);
		}
		if (!cmpd()) {
			printf("\nResults for %d do NOT match\n", i);
			break;
		}
	}
	printf("\n");

	printf("Dijkstra O(N^2):\t\t%ld\n", t1);
	printf("Dijkstra unstable:\t\t%ld\n", t2);
	printf("Ratio:\t\t\t\t%.2f\n", (float)t1/t2);

	/* printD(); */

	return 0;
}

And here’s a big 1200 node graph: dist2.txt.

Have fun and good luck. Always open to comments.

In this article, I describe Gauss’ algorithm for solving n linear equations with n unknowns. I also give a sample implementation in C.

The Problem

Let’s say you want to solve the following system of 3 equations with 3 unknowns:

Humans learn that there a two ways to solve this system. Reduction and substitution. Unfortunately, neither of these methods is suitable for a computer.

A simple algorithm (and the one used everywhere even today), was discovered by Gauss more than two hundred years ago. Since then, some refinements have been found, but the basic procedure remains unchanged.

Gaussian Elimination

Start by writing the system in matrix form:

If you recall how matrix multiplication works, you’ll see that’s true. If not, it’s enough to notice how the matrix is written: the coefficients of x, y and z are written, side by side, as the rows of a 3×3 matrix; x, y and z are then written as rows of a 3×1 matrix; finally, what’s left of the equality sign is written one under the other as a 3×1 matrix.

So far, this doesn’t actually help, but it does make the following process easier to write. The goal is, through simple transformations, to reach the system, where a, b and c are known.

How do you transform the initial system into the above one? Here’s Gauss’ idea.

Start with the initial system, then perform some operations to get 0s on the first column, on every row but the first.

The operations mentioned are multiplying the first rows by -3/2 and substracting it from the second. Then multiplying the first rows by -1 and substracting it from the third. What is -3/2? It’s first element of the second row divided by the first element of the first row. And -1? It’s the first element of the third row divided by the first element of the first row. NOTE The changes to row 1 are never actually written back into the matrix.

For now we’re done with row 1, so we move on to row 2. The goal here is to get every row on the second column under row 2 to 0. We do this by multiplying the second rows by 4 (i.e. 2 / (1 / 2)) and substracting it from the third rows.

Now it’s easy to find the value of z. Just multiply the third column by -1 (i.e. -1/1).

ERRATA: The 7 in the above matrix should be an 8.

Knowing the value of z, we can now eliminate it from the other two equations.

Now, we can find the value of y and eliminate y from the first equation.

And, finally, the value of x is:

And with that, we’re done.

The Programme

Unfortunately, this is easier said than done. The actual computer programme has to take into account divisions by zero and numerical instabilities. This adds to the complexity of forwardSubstitution().

Here’s the code in C (gauss.c):

#include <stdio.h>

int n;
float a[10][11];

void forwardSubstitution() {
	int i, j, k, max;
	float t;
	for (i = 0; i < n; ++i) {
		max = i;
		for (j = i + 1; j < n; ++j)
			if (a[j][i] > a[max][i])
				max = j;

		for (j = 0; j < n + 1; ++j) {
			t = a[max][j];
			a[max][j] = a[i][j];
			a[i][j] = t;
		}

		for (j = n; j >= i; --j)
			for (k = i + 1; k < n; ++k)
				a[k][j] -= a[k][i]/a[i][i] * a[i][j];

/*		for (k = 0; k < n; ++k) {
			for (j = 0; j < n + 1; ++j)
				printf("%.2f\t", a[k][j]);
			printf("\n");
		}*/
	}
}

void reverseElimination() {
	int i, j;
	for (i = n - 1; i >= 0; --i) {
		a[i][n] = a[i][n] / a[i][i];
		a[i][i] = 1;
		for (j = i - 1; j >= 0; --j) {
			a[j][n] -= a[j][i] * a[i][n];
			a[j][i] = 0;
		}
	}
}

void gauss() {
	int i, j;

	forwardSubstitution();
	reverseElimination();

	for (i = 0; i < n; ++i) {
		for (j = 0; j < n + 1; ++j)
			printf("%.2f\t", a[i][j]);
		printf("\n");
	}
}

int main(int argc, char *argv[]) {
	int i, j;

	FILE *fin = fopen("gauss.in", "r");
	fscanf(fin, "%d", &n);
	for (i = 0; i < n; ++i)
		for (j = 0; j < n + 1; ++j)
			fscanf(fin, "%f", &a[i][j]);
	fclose(fin);

	gauss();

	return 0;
}

In the above code, the first two for-s of forwardSubstitution(), just swap two rows so as to diminish the possibilities of some bad rounding errors. Also, if it exists with a division by zero error, that probably means the system cannot be solved.

And here’s the input file for the example (gauss.in) (save it as gauss.in):
3
2 1 -1 8
-3 -1 2 -11
-2 1 2 -3


That’s it. Good luck. Always open to comments.

In this article I describe Dijkstra’s algorithm for finding the shortest path from one source to all the other vertexes in a graph. Afterwards, I provide the source code in C of a simple implementation.

To understand this you should know what a graph is, and how to store one in memory. If in doubt check this and this.

Another solution for this problem is the Bellman-Ford algorithm.

The Problem

Given the following graph calculate the length of the shortest path from node 1 to every other node.

Lets take the nodes 1 and 3. There are several paths (1 -> 4 -> 3, 1 -> 2 -> 3, etc.), but the shortest of them is 1 -> 4 -> 2 -> 3 of length 9. Our job is to find it.

The Algorithm

Dijkstra’s algorithm is one of the most common solutions to this problem. Even so, it only works on graphs which have no edges of negative weight, and the actual speed of the algorithm can vary from O(n*lg(lg(n))) to O(n²).

The idea is somewhat simple:

Take the length of the shortest path to all nodes to be infinity. Mark the length of the shortest path to the source as 0.

Now, we already know that the graph has no edges of negative weight so the a path of length 0 is the best we can come up with. The path to the source is 0, so it’s optimal.

This algorithm works by making the paths to one more node optimal at each step. So, at the kth step, you know for sure that there are at least k nodes to which you know the shortest path.

At each step, choose the node, which is not yet optimal, but which is closest to the source; i.e. the node to which the current calculated shortest path is smallest. Then, from it, try to optimise the path to every node connected to it. Finally, mark the said node as optimal (visited, if you prefer). In the previous example, the node which is closest to the source and is not yet optimal is the source. From it, you can optimise the path to nodes 2 and 4.

At this point, the only visited/optimal node is 0. Now we have to redo this step 4 more times (to ensure that all nodes are optimal).

The next node to consider is 4:

It’s worthwhile to note that at this step, we’ve also found a better path to node 2.
Next is node 2:

Finally, we look at nodes 5 and 3 (none of which offer any optimisations):

The actual code in C looks something like this:

void dijkstra(int s) {
        int i, k, mini;
        int visited[GRAPHSIZE];

        for (i = 1; i <= n; ++i) {
                d[i] = INFINITY;
                visited[i] = 0; /* the i-th element has not yet been visited */
        }

        d[s] = 0;

        for (k = 1; k <= n; ++k) {
                mini = -1;
                for (i = 1; i <= n; ++i)
                        if (!visited[i] && ((mini == -1) || (d[i] < d[mini])))
                                mini = i;

                visited[mini] = 1;

                for (i = 1; i <= n; ++i)
                        if (dist[mini][i])
                                if (d[mini] + dist[mini][i] < d[i])
                                        d[i] = d[mini] + dist[mini][i];
        }
}

The Programme

Putting the above into context, we get the O(n²) implementation. This works well for most graphs (it will not work for graphs with negative weight edges), and it’s quite fast.

Here’s the source code in C (dijkstra.c):

#include <stdio.h>

#define GRAPHSIZE 2048
#define INFINITY GRAPHSIZE*GRAPHSIZE
#define MAX(a, b) ((a > b) ? (a) : (b))

int e; /* The number of nonzero edges in the graph */
int n; /* The number of nodes in the graph */
long dist[GRAPHSIZE][GRAPHSIZE]; /* dist[i][j] is the distance between node i and j; or 0 if there is no direct connection */
long d[GRAPHSIZE]; /* d[i] is the length of the shortest path between the source (s) and node i */

void printD() {
	int i;
	for (i = 1; i <= n; ++i)
		printf("%10d", i);
	printf("\n");
	for (i = 1; i <= n; ++i) {
		printf("%10ld", d[i]);
	}
	printf("\n");
}

void dijkstra(int s) {
	int i, k, mini;
	int visited[GRAPHSIZE];

	for (i = 1; i <= n; ++i) {
		d[i] = INFINITY;
		visited[i] = 0; /* the i-th element has not yet been visited */
	}

	d[s] = 0;

	for (k = 1; k <= n; ++k) {
		mini = -1;
		for (i = 1; i <= n; ++i)
			if (!visited[i] && ((mini == -1) || (d[i] < d[mini])))
				mini = i;

		visited[mini] = 1;

		for (i = 1; i <= n; ++i)
			if (dist[mini][i])
				if (d[mini] + dist[mini][i] < d[i])
					d[i] = d[mini] + dist[mini][i];
	}
}

int main(int argc, char *argv[]) {
	int i, j;
	int u, v, w;

	FILE *fin = fopen("dist.txt", "r");
	fscanf(fin, "%d", &e);
	for (i = 0; i < e; ++i)
		for (j = 0; j < e; ++j)
			dist[i][j] = 0;
	n = -1;
	for (i = 0; i < e; ++i) {
		fscanf(fin, "%d%d%d", &u, &v, &w);
		dist[u][v] = w;
		n = MAX(u, MAX(v, n));
	}
	fclose(fin);

	dijkstra(1);

	printD();

	return 0;
}

And here’s a sample input file(dist.txt):

10
1 2 10
1 4 5
2 3 1
2 4 3
3 5 6
4 2 2
4 3 9
4 5 2
5 1 7
5 3 4


The graph is given as an edge list:

• the first line contains e, the number of edges
• the following e lines contain 3 numbers: u, v and w signifying that there’s an edge from u to v of weight w

That’s it. Good luck and have fun. Always open to comments.

Finding the shortest path

UPDATE In response to campOs‘ comment.

Now we know the distance between the source node and any other node (the distance to the ith node is remembered in d[i]). But suppose we also need the path (which nodes make up the path).

Look at the above code. Where is d modified? Where is the recorded distance between the source and a node modified? In two places:

Firstly, d[s] is initialised to be 0.

d[s] = 0;

And then, when a new shortest path is found, d[i] is updated accordingly:

for (i = 1; i <= n; ++i)
	if (dist[mini][i])
		if (d[mini] + dist[mini][i] < d[i])
			d[i] = d[mini] + dist[mini][i];

The important thing to notice here is that when you update the shortest distance to node i, you know the previous node in the path to i. This is, of course, mini. This suggests the solution to our problem.

For every node i other than the source, remember not only the distance to it, but also the previous node in the path to it. Thus we have a new array, prev.

Now, we need to make to modifications.
First, we initialise the value of prev[i] to something impossible (say -1) at the start of dijkstra().

for (i = 1; i <= n; ++i) {
	d[i] = INFINITY;
	prev[i] = -1; /* no path has yet been found to i */
	visited[i] = 0; /* the i-th element has not yet been visited */
}

Secondly, we update the value of prev[i] every time a new shortest path is found to i.

for (i = 1; i <= n; ++i)
	if (dist[mini][i])
		if (d[mini] + dist[mini][i] < d[i]) {
			d[i] = d[mini] + dist[mini][i];
			prev[i] = mini;
		}

Good. For every node reachable from the source we know which node is just before it in the shortest path. For the above example, we would have the following array:
i - prev[i]
1 - -1
2 - 4
3 - 2
4 - 1
5 - 4


Using this, how do you get the path? Let’s say you want to get to 3. Which node comes right before 3? Node 2. Which node comes right before node 2? Node 4. Which node comes before 4? Node 1. We’ve reached the source, so we’re done. Go through this list backwards and you get the path: 1 -> 4 -> 2 -> 3. This is easily implemented with recursion.

void printPath(int dest) {
	if (prev[dest] != -1)
		printPath(prev[dest]);
	printf("%d ", dest);
}

Here is the updated source: dijkstraWithPath.c.

Good luck.

In this article, I describe the Bellman-Ford algorithm for finding the one-source shortest paths in a graph, give an informal proof and provide the source code in C for a simple implementation.

To understand this you should know what a graph is, and how to store one in memory. If in doubt check this and this.

Another solution to this problem is Dijkstra’s algorithm.

The Problem

Given the following graph, calculate the length of the shortest path from node 1 to node 2.

It’s obvious that there’s a direct route of length 6, but take a look at path: 1 -> 4 -> 3 -> 2. The length of the path is 7 - 3 - 2 = 2, which is less than 6. BTW, you don’t need negative edge weights to get such a situation, but they do clarify the problem.

This also suggests a property of shortest path algorithms: to find the shortest path form x to y, you need to know, beforehand, the shortest paths to y’s neighbours. For this, you need to know the paths to y’s neighbours’ neighbours… In the end, you must calculate the shortest path to the connected component of the graph in which x and y are found.

That said, you usually calculate the shortest path to all nodes and then pick the ones you’re intrested in.

The Algorithm

The Bellman-Ford algorithm is one of the classic solutions to this problem. It calculates the shortest path to all nodes in the graph from a single source.

The basic idea is simple:
Start by considering that the shortest path to all nodes, less the source, is infinity. Mark the length of the path to the source as 0:

Take every edge and try to relax it:

Relaxing an edge means checking to see if the path to the node the edge is pointing to can’t be shortened, and if so, doing it. In the above graph, by checking the edge 1 -> 2 of length 6, you find that the length of the shortest path to node 1 plus the length of the edge 1 -> 2 is less then infinity. So, you replace infinity in node 2 with 6. The same can be said for edge 1 -> 4 of length 7. It’s also worth noting that, practically, you can’t relax the edges whose start has the shortest path of length infinity to it.

Now, you apply the previous step n - 1 times, where n is the number of nodes in the graph. In this example, you have to apply it 4 times (that’s 3 more times).

That’s it, here’s the algorithm in a condensed form:

void bellman_ford(int s) {
        int i, j;

        for (i = 0; i < n; ++i)
                d[i] = INFINITY;

        d[s] = 0;

        for (i = 0; i < n - 1; ++i)
                for (j = 0; j < e; ++j)
                        if (d[edges[j].u] + edges[j].w < d[edges[j].v])
                                d[edges[j].v] = d[edges[j].u] + edges[j].w;
}

Here, d[i] is the shortest path to node i, e is the number of edges and edges[i] is the i-th edge.

It may not be obvious why this works, but take a look at what is certain after each step. After the first step, any path made up of at most 2 nodes will be optimal. After the step 2, any path made up of at most 3 nodes will be optimal… After the (n - 1)-th step, any path made up of at most n nodes will be optimal.

The Programme

The following programme just puts the bellman_ford function into context. It runs in O(VE) time, so for the example graph it will do something on the lines of 5 * 9 = 45 relaxations. Keep in mind that this algorithm works quite well on graphs with few edges, but is very slow for dense graphs (graphs with almost n² edges). For graphs with lots of edges, you’re better off with Dijkstra’s algorithm.

Here’s the source code in C (bellmanford.c):

#include <stdio.h>

typedef struct {
	int u, v, w;
} Edge;

int n; /* the number of nodes */
int e; /* the number of edges */
Edge edges[1024]; /* large enough for n <= 2^5=32 */
int d[32]; /* d[i] is the minimum distance from node s to node i */

#define INFINITY 10000

void printDist() {
	int i;

	printf("Distances:\n");

	for (i = 0; i < n; ++i)
		printf("to %d\t", i + 1);
	printf("\n");

	for (i = 0; i < n; ++i)
		printf("%d\t", d[i]);

	printf("\n\n");
}

void bellman_ford(int s) {
	int i, j;

	for (i = 0; i < n; ++i)
		d[i] = INFINITY;

	d[s] = 0;

	for (i = 0; i < n - 1; ++i)
		for (j = 0; j < e; ++j)
			if (d[edges[j].u] + edges[j].w < d[edges[j].v])
				d[edges[j].v] = d[edges[j].u] + edges[j].w;
}

int main(int argc, char *argv[]) {
	int i, j;
	int w;

	FILE *fin = fopen("dist.txt", "r");
	fscanf(fin, "%d", &n);
	e = 0;

	for (i = 0; i < n; ++i)
		for (j = 0; j < n; ++j) {
			fscanf(fin, "%d", &w);
			if (w != 0) {
				edges[e].u = i;
				edges[e].v = j;
				edges[e].w = w;
				++e;
			}
		}
	fclose(fin);

	/* printDist(); */

	bellman_ford(0);

	printDist();

	return 0;
}

And here’s the input file used in the example (dist.txt):
5
0 6 0 7 0
0 0 5 8 -4
0 -2 0 0 0
0 0 -3 9 0
2 0 7 0 0

That’s an adjacency matrix.

That’s it. Have fun. Always open to comments.

The 0-1 Knapsack Problem (AKA The Discrete Knapsack Problem) is a famous problem solvable by dynamic-programming. In this article, I describe the problem, the most common algorithm used to solve it and then provide a sample implementation in C.

If you’ve never heard of the Knapsack Problems before, it will help to read this previous post.

The Problem

The Discrete (0-1) Knapsack Problem usually sounds like this:

Little Red Riding Hood wants to bring grandma a basket of goodies. She has an unlimited supply of n types of sweets each weighting c[i] and having the nutritional value of v[i]. Her basket can hold at most W kilograms of sweets.

Given n, c, v and W, figure out which sweets and how many to take so that the nutritional value in maximal.

So, for this input:
n = 3
c = {8, 6, 4}
v = {16, 10, 7}
W = 10


LRRH should take one of 3 and one of 2, amassing 17 nutritional points.

You’re usually dealling with a knapsack problem when you’re give the cost and the benefits of certain objects and asked to obtain the maximum benefit so that the sum of the costs is smaller than a given value. You have got the Discrete Knapsack Problem when you can only take the whole object or none at all and you have an unlimited supply of objects.

The Algorithm

This is a dynamic-programming algorithm.

The idea is to first calculate the maximum benefit for weight x and only after that to calculate the maximum benefit for x+1. So, on the whole, you first calculate the maximum benefit for 1, then for 2, then for 3, …, then for W-1 and, finally, for W. I store the maximum benefits in an array named a.

Start with a[0] = 0. Then for every a between 1 … W use the formula:
a[i] = max{vj + a(i − cj) | cj ≤ i }

The basic idea is that to reach weight x, you have to add an object of weight w to a previous maximum benefit. More specifically, you have to add w to x - w. Now, there will probably be several ways to reach weight x, so you have to choose the one that maximises the benefit. That’s what the max is for.

Basically, the formula says: “To calculate the benefit of weight x, take every object (value: v; weight: w) and see if the benefit for x - w plus v is greater than the current benefit for x. If so, change it.”

So, for the example, the programme would output (and do) this:
Weight 0; Benefit: 0; Can't reach this exact weight.
Weight 1; Benefit: 0; Can’t reach this exact weight.
Weight 2; Benefit: 0; Can’t reach this exact weight.
Weight 3; Benefit: 0; Can’t reach this exact weight.
Weight 4; Benefit: 7; To reach this weight I added object 3 (7$ 4Kg) to weight 0.
Weight 5; Benefit: 7; To reach this weight I added object 3 (7$ 4Kg) to weight 1.
Weight 6; Benefit: 10; To reach this weight I added object 2 (10$ 6Kg) to weight 0.
Weight 7; Benefit: 10; To reach this weight I added object 2 (10$ 6Kg) to weight 1.
Weight 8; Benefit: 16; To reach this weight I added object 1 (16$ 8Kg) to weight 0.
Weight 9; Benefit: 16; To reach this weight I added object 1 (16$ 8Kg) to weight 1.
Weight 10; Benefit: 17; To reach this weight I added object 2 (10$ 6Kg) to weight 4.


The Programme

This programme runs in pseudo-plynominal time O(n * W). i.e. Slow as hell for large very values of W. Also because it holds to arrays of at least length W, it’s also horribly memory inefficient. Unfortunately, there’s not much you can do.

Here’s the code in C (knapsack10.c):

#include <stdio.h>

#define MAXWEIGHT 100

int n = 3; /* The number of objects */
int c[10] = {8, 6, 4}; /* c[i] is the *COST* of the ith object; i.e. what
				YOU PAY to take the object */
int v[10] = {16, 10, 7}; /* v[i] is the *VALUE* of the ith object; i.e.
				what YOU GET for taking the object */
int W = 10; /* The maximum weight you can take */ 

void fill_sack() {
	int a[MAXWEIGHT]; /* a[i] holds the maximum value that can be obtained
				using at most i weight */
	int last_added[MAXWEIGHT]; /* I use this to calculate which object were
					added */
	int i, j;
	int aux;

	for (i = 0; i <= W; ++i) {
		a[i] = 0;
		last_added[i] = -1;
	}

	a[0] = 0;
	for (i = 1; i <= W; ++i)
		for (j = 0; j < n; ++j)
			if ((c[j] <= i) && (a[i] < a[i - c[j]] + v[j])) {
				a[i] = a[i - c[j]] + v[j];
				last_added[i] = j;
			}

	for (i = 0; i <= W; ++i)
		if (last_added[i] != -1)
			printf("Weight %d; Benefit: %d; To reach this weight I added object %d (%d$ %dKg) to weight %d.\n", i, a[i], last_added[i] + 1, v[last_added[i]], c[last_added[i]], i - c[last_added[i]]);
		else
			printf("Weight %d; Benefit: 0; Can't reach this exact weight.\n", i);

	printf("---\n");

	aux = W;
	while ((aux > 0) && (last_added[aux] != -1)) {
		printf("Added object %d (%d$ %dKg). Space left: %d\n", last_added[aux] + 1, v[last_added[aux]], c[last_added[aux]], aux - c[last_added[aux]]);
		aux -= c[last_added[aux]];
	}

	printf("Total value added: %d$\n", a[W]);
}

int main(int argc, char *argv[]) {
	fill_sack();

	return 0;
}

That’s it. Good luck. Always open to comments.

In this article, I describe the greedy algorithm for solving the Fractional Knapsack Problem and give an implementation in C.

The Problem

The Fractional Knapsack Problem usually sounds like this:

Ted Thief has just broken into the Fort Knox! He sees himself in a room with n piles of gold dust. Because the each pile has a different purity, each pile also has a different value (v[i]) and a different weight (c[i]). Ted has a knapsack that can only hold W kilograms.

Given n, v, c and W, calculate which piles Ted should completely put into his knapsack and which he should put only a fraction of.

So, for this input:
n = 5
c = {12, 1, 2, 1, 4}
v = {4, 2, 2, 1, 10}
W = 15


Ted should take piles 2, 3, 4 and 5 completely and about 58% of pile 1.

You’re usually dealling with a knapsack problem when you’re give the cost and the benefits of certain objects and asked to obtain the maximum benefit so that the sum of the costs is smaller than a given value. You’ve got the fractional knapsack problem when you can take fractions (as opposed to all or nothing) of the objects.

The Algorithm

This is a standard greedy algorithm. In fact, it’s one of the classic examples.

The idea is to calculate for each object the ratio of value/cost, and sort them according to this ratio. Then you take the objects with the highest ratios and add them until you can’t add the next object as whole. Finally add as much as you can of the next object.

So, for our example:
v = {4, 2, 2, 1, 10}
c = {12, 1, 2, 1, 4}
r = {1/3, 2, 1, 1, 5/2}


From this it’s obvious that you should add the objects: 5, 2, 3, 4 and then as much as possible of 1.
The output of my programme is this:
Added object 5 (10$, 4Kg) completly in the bag. Space left: 11.
Added object 2 (2$, 1Kg) completly in the bag. Space left: 10.
Added object 3 (2$, 2Kg) completly in the bag. Space left: 8.
Added object 4 (1$, 1Kg) completly in the bag. Space left: 7.
Added 58% (4$, 12Kg) of object 1 in the bag.
Filled the bag with objects worth 15.48$.

The Programme

Now, you could implement the algorithm as stated, but for practical reasons you may wish to trade speed for simplicity. That’s what I’ve done here: instead of sorting the objects, I simply go through them every time searching for the best ratio. This modification turns an O(n*lg(n)) algorithm into an O(n²) one. For small values of n, this doesn’t matter and n is usually small.

Here’s the code in C (fractional_knapsack.c):

#include <stdio.h>

int n = 5; /* The number of objects */
int c[10] = {12, 1, 2, 1, 4}; /* c[i] is the *COST* of the ith object; i.e. what
				YOU PAY to take the object */
int v[10] = {4, 2, 2, 1, 10}; /* v[i] is the *VALUE* of the ith object; i.e.
				what YOU GET for taking the object */
int W = 15; /* The maximum weight you can take */

void simple_fill() {
	int cur_w;
	float tot_v;
	int i, maxi;
	int used[10];

	for (i = 0; i < n; ++i)
		used[i] = 0; /* I have not used the ith object yet */

	cur_w = W;
	while (cur_w > 0) { /* while there's still room*/
		/* Find the best object */
		maxi = -1;
		for (i = 0; i < n; ++i)
			if ((used[i] == 0) &&
				((maxi == -1) || ((float)v[i]/c[i] > (float)v[maxi]/c[maxi])))
				maxi = i;

		used[maxi] = 1; /* mark the maxi-th object as used */
		cur_w -= c[maxi]; /* with the object in the bag, I can carry less */
		tot_v += v[maxi];
		if (cur_w >= 0)
			printf("Added object %d (%d$, %dKg) completly in the bag. Space left: %d.\n", maxi + 1, v[maxi], c[maxi], cur_w);
		else {
			printf("Added %d%% (%d$, %dKg) of object %d in the bag.\n", (int)((1 + (float)cur_w/c[maxi]) * 100), v[maxi], c[maxi], maxi + 1);
			tot_v -= v[maxi];
			tot_v += (1 + (float)cur_w/c[maxi]) * v[maxi];
		}
	}

	printf("Filled the bag with objects worth %.2f$.\n", tot_v);
}

int main(int argc, char *argv[]) {
	simple_fill();

	return 0;
}

That’s it. Good luck.

Always open to comments.

Update: The next article in this series is The 0-1 Knapsack Problem.

In this article I describe the Floyd-Warshall algorithm for finding the shortest path between all nodes in a graph. I give an informal proof and provide an implementation in C.

Shortest paths

The shortest path between two nodes of a graph is a sequence of connected nodes so that the sum of the edges that inter-connect them is minimal.

Take this graph,

There are several paths between A and E:
Path 1: A -> B -> E 20
Path 2: A -> D -> E 25
Path 3: A -> B -> D -> E 35
Path 4: A -> D -> B -> E 20


There are several things to notice here:

1. There can be more then one route between two nodes
2. The number of nodes in the route isn’t important (Path 4 has 4 nodes but is shorter than Path 2, which has 3 nodes)
3. There can be more than one path of minimal length

Something else that should be obvious from the graph is that any path worth considering is simple. That is, you only go through each node once.

Unfortunately, this is not always the case. The problem appears when you allow negative weight edges. This isn’t by itself bad. But if a loop of negative weight appears, then there is no shortest path. Look at this example:

Look at the path B -> E -> D -> B. This is a loop, because the starting node is the also the end. What’s the cost? It’s 10 - 20 + 5 = -5. This means that adding this loop to a path once lowers the cost of the path by 5. Adding it twice would lower the cost by 2 * 5 = 10. So, whatever shortest path you may have come up with, you can make it smaller by going through the loop one more time. BTW there’s no problem with a negative cost path.

The Floyd-Warshall Algorithm

This algorithm calculates the length of the shortest path between all nodes of a graph in O(V³) time. Note that it doesn’t actually find the paths, only their lengths.

Let’s say you have the adjacency matrix of a graph. Assuming no loop of negative values, at this point you have the minimum distance between any two nodes which are connected by an edge.
A B C D E
A 0 10 0 5 0
B 10 0 5 5 10
C 0 5 0 0 0
D 5 5 0 0 20
E 0 10 0 20 0

The graph is the one shown above (the first one).

The idea is to try to interspace A between any two nodes in hopes of finding a shorter path.
A B C D E
A 0 10 0 5 0
B 10 0 5 5 10
C 0 5 0 0 0
D 5 5 0 0 20
E 0 10 0 20 0

Then try to interspace B between any two nodes:
A B C D E
A 0 10 15 5 20
B 10 0 5 5 10
C 15 5 0 10 15
D 5 5 10 0 15
E 20 10 15 15 0

Do the same for C:
A B C D E
A 0 10 15 5 20
B 10 0 5 5 10
C 15 5 0 10 15
D 5 5 10 0 15
E 20 10 15 15 0

Do the same for D:
A B C D E
A 0 10 15 5 20
B 10 0 5 5 10
C 15 5 0 10 15
D 5 5 10 0 15
E 20 10 15 15 0

And for E:
A B C D E
A 0 10 15 5 20
B 10 0 5 5 10
C 15 5 0 10 15
D 5 5 10 0 15
E 20 10 15 15 0

This is the actual algorithm:

# dist(i,j) is "best" distance so far from vertex i to vertex j 

# Start with all single edge paths.
 For i = 1 to n do
     For j = 1 to n do
         dist(i,j) = weight(i,j) 

 For k = 1 to n do # k is the `intermediate' vertex
     For i = 1 to n do
         For j = 1 to n do
             if (dist(i,k) + dist(k,j) < dist(i,j)) then # shorter path?
                 dist(i,j) = dist(i,k) + dist(k,j)

The Programme

Here’s the code in C(floyd_warshall.c):

#include <stdio.h>

int n; /* Then number of nodes */
int dist[16][16]; /* dist[i][j] is the length of the edge between i and j if
			it exists, or 0 if it does not */

void printDist() {
	int i, j;
	printf("    ");
	for (i = 0; i < n; ++i)
		printf("%4c", 'A' + i);
	printf("\n");
	for (i = 0; i < n; ++i) {
		printf("%4c", 'A' + i);
		for (j = 0; j < n; ++j)
			printf("%4d", dist[i][j]);
		printf("\n");
	}
	printf("\n");
}

/*
	floyd_warshall()

	after calling this function dist[i][j] will the the minimum distance
	between i and j if it exists (i.e. if there's a path between i and j)
	or 0, otherwise
*/
void floyd_warshall() {
	int i, j, k;
	for (k = 0; k < n; ++k) {
		printDist();
		for (i = 0; i < n; ++i)
			for (j = 0; j < n; ++j)
				/* If i and j are different nodes and if
					the paths between i and k and between
					k and j exist, do */
				if ((dist[i][k] * dist[k][j] != 0) && (i != j))
					/* See if you can't get a shorter path
						between i and j by interspacing
						k somewhere along the current
						path */
					if ((dist[i][k] + dist[k][j] < dist[i][j]) ||
						(dist[i][j] == 0))
						dist[i][j] = dist[i][k] + dist[k][j];
	}
	printDist();
}

int main(int argc, char *argv[]) {
	FILE *fin = fopen("dist.txt", "r");
	fscanf(fin, "%d", &n);
	int i, j;
	for (i = 0; i < n; ++i)
		for (j = 0; j < n; ++j)
			fscanf(fin, "%d", &dist[i][j]);
	fclose(fin);

	floyd_warshall();

	return 0;
}

Note that of the above programme, all the work is done by only five lines (30-48).

That’s it. Good luck. Always open to comments.

In this article I give an informal definition of a graph and of the minimum spanning tree. Afterwards I describe Prim’s algorithm and then follow its execution on an example. Finally, the code in C is provided.

Graphs

Wikipedia gives one of the common definitions of a graph:

In computer science, a graph is a kind of data structure, specifically an abstract data type (ADT), that consists of a set of nodes and a set of edges that establish relationships (connections) between the nodes. The graph ADT follows directly from the graph concept from mathematics.
Informally, G=(V,E) consists of vertices, the elements of V, which are connected by edges, the elements of E. Formally, a graph, G, is defined as an ordered pair, G=(V,E), where V is a finite set and E is a set consisting of two element subsets of V.

This is a graph:

It’s a set of nodes (A, B, C, D and E) and the edges (lines) that interconnect them.

An important thing to note about this graph is that the edges are bidirectional, i.e. if A is connected to B, then B is connected to A. This makes it an undirected graph.

A common extension is to attribute weights to the edges. This is what I’ve done with the previous graph:

Minimum spanning trees

Basically a minimum spanning tree is a subset of the edges of the graph, so that there’s a path form any node to any other node and that the sum of the weights of the edges is minimum.

Here’s the minimum spanning tree of the example:

Look at the above image closely. It contains all of the initial nodes and some of the initial edges. Actually it contains exactly n - 1 edges, where n is the number of nodes. It’s called a tree because there are no cycles.

You can think of the graph as a map, with the nodes being cities, the edges passable terrain, and the weights the distance between the cities.

It’s worth mentioning that a graph can have several minimum spanning trees. Think of the above example, but replace all the weight with 1. The resulting graph will have 6 minimum spanning trees.

Given a graph, find one of its minimum spanning trees.

Prim’s Algorithm

One of the classic algorithms for this problem is that found by Robert C. Prim. It’s a greedy style algorithm and it’s guaranteed to produce a correct result.

In the following discussion, let the distance from each node not in the tree to the tree be the edge of minimal weight between that node and some node in the tree. If there is no such edge, assume the distance is infinity (this shouldn’t happen).

The algorithm (greedily) builds the minimal spanning tree by iteratively adding nodes into a working tree:

1. Start with a tree which contains only one node.
2. Identify a node (outside the tree) which is closest to the tree and add the minimum weight edge from that node to some node in the tree and incorporate the additional node as a part of the tree.
3. If there are less then n - 1 edges in the tree, go to 2

For the example graph, here’s how it would run:

Start with only node A in the tree.

Find the closest node to the tree, and add it.

Repeat until there are n - 1 edges in the tree.

The Programme

The following programme just follows the algorithm. It runs in O(n²) time.

Here’s the code in C (prim.c):

#include <stdio.h>

/*
	The input file (weight.txt) look something like this
		4
		0 0 0 21
		0 0 8 17
		0 8 0 16
		21 17 16 0

	The first line contains n, the number of nodes.
	Next is an nxn matrix containg the distances between the nodes
	NOTE: The distance between a node and itself should be 0
*/

int n; /* The number of nodes in the graph */

int weight[100][100]; /* weight[i][j] is the distance between node i and node j;
			if there is no path between i and j, weight[i][j] should
			be 0 */

char inTree[100]; /* inTree[i] is 1 if the node i is already in the minimum
			spanning tree; 0 otherwise*/

int d[100]; /* d[i] is the distance between node i and the minimum spanning
		tree; this is initially infinity (100000); if i is already in
		the tree, then d[i] is undefined;
		this is just a temporary variable. It's not necessary but speeds
		up execution considerably (by a factor of n) */

int whoTo[100]; /* whoTo[i] holds the index of the node i would have to be
			linked to in order to get a distance of d[i] */

/* updateDistances(int target)
	should be called immediately after target is added to the tree;
	updates d so that the values are correct (goes through target's
	neighbours making sure that the distances between them and the tree
	are indeed minimum)
*/
void updateDistances(int target) {
	int i;
	for (i = 0; i < n; ++i)
		if ((weight[target][i] != 0) && (d[i] > weight[target][i])) {
			d[i] = weight[target][i];
			whoTo[i] = target;
		}
}

int main(int argc, char *argv[]) {
	FILE *f = fopen("dist.txt", "r");
	fscanf(f, "%d", &n);
	int i, j;
	for (i = 0; i < n; ++i)
		for (j = 0; j < n; ++j)
			fscanf(f, "%d", &weight[i][j]);
	fclose(f);

	/* Initialise d with infinity */
	for (i = 0; i < n; ++i)
		d[i] = 100000;

	/* Mark all nodes as NOT beeing in the minimum spanning tree */
	for (i = 0; i < n; ++i)
		inTree[i] = 0;

	/* Add the first node to the tree */
	printf("Adding node %c\n", 0 + 'A');
	inTree[0] = 1;
	updateDistances(0);

	int total = 0;
	int treeSize;
	for (treeSize = 1; treeSize < n; ++treeSize) {
		/* Find the node with the smallest distance to the tree */
		int min = -1;
		for (i = 0; i < n; ++i)
			if (!inTree[i])
				if ((min == -1) || (d[min] > d[i]))
					min = i;

		/* And add it */
		printf("Adding edge %c-%c\n", whoTo[min] + 'A', min + 'A');
		inTree[min] = 1;
		total += d[min];

		updateDistances(min);
	}

	printf("Total distance: %d\n", total);

	return 0;
}

And here’s a sample input file (dist.txt). It’s the example graph:
5
0 10 0 5 0
10 0 5 5 10
0 5 0 0 0
5 5 0 0 20
0 10 0 20 0


The code’s commented and there shouldn’t be any problems.

Good luck. Always open to comments.