Early Starting or Deep Thinking?

I wish the first idea comes to mind was the perfect one. But usually, it is not even close enough to be a good solution – it is often greedy and very straight-forward. The more you learn about the problem and its characteristics, the better and more efficient solution will pop up. If you start coding in the first hour, you will probably never ever finish the task. I’d like to share a case study from Jon Bentley’s amazing book, Programming Pearls.

The story is about Andrew Appel and his experience on reducing the run time of a system from one year to one day. They had to simulate the motions of N objects in 3D space, given their masses and initial positions and velocities. They basically were interested to predict the positions of planets, stars, galaxies 10 years, 100 years, 1000 years later. In a 2D space the input may look like the image on the left: a large set of vectors.

The regular simulation programs divide time into small steps and computes the progress of each object at each step. Due to the existing of gravitational field and mass attractions, the computational cost were proportional to N². Appel estimated that 1000 time steps of such a system with N=10000 objects would require more than one year on a VAX-11/780 or a day on a Cray-1. Andrew had to develop a system far more efficient than a one-year-running turtle. After several speedup improvements, he reduced the runtime to less than a day on VAX-11/780 — almost 400 times faster than the initial setup.

He managed to change the concept on many levels to make such a huge difference.

1. Data Structures: Foremost, O(N²) had to go. He decided to represent objects as leaves on a binary tree; higher nodes were representing the cluster of objects. The force operating on a particular object can be approximated  by the force exerted by the large clusters. This alone reduced complexity to O(N logN) by a factor of 12.
2. Algorithm Tuning: Initial algorithm has to use tiny time steps to handle the rare case that two particles come closer to one another. Usages of tree-based structure enabled investigation of such rare cases possible independently from time steps. As a result, Andrew doubled the time step size and halved the run time.
3. Data Structure Reorganization: The tree that was representing the initial objects was quite bad in representing the later sets. Reconfiguring the tree at every new step was a bit costly but reduced the number of local calculations and halved the total runtime.
4. Code Tuning: Due to additional numerical accuracy provided by the tree, 64-bit floating point number could be replaced by 32 bit single-precision, halved the runtime. Additionally, profiling the application showed that 98% percent of runtime was being spend in one procedure. Rewriting that procedure in assembly  increased the speed by a factor of 2,5.
5. Hardware Extensions: Moving the program to a similar machine with a floating point accelerator halved the total runtime.

Finally new system was requiring only one day to simulate 10k objects. Most of the work done by Andrew Appel were visible without writing a single line of code. If your inventions don’t rely on the statistical results to be proved, you should be able to make all decisions without writing a single line of code. Being able to predict the system’s response on paper can make you the the one makes the differences.

Even in professional environments, people tend to start coding as soon as possible. Initial design doesn’t only lead you functional mistakes and a program with tones of bugs, but may blind you to see the problem from different aspects and representing data in a different ways to fasten the system. Only good estimators can code good software. Therefore, I’m not interested in which new fancy technologies you know, far more interested if you have a wide range from top to hardware, able to see the bottlenecks and improve them on paper. Then, any developer can write the code.