A Simple Design is Easier to Optimize

We had to rebuild the VCAPS system at Ford. One motivation for this was the slow speed of the system, we had to meet a specific timming. Management wanted 15 days, but would accept 30, normally we made it in about 45.
Now we needed to find someway to move our data from the old system to the new. We created what we called the Replicator. A program in the old database serialized out to a file. A parser in the new database read the file and rebuilt the database.
The Replicator that was originally built was very complex. It had a great deal of flexibility and was built as a framework. The parser portion was both complex and difficult to change. We also had a problem with the Replicator being slow. We had a requirement to move the data overnight, but the Replicator was going to take a day maybe two.
As the new database representation was being implemented it often required us to change the Replicator. But it was too complex making it hard to change and slowing us down. We had to have someone full time just to maintain the Replicator and attempts at optimization were not going well. When ever the team needed a change to the Replicator we had to wait till the "Replicator Guy" could do it.
The only thing to do was get the team together to design a better solution. As we worked through various ideas we came up with a good one. Why not use the parser that came with GemStone itself?
We also came up with another idea for simplifying the whole thing. Our old database was a complex graph, but our simpler representation in the new database was strictly a tree. We decided to treat the old database as if it was a tree also. We could then do a depth first traversal to both write the file and read the file. This further simplified our design.
We did a spike solutionto see if it would work, and it did. We wrote automated unit testsas we were writing the Replicator so that we could refactor out complexity when ever possible. We didn't need a parser and the simpler design reduced the code base even further. When we finished the new design we had about 1/6 as much code.
This new system didn't have a framework like base. It just did what it needed to do and no more. We found it to be much easier to maintain and much easier to add just what we needed as we needed it. We found that we could collectively own it as well. Now anyone could change it as needed.
As an added bonus the simpler system ran in a few hours instead of days. We had left our optimizing until last so this was a bonus indeed. We used a profiling tool to measure the performance and locate the bottle necks. Because we had built the system simply with lots of little pieces we further reduced the run time to an hour with only three days of work.


Experience Software

ExtremeProgramming.org home | XP Lessons Learned | Simplicity | Email the webmaster

Copyright 1999 J. Donovan Wells all rights reserved.