We had to rebuild the VCAPS system
at Ford. One motivation for this was the slow speed of the system, we had to meet a specific timming. Management
wanted 15 days, but would accept 30, normally we made it in about 45.
Now we needed to find someway to
move our data from the old system to the new. We created what we called the Replicator. A program in the old database
serialized out to a file. A parser in the new database read the file and rebuilt the database.
The Replicator that was originally
built was very complex. It had a great deal of flexibility and was built as a framework. The parser portion was
both complex and difficult to change. We also had a problem with the Replicator being slow. We had a requirement
to move the data overnight, but the Replicator was going to take a day maybe two.
As the new database representation
was being implemented it often required us to change the Replicator. But it was too complex making it hard to change
and slowing us down. We had to have someone full time just to maintain the Replicator and attempts at optimization
were not going well. When ever the team needed a change to the Replicator we had to wait till the "Replicator
Guy" could do it.
The only thing to do was get the
team together to design a better solution. As we worked through various ideas we came up with a good one. Why not
use the parser that came with GemStone itself? |
We also came up with another idea
for simplifying the whole thing. Our old database was a complex graph, but our simpler representation in the new
database was strictly a tree. We decided to treat the old database as if it was a tree also. We could then do a
depth first traversal to both write the file and read the file. This further simplified our design.
We did a spike solution to see if it would work, and it did.
We wrote automated unit
tests as
we were writing the Replicator so that we could refactor out complexity when ever possible. We didn't need a parser and the simpler
design reduced the code base even further. When we finished the new design we had about 1/6 as much code.
This new system didn't have a framework
like base. It just did what it needed
to do
and no more. We found it to be much easier to maintain and much easier to add just what we needed as we needed
it. We found that we could collectively
own it
as well. Now anyone could change it as needed.
As an added bonus the simpler system
ran in a few hours instead of days. We had left our optimizing until last so this was a bonus indeed. We used a profiling tool to measure
the performance and locate the bottle necks. Because we had built the system simply with lots of little pieces
we further reduced the run time to an hour with only three days of work. 
 
Experience Software |