Saturday, March 26, 2011

Reviving data from the dead

Part of my job requires that I build websites for clients that have extensive and severely damaged and dying (or dead) member databases.

I build website front-end (user experience) and back-end (office experience) using a sophisticated content management system that's integrated with office management functionality (member data, accounting, e-commerce, etc.).

During the web development process for any new client, I have to extract, revive and clean up member data that's been damaged to death, rendered useless and decomposing in the client's previous web format (in fact, the main reason clients come to the company I work for is because they need to resuscitate their data).

My job, then, is to "Frankenstein" dead data with a jolt of new life. Doing this requires roll-up-your-sleeves, tedious gruntwork that takes hours, days or even weeks, depending on the size of and damage done to the database during its former life.

And, naturally, looming deadlines add to the intensity of the job. Plus other projects get shoe-horned into the mix, so the data clean-up process seems endless. And, ultimately, no matter how hard you scrub the data, when you go to integrate it into your new system, it'll come back riddled with heart-stopping errors - like anything you submit for the first time.

The whole process is akin to the proverbial "Sword of Damocles," with an impending doom that not only consumes your day-to-day existence but has a nasty tendency to infiltrate your dreams. During the development process, my dreams become vivid metaphors of anxiety such as: driving an out of control car; being lost in a thick and unfamiliar forest; being surrounded by faceless humanoid creatures; losing my spouse; losing my dog; and, yes, losing my job.

With this kind of internal psychopathy that permeates coupled with the fact that I've done so much new client data clean-up and web development, you'd think I'd have a "work smarter" game plan in place.

And I think maybe I do (that is, once I put my current project gently to bed).

It used to be: 1) get the design in place and 2) scrub the data.

From this point on, it'll be: 1) scrub the data and 2) get the design in place.

What's the difference? Actually it's based on the old adage to do the most painful stuff first. That way, the rest of it doesn't seem so bad... 

No comments:

Post a Comment