I’m doing it wrong…

At some point in your career, you have to realize that you’re going about it in the wrong way.   It may hit you like a ton of bricks, or it might be a subtle realization, but either way you realize that things aren’t working out for you like you expected.  I’ve had a couple of those moments throughout my career; one was shortly after I flunked out of graduate school.  Nothing says “you’re doing it wrong” than sitting outside of your advisor’s office for a meeting that never happens.

I’ve had other epiphanies in my career, such as the time when my ethical standards were a little higher than my employers; when I got sent home by a GM after a discussion over my responsibilities, I started polishing my resume.   I was doing it wrong by working for the wrong company.

Recently, I’ve begun to realize that I’m not living up to my full potential in my career.  I’ve spent the last several years building an enterprise solution for my company that has become the core product of that company.  It’s a good product, and I’m proud of it.  However, like many small companies that have grown up fast,  our company is built on a complex ecosystem of ever-changing goals and feature requests.  We built a system based on assumptions, and we’ve become one of the leaders of our industry because we’re often the first to deliver a product for a niche market.  Many of the assumptions we made didn’t pan out, and the applications we’ve built have slowly degenerated into a mass of tangled wire and unrealistic expectations.  I realized this as I’ve struggled to add a new feature and retrofit it into this existing solution; it’s taking more and more time to solve development problems because we’re not sure what features are still being used by some employee in a dark corner of the building.

As I was rewriting a stored procedure for the fifth time trying to eke out a few more milliseconds of performance, I realized that I was thinking like an engineer.  Engineers find creative solutions to problems in a very hands-on way; they worry about wiring things together so that they work, and they work well.  Engineers are worried about the microcosm; as every geek’s favorite engineer (Scotty from Star Trek) would say “In four hours, the ship blows up.”  That’s pretty straightforward; under condition x, outcome y is to be expected in a certain amount of time.

The problem?  My title says Architect.  I’m supposed to be thinking about the big picture, not just how a couple of applications are wired together.  I’m supposed to understand (and enforce) the rules about how events become data, and how data becomes information.  I should be more concerned with defining the specifications for our system than trying to figure out this damned stored procedure (for the fifth time).  Maybe we shouldn’t even have this particular stored procedure; maybe with a little tweaking, we could eliminate the problem altogether.

So what does this mean for me?  Well, as part of my New Year’s resolutions, I’ve been determined to learn something new every month.  This month, I’ve been focused on what does it mean to be a Data Architect, and I’ve been trying to find a little time every day to transform myself from an engineer to an architect.  I’m not going to master all of these subjects at once, but here’s my working list (from high-level goals to specific action items).  I expect this list to evolve, but it’s a start.

High Level Goal: A Data Architect needs to establish the standards for information and data in the enterprise.

• I need to document the information architecture of our division of the company, using a standard data flow diagram notation.  I need to spend some time daily refreshing my memory on that notation.
• I need spend time with employees throughout the organization, discovering what the business entities are, and what the vocabulary for those entities are. 
• After discovery, I need to publish a standard vocabulary document and data-dictionary, showing how we capture that information today:
  • I need to propose changes to our business vocabulary, and
  • I need to propose changes to our database schema to standardize our notation.

High Level Goal: A Data Architect needs to understand the nature of the enterprise’s information on all levels: physical, logical, and procedural.

• I need to talk to our production DBA’s an understand how our database servers are set up physically, including the clustering structure, the drive arrays, the SAN, etc.
• I need to talk to our engineers to understand how data gets to the databases.
• I need to talk to our product owners to understand what information they want from the data, and what’s the best way to deliver it.

High Level Goal: A Data Architect needs to recommend the best architecture for information management, including a plan on how to get there from here.

• I need to refresh my memory on all aspects of SQL Server, not just the parts I use on a daily basis.
• After discovery, I need to recommend ways to improve efficiency in our data capture processes.
• I need to listen to all voices in the organization, even those I don’t normally agree with.  I can’t afford to throw away good ideas simply because I don’t always like the originator of those ideas.

More to come, but this is what I’ve been working on so far this month (February 2011).