Datasource is $format, so let’s make a giant $format parser instead of a business app!

TL;DR: It takes a LOT less time to analyze requirements and find domain boundaries for data access that will guide the creation of simple, business-oriented abstractions that are maintainable and testable than it does to fix a leaky, broken API later on!

Full version: I’m code reviewing a coworker’s changes to a new-ish part of back end infrastructure we use for content indexing, and I’m just realizing that the original author accidentally turned the whole thing into an XML parser because every class has to understand what an XElement is since that’s what’s passed throughout the object graph to get data. After a bit of analysis it turns out there are only a few data access use cases in the whole application: content id, language, image URLs, and publish date. The original author even created XElement extension methods with handy domain names for accessing that data in a utility class. BUT IT’S HARDER, UGLIER, AND LESS TESTABLE TO MAKE AND USE EXTENSIONS THAN IT IS TO JUST MAKE A ‘ContentItem’ CLASS WITH THOSE PROPERTIES!!

I hate it when people do that. When your library/framework/API leaks an entire library/framework/API that lies underneath it, you’re basically serving up a huge plate of fail to everyone who needs to use your stuff.

Another recent example: I had to access a report system a few gigs ago where underlying source was a large collection of Excel files in OpenXml format. Instead of plotting data access boundaries after analyzing the report requirements and creating a reasonable API, the programmer in charge of the system just made a really slim wrapper around the OpenXmlSDK that forced anyone who wanted report data to learn enough SpreadsheetML to be really dangerous.

When I got involved and saw the mess the data access API was, I looked at the existing reports and figured out that there were only a few types of data that were needed and a handful of structures and arguments required to fetch it. I wrote a domain façade for the API and gave friendly names to those core data access functions and then nobody on the report team needed to know SpreadsheetML anymore. The report system developer found out I’d done that after he added code to his API to process charts (ChartML about doubled his API size) and he flipped out because he felt 1) like the facade was hiding the “real power” of the report system API, and 2) his 200 new chart data access methods looked really ugly bolted onto the tiny façade.

It took all of 15 minutes to look over the actual consumption-side requirements for chart data with him and show him how to add a method like List<int> GetChartValuesByDay(DateTime start, DateTime finish) to the façade and totally blow his mind. He stared at what we’d done for a minute or two in silence before jerking his head up. “Oooooooooooooooooooh, that’s why nobody wants to use this thing,” he said. The next sprint he wrote a new API that exposed the façade directly and made all of the OpenXml stuff internal, and suddenly all of the report developers started coming up with new ideas for data gathering because it wasn’t a nightmare to use the API anymore. Turns out the “real power” of the original API was in preventing people from making productive use of the data.

Why do we have to learn this stuff over and over again?!

This entry was posted in Architecture, Coding. Bookmark the permalink.

2 Responses to Datasource is $format, so let’s make a giant $format parser instead of a business app!

  1. Rick says:

    Yea, but can you sing or dance?

  2. Jesse Jacob says:

    Rick! You KNOW I can’t sing or dance! MUST YOU MOCK ME SO?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s