RELEASE 03

One theme, two lenses. Tom Barber on the contract nobody signed when a report went live on a warehouse table. Amelia Prasad on the cost of unclear data presentations.

{{current_date_full}} · Read online

Welcome to the Concept to Cloud newsletter. One theme an edition, two takes on it, ours. If it would not earn a place in our own inbox, it does not go in yours.

This edition is about what happens once people depend on your data. Tom takes the table: the unwritten contract it picks up the moment someone reports on it, and how to change it without breaking everyone downstream. Amelia takes the presentation layer: what unclear data presentations cost. Grab a coffee.

By Tom Barber

We've all been there. You open a report, and the numbers you've relied on for the last 12 months are nowhere to be seen. The screen just shows a bright red warning saying the data it's looking for can no longer be found. Then you cast your mind back to Tuesday of last week, when someone said, "Does no one still use this table?" and everybody in the office shrugged. Now you know the answer: yes, they do use that table. This is the story of the moment someone builds a report on your table. The table becomes a contract.

In any data warehouse, there is no formal legal contract, just a contract formed by use: people building on the data you provide, in the shape you currently provide it. That lets people access the data readily and report on it regularly, either self-serve or on behalf of other people in the organisation.

Inside this data, we expect:

None of this is likely to be written down anywhere, but it is the structure this table has held for long enough that people trust and understand it.

Data inaccuracy means a number of different things:

Bear all of this in mind when you think about the contract between table and report. It isn't only reputational damage. The financial repercussions could be massive, including fines and penalties for leaking the wrong data. This can cost companies who have incorrectly reported the data anywhere up to 15% to 25% of gross annual revenue.

I think over our careers, we've all fallen foul of skipping the required procedures when making these changes. That might be defining a schema boundary, or putting views on top of underlying tables to protect their integrity. Sometimes speed is the reason, sometimes uncertainty or ambiguity, but whatever it is, as data engineers we have to put best practice in place so these mishaps don't happen again.

The real issue with any of these reporting mishaps is that we often find out at the worst possible time: when the data is due to be delivered, consumed and accepted by whoever it was meant for.

Sometimes you don't find out until much later, which is worse. You might find out via query logs or lineage tooling, or just a helpdesk message on a Thursday afternoon, when someone says they can no longer get to the report they've looked at every Thursday for the past 6 months. So the question is: what do you do about it, and how do you fix it rather than sitting there with your head in your hands wondering what comes next?

So how do we overcome the missing contract between table and report? Well, documentation is as good a place as any to start, and it can live in multiple places:

Put the best practices in place: a staging environment, and the ability to make changes without breaking downstream reporting.

When a change request comes in for a report, test both the data and the report in staging before deploying to production. When a production upgrade happens, make sure every stakeholder knows about it and understands the release notes, so they know what's coming.

This is all super important. And the feedback from those users shouldn't be a shrug and a tacit acknowledgement. It should be actual testing, to make sure your changes work not just for that report, but for anything else near it.

If you are going to change a table, how do you do it without breaking too much, or producing change sets so large that no one can be bothered to read them?

The answer is sort of "be careful," but in reality it's to be additive only by default. Try not to remove stuff, even if that means legacy columns sit there for a while until you're sure nothing still depends on them.

Use parallel columns or a dual write window, and a deprecation clock with a date on it, so people know when things are going away. Then make sure the people who care about that data (the stakeholders and users who consume it) know, before that clock starts and before that window shuts, when their data is going to go missing. That way, they can either adjust or argue their case for why the data should stay in its current form in your warehouse.

So what does this give you? A far more resilient warehouse, one that can accept change without destroying existing reports. Changes get documented and broadcast to everyone who consumes the data. And you get a runbook, something you can pick up each time, that lets anybody on the team make those changes in a way that's consistent with everyone else in the organisation.

By Amelia Prasad

One of my favorite things to design is a dashboard. It mixes usability, data, statistical methods, everything, all neatly tied in a bow, presented right in the face of your user, every day, slowly becoming a pillar of the landscape. Dashboards are where it's at. With this kind of visibility, they seem simple, but they are critical for a business in ways seen and unseen. Numbers are neutral, but these numbers communicate. Data tells a story. Tell that story wrong, and the messaging signal in your company turns into a game of telephone, the message warbling a little more at every hand it passes through, until the far outreaches of the company are working from a version the source never told. Show people the wrong picture and they will spend the quarter asking the wrong questions, confidently, with data to point at.

Edward Tufte defined graphical excellence as what "gives to the viewer the greatest number of ideas in the shortest time with the least ink in the smallest space." A dashboard that clears that bar is not a reporting artifact. It is the speed at which your team thinks together.

A dashboard does not just report on the work. It frames the questions your team, and anyone who sees it, asks going forward. Here is the part that matters to a leader, and it has nothing to do with charts. Most people in a business do not understand the implication of numbers in table form, the more granularly you need to digest data, the more you lose the big picture, the goals, the objectives. If you can't touch base and view that data on a regular basis, it has a human impact. There's loss of motivation, direction, and outcome. The worst part is, get these dashboards wrong and everyone in the company can suffer from it, not just those that asked for the query.

So let's go through some of the easy traps, how to fix them, and most importantly the business impact from a longevity perspective.

Your dashboard has myopia. A dashboard can be accurate, readable, fun to interact with and still lie by omission, giving people a sense of a reality that may be untrue. Your dashboard may reflect all the things that are negative, but leave out what's neutral, what's positive. Context matters, not for a single panel, but a user can derive their own questions from tying one dashboard panel to another. Make information available to them, so they can have the perspectives to push further.

Have some standards. One of the things that drives me nuts on a dashboard is when all of the charts show different time scales and it's not clear. If you have a robust data team, it may not happen, but it still might - understanding data isn't understanding how to tell a story. Make sure the surfaces you tell that story on give the same signals, for the same time frames, for the same region, for any other axes that are relevant.

Too much information. Progressive disclosure is a fantastic theme in user experience design. You don't want to overwhelm your reader, you want to meet them where they're at and give them the tools to dig deeper where they may. This often comes down to interaction design and a dashboard that expands and contracts in a clean way, keeping the focus clear at every turn. But why are they digging deeper? Because they are looking for an answer and the data connects it. Have the information ready, but hierarchical if possible.

What's the impact of getting these things wrong? Sending the wrong signal muddies the thinking from the getgo. In an age where we rarely have watercooler talk anymore, those misunderstandings can carry themselves much longer and have a greater impact on the frustration, direction, and contributions of your teams.

I recognize that this is a vague reference, as different teams interact differently, but when you're creating platforms made to serve a number of different users and you can't predict where the next genius idea in your company will come from, getting the dashboard right is a critical structural piece to not miss. It's just another form of visibility, transparency, and continuity that isn't loud, just steady.

FREE THIS MONTH: THE PRODUCT BLINDSPOT REVIEW

Before the paid launch we're giving away a handful of Product Blindspot Reviews. A working session with our product, design and engineering leads, plus a short written read on what is holding your product back and what we would do about it in the next 30 days. Normally $1,200, free for the launch cohort, in exchange for honest feedback.

Concept To Cloud · product engineering for regulated and PE-backed companies · concepttocloud.com

You are getting this because you subscribed. Unsubscribe · Preferences