Home » Data lineage » Lineage first – or context is king
Neon spaghetti 2 HIGH RES scaled

By Philip Miller, Co-Founder and Chief Innovation Officer

google 1

It’s always fun to see how Google completes a search with suggestions.

In this case, it gives me a lot of suggestions that make sense. Indeed, even when I played with my cookie settings and made my VPN go in from different countries etc, the result was that there were nearly identical answers all the time. Go on have a try… I’ll wait…

Same or similar? I thought so.

Let’s try another – ‘Rishi Sunak is…’

This time we have some real differences, different cookie settings probably determining those differences.

This one has no personalization…

google 2

Whereas this one does have personalization…

google 3

The point is that Google works with context – every response is somehow personalized. It’s very clever and designed to give you a better experience (even if some might disagree) by giving you a tailored result set in line with your previous [search] history.

Let’s examine what this actually means and get a couple of definitions of ‘context’.

Noun: context; plural noun: contexts

(OXFORD LANGUAGE)

  • The circumstances that form the setting for an event, statement, or idea, and in terms of which it can be fully understood

(OXFORD UNION)

  • The situation within which something exists or happens, and that can help explain it

So it’s pretty clear – context informs your understanding of a word or thing.

The most famous example is probably ‘eats, shoots and leaves’. Given the context – in this case, the presence or absence of a comma – it could be referring to a panda or… well, something quite different altogether. Context-free, the phrase is ambiguous at best and at worst misunderstood.

Today I asked a friend to be ‘present’. They could have taken that to mean turn up somewhere or for it to be a state of mind – I hadn’t been clear enough, so the meaning was lost at a moment I needed it to be obvious.

Assuming context can be a costly error.

This leads me to one of my favourite phrases: ‘context is king’.

This perfectly distils the reason that Solidatus exists:

Context (such as where, who or when things are, or what or why things did) becomes at least as important as the data used in most cases. It is how we communicate what form an activity takes.

Or put it another way: establishing context is critical to understanding the meaning.

The Solidatus way

All of this is scene setting, though, a form of exposition to bring us to the reason I’m writing, and that is to explain our philosophy at Solidatus.

We work to the principle that establishing context is the single most important thing you can do for an organization. It is exactly how we learn language as infants – a word is just that: a word. Learning a definition of a word without other vocabulary is different from learning it when we can express ourselves properly. The first words we learn are defined by the context in which the word is used: Mum, Dad, food, toy etc.

It is the same with data, people, and processes. We can follow what a person is doing, establish the context and make a guess as to what they do. We might be right or wrong, but the more context we can establish the better our guess will be. Informed guesswork is different from proof.

Lineage first

We call this process lineage first.

We map out a connected journey of a piece of data with as many of its interactions as we can find to elegantly expose purpose and use. That journey will include many elements, most certainly the structures of the organization – whether physical, legal, geographic or resource – as well as inputs, outputs, and the reason why this is there in the first place.

In doing things this way we put the journey itself at the centre of an organization. Things are not there for the sake of them, but because they are justified to be there. They are defined and understood in context. The rules of their use and their actual use (which should be the same – but not always are) are mapped.

High quality becomes a product of the system, not an impossible goal that needs to be rectified later. The system itself becomes self-describing and designing, and waste is illuminated and then eliminated. Duplications of functions are unnecessary as the organization understands how it operates.

Best of all, this whole process is both democratic and close to the point of change. People own what they should own, and they have a licence to operate in a way that in the past they would not have had.

It isn’t reinventing the wheel; it’s taking a step back from a bureaucratic abyss where context is governed without context (or experience) far away from the point of implementation.

As my amazing colleague, Lorraine Waters, put it, there are 3 root causes of data issues

  1. Poor solution design
  2. Poor system controls
  3. Poor change control

As my favourite literary detective might have put it if had he been an information architect, when you eliminate the avoidable, what you are left with is a properly designed system.

Lineage first!

This is the first in a series of blog posts on the theme of ‘lineage first’. Look out for future instalments, where we’ll go on to discuss lineage as reality, and lineage as a planning (and impact analysis) tool. We’ll finish with a review of the features to look for in a data lineage tool.

Download your A-Z Guide

Read our definition of ‘data lineage’ along with many others in our A-Z guide.

Written by: Philip Dutton

Co-Founder & CEO at Solidatus
Philip is a Senior System Architect and Project Manager with over 20 years’ experience within Financial Services.