Home » Data lineage » Page 2
New and views HIGH RES scaled

Although our software was born in London, we now call many of the world’s great cities home. Earlier this year, we set up shop in Houston, Texas to meet the demand for next generation data management solutions throughout North America. Since then, we’ve attended a myriad of events across the continent and have welcomed many new colleagues that share our passion for reimagining data management.

Last week, three of those colleagues – Solidatus’ Global Strategic Account Director, Mary Anne Bullock; Advisory Solutions Engineer, Kevin Shannon; and our Director of Business Development, Glenn Aluce – joined our CEO, Philip Dutton and our Director of Analyst and Industry Relations, Ashlee Dutton at A-Team’s data management summit in New York City.

ashlee

We listened and learnt from those who attended, absorbing their insights, and suggested solutions to their complex problems. Ashlee Dutton commented on her experience at the summit: “There is a change happening across many industries. When it comes to an organization’s data strategy, they’re now shifting from defense to offense. Data is more than just another thing to report on – it’s influencing how the C-suite are steering the direction of their businesses.”  

The theme of the event was centered on leveraging data to drive business and compliance insight – topics that Solidatus regularly speaks to. Addressing a packed venue, Philip Dutton joined the Chief Data Officer at Bank of New York Mellon (BNY Mellon), Eric Hirschhorn, on stage to deliver a keynote about how our data blueprints are being used for knowledge, context and better decision making. BNY Mellon is the world’s largest custodian bank, managing $43 trillion of client assets. Hirschhorn played a pivotal role in the bank acquiring the Solidatus solution to transform their data management program. 

eric

To summarize Philip’s and Eric’s key points: 

  • Change is inevitable, preparation is paramount. 

“The data world has changed so dramatically, so why hasn’t our approach or tooling? This is why we developed Solidatus – to deliver a solution that provides unique capabilities within data management to enable a fundamental shift in thinking and approach. We’re enabling organizations to create living and dynamic blueprints of their data, allowing the business to simplify the complexity and give context allowing quicker and better data driven decisions,” said Philip. 

  • Everything is connected. Data doesn’t exist in a vacuum.  

At Solidatus, we know that everything is connected, and that there is a global demand for active metadata: the continual visualization of systems, infrastructure, processes and critical data elements. Data interacts and impacts an organization’s entire ecosystem, so it must be modeled as such. Data’s WHY is more important than the WHAT or the HOW. No matter how far along you are on your data journey, you need to know where you are to understand where you want to get to. Solidatus allows you to see the data held across your enterprise architecture. Philip added: “Let’s not continue to boil the ocean, let’s start with the kettle.”

  • Building a strong data foundation unlocks value across your business. 

Eric talked about the immeasurable value that will result from implementing Solidatus. With Solidatus, he and his colleagues can understand the flow of their data for regulatory filings; ensure their risk picture is correct; trace the journey of their market data; and comply with jurisdictional concerns around where data is stored. Having a unified view of their data landscape is vital for building new products and services, and to ensure their data is timely, complete, accurate and consistent. These are just a few of the ways in which Solidatus can be used. Departments across BNY Mellon are planning on using our software for multiple different use cases. The goal is to move from a siloed to an enterprise data management governed view.

IMG 1999

Soon after Philip’s and Eric’s keynote, Solidatus took the stage again. Mary Anne joined leading data practitioners and innovators on a panel titled, “Shifting to agile data management practices to ensure enterprize-wide data access, analytics and visualization.”

Mary Anne made several key observations:

  • The main benefits of data democratization are timely and equitable access to data. 

When users across your business have access to data that is held in context, numerous other benefits span from it. These include:

– Accelerating transformation programs and product innovation. 

– Delivering rapid insights with confidence and trust.

– Reducing expenses associated with manual efforts and subject matter expertise.

– Building a sustainable knowledge base – once connections are built, they are retained throughout the life of the project.

  • Traditional data management development approaches fail because of rigidity.  

An agile approach to data management stops organizations churning through multiple system implementations which lead to uncoordinated reporting and analytics because of tight coupling with the systems that are being deployed. Most existing data functions are governance- driven, which has put a burden on data producers to maintain various catalogs and data asset inventories, along with various sets of complex business rules.

  • To create value from data you need an active data culture. 

To get businesses to think of data as an asset, any policies, procedures, and governance structures must be supported by a culture that encourages open communication and a common understanding between those that prepare and process data sets, and those who consume that data for analysis, insights and reporting. Adopting a commercial mindset is fundamental to setting up successful data functions.

We believe that Solidatus holds the keys to a new frontier of data management, so we were delighted to be recognized as such by our peers by being awarded A-Team’s Best Data Discovery and Catalog Solution at the summit.

CL6A0057

Get in touch to find out how we can help you usher in a new era of data decision-making

Find out how you can create a living blueprint that can provide you with the knowledge and context for better-quality decision-making, robust impact analysis and reduced program time, risks and costs.  

Latest news

Neon spaghetti HIGH RES scaled

Solidatus’ CIO delivers a session on what data culture should really mean at Big Data London, wins A-Team data award in New York and meets hundreds of attendees at their event booths

Big Data London – the UK’s leading data and analytics conference and exhibition – opened its doors on the 21st of September. Over two days, Kensington’s Olympia London played host to over 100 leading technology vendors and consultants, and tens of thousands of attendees.  

Big Data London is a special time of the year for the team at Solidatus. It’s a chance for us to celebrate our industry and the impact data is having on businesses across the world. But more importantly, it gives us an opportunity to connect with our customers and partners, and a chance to escape digital meetings over Zoom! 

big data london booth

The march towards an understanding of the complexities of data and how to manage them is a challenging and exciting goal – one that Big Data London’s esteemed speakers held in their sights across the dozen theatres that hosted their sessions. Our Chief Innovation Officer, Philip Miller, was one of them.  

philip miller speaking at big data london
philip miller presenting at big data london

Data consumers are found in every function and at every level of the organization. Consequently, vast amounts of data are collected with little context as to its origin or meaning. Addressing a full house, Miller spoke of the importance of building a data culture within a business to ignite the fire to fuel innovation.

“The data world has changed so dramatically, so why hasn’t our approach or tooling? This is why we developed Solidatus – to deliver a solution that provides unique capabilities within data management to enable a fundamental shift in thinking and approach that allows businesses to simplify the complexity and give context for better data-driven decisions… Solidatus is the golden thread that connects your data to the processes that create it, to the policies that guide it, and the obligations that regulate it,” said Miller.  

In an uncertain world, it pays to understand how data moves through an organization. Miller went on to say, “We may not have much control over events in the world that drive the need for resilience, but we can position ourselves to make the most of necessary change and view our organizations with new perspectives.” 

Part of building an active data culture is about understanding the context in which your data is used.

“Our philosophy is that everything is connected. It goes beyond just seeing how data flows from application to application to how it then eventually ends up on a report. Our software is a tool by which organizations can gain clarity over their entire ecosystem and understand WHY their data is being used and how it is connected,” Miller added.

Speaking a common language across a global business is hard. But if you’re to succeed in the coming months, years, and decades, then embedding data literacy into your organization is essential.

“If we want to increase data literacy and provide an up-to-date and dynamic self-serve environment to all your data consumers, you need to move away from dependencies on countless systems, people or tools when you try to understand how data is used across your organization. A Solidatus blueprint is the Rosetta Stone that allows you to build the common language that enables an active data culture,” said Miller.

Throughout his presentation, Miller reiterated the importance of data in constructing powerful business strategies, and how by using a tool like Solidatus, you bring an agile, data-centric approach to decision- making.

For an audience at Big Data London, it may seem obvious, but Miller reminded us that today, more than ever, the board of a company has less and less in the way of excuses for why they have not met key data regulations. Chief Data Officers and their immediate staff are not the only people who should be concerned about data; everyone must be a data steward.

Watch the full session on demand:

As our time at Big Data London came to an end, across the Atlantic, Solidatus’ CEO, Philip Dutton, and Chief Data Officer for Bank of New York Mellon, Eric Hirschhorn, were delivering a similar message at the Data Management Summit in New York. Dutton and Hirschhorn spoke of breaking down silos across your business to build an enterprise data management-governed view, as well as the importance of ensuring your data operations matched your data aspirations.

eric
IMG 1999

To finish a productive and awarding week, Solidatus took A-Team’s Best Data Discovery and Catalog Solution at the summit!

IMG 2026

Latest news

IMG 1999
22.07 000004 background scaled

Good data governance includes a set of robust processes, policies and standards that ensure the efficient and effective use of information. It stipulates who is responsible for data, how it can be used and how its quality is assured. Part of NOW: Pensions’ newly defined Data Governance Program included creating a new business data model to improve and organize their data structure. Once built, the model was then enriched through defining a set of data quality rules that prevented the uptake of bad data, as well as measuring the quality of the data that was already held by NOW: Pensions. 

The team at NOW: Pensions identified three main challenges:

The ‘as-is’ data wasn’t clear. 

  1. A business data model was needed to guarantee that the right data was available to users and that there was one source of truth. 
  2. Data quality rules were required to measure current data quality and to stop bad data re-permeating systems. 

Key to addressing all these challenges was achieving an understanding of where data originated and what data flowed to where. 

How NOW: Pensions used Solidatus to address their key challenges

Data Distress: Is the Data Office on the Brink of Breakdown?

Mapping data lineage

NOW: Pensions used our powerful data lineage functionality to map their ‘as-is’ data lineage from its point of origin right through to the data warehouse. This gave them an accurate snapshot of how their databases were structured and provided cataloguing of the technical metadata for every column within each individual table, such as data type and field length. Individual source system models could then be joined to show the transitions from the source systems through the staging databases to their data warehouse.  

Creating a Business Glossary

NOW: Pensions created a business glossary to provide a common language for all the terminology within each source system. Solidatus Reference Models allow subject-matter experts (SMEs) across the organization to record the terms their area uses; these can then be shared, centrally collated, and disseminated across the business to ensure consistency of understanding and eliminate confusion.  

Building a Business Data Model

Once a new business data model was created within Solidatus, the source systems could then be mapped to entities within the lineage models so that the data architects and developers were confident with what each of the column headers referred to. Business rules could then be defined to ensure that all the data fields in the new model were correct even though the data came from a variety of systems. 

NOW: Pensions is using Solidatus as their central data governance tool. This has resulted in:

  • A single workflow for managing collaboration among users and facilitating the effective development of Models.  
  • ‘As-is’ models providing a clear view of where the data entered their systems, which was invaluable in identifying where to execute Data Quality Rules. 
  • A complete view of how their entire data landscape is structured. 
  • A common language for their terminology. 
  • Full transparency across their data landscape. 
  • A Business Data Model built with the confidence that only correct and timely data can be viewed with the proper business rules in place to consolidate various sources.

As part of NOW: Pensions’ ongoing Data Governance Program, the company will soon be mapping access rights to each data view and using it as a centralized access control log.

“When we started our data governance project, we did not fully understand how our data was produced, transformed and, ultimately, used. We hadn’t mapped our data’s lineage or populated a data dictionary. Now, all that has changed. With Solidatus, our data estate is now mapped, modelled and catalogued. In a single view, I can show the business where their data resides, how it flows through systems and applications, what data quality rules apply and what data is subject to GDPR. Solidatus is central to the way that we govern data.” 

Phil Yeoman, Group CDO, Cardano and NOW: Pensions

Get in touch to find out how we can help you deliver on your governance program.

Find out how you can create a living blueprint that can provide you with the knowledge and context for better-quality decision-making, robust impact analysis and reduced program time, risks and costs. 

Latest news

photo 1505664194779 8beaceb93744

Solidatus Co-CEO Philip Dutton recently caught up with Baz Khuti, President US Modak, about the difference between data and metadata, data lineage, and its role in the cloud space.

First up in the discussion was the somewhat grey area of the difference between data and metadata. A fan of analogies, Philip drew the perfect comparison:

“The Library of congress has 164 million items. How do you find the information you’re looking for? You do that by going to the index which points you to the right area. That is the metadata – describing where it is. And what it is. It gives us more classification and categorisation to help us determine if this the information we are looking for. Metadata gives the ability to find and understand the data you are looking for. If we move from a library to the internet, we can see that in this instance the data is the internet. We have trillions of webpages. How do you find what you are looking for? You go to a search engine which has indexed the metadata. That really is the crux of what metadata is – it’s the description of the data we are looking for.”

Philip and Baz went on to chat about important key points regarding lineage and cloud space:

  • The role data lineage plays in metadata – it is all about connectivity. Lineage is the hyperlink between documents, the connection between data sources
  • The opportunities and challenges of moving to the cloud – it generates infinite possibilities, but it’s important to not replicate mistakes from the past
  • Cloud gives us greenfield opportunities to build better by default and by design – but we must first understand our data before it is moved

Watch their interview for CDO Magazine on-demand now to learn more and look out for the next segment on regulations and the responsibilities organizations bear when it comes to compliance.

Latest news

52211883534 7fe30b9955 o

Last week, NASA blew the world away with the first images from its James Webb Space Telescope. Promising to far exceed the wonder and quality of its predecessor, Hubble, the JWST has delivered a ground-breaking new view of our universe. These images have told the story of “the hidden universe through every phase of cosmic history” in a way that has never been seen by the naked eye of man.

A “singular and historic moment”, the JWST has and will continue to explore the depths of our solar system, as well as what lies beyond it and “the most distant observable galaxies in the early universe.”
Scientists believe that the data derived from this will answer questions mankind has asked for millennia, as well as questions we do not yet know that need to be asked. It will help us better understand neighbouring planets, the universe and our own place within it. The possibilities this opens up are, for us, utterly endless. The voyage of discovery that lies ahead will undoubtedly be as immense as space itself, and the value it will bring us as humans – for ourselves and our planet – will change the face of our lives and this world for generations to come.

Parallel data universes

And this got us thinking something a bit crazy – how different is the data of our universe, planets, exoplanets, galaxies and solar systems from the data of a business? Okay, we know this is a bit ambitious, but take a step back for a moment. Forgetting sheer size and scientific importance, there are many parallels to be drawn. The data that builds the fabric of an organisation is vast, rich and seemingly endless. The different facets and ecosystems of that data can be seemingly unfathomable to many trying to grapple with its complexities. Is understanding what exists beyond us in the deep, infinite spans of space really all that different to understanding one’s own data?

Businesses have a mammoth task ahead of them when it comes to data management. At its most complex, data can be messy, confusing and even misleading. But at its best, data can be a powerful force driving organisations forward – not only through compliance, but also innovation. Getting to this point in a data journey is no easy feat, though. It requires data quality, sound data governance and efficient data lineage to fully understand what data there is, how it is moving through an organisation, who it is available to and how it is reported on.

Achieving this requires more than just good will and a few strategically drawn up Excel spreadsheets. For this, you need the James Webb Space Telescope of data management. You need a tool that is going to not just show you your data in clear, HD view, but also show your metadata, how it is all moving, who is responsible and, most importantly, the value it can deliver to your business. Something that will reveal the data and findings you were looking for in a matter of minutes, rather than days, weeks or months.

Professor Gillian Wright, the British researcher who is co-principal investigator on one of Webb’s four infrared instruments said, “Whenever you look at the sky in a new way, you see things that you didn’t expect.”

Discovery is knowledge, and knowledge is power

When looking at your data through the lens of a Solidatus model, you can see it in a new way; in a way you might not have expected. It will answer questions you have been asking for years. It will give you solutions for the problems you knew you had, and those you didn’t. It will answer questions you hadn’t even thought to ask yet. It will take you on a journey through a solar system of data that orbits your organisation, uncovering hidden gems that can change the way you do business. It will tell your story, from inception to current day, showing you the way forward through innovation and creativity.

As the Director of UK Astronomy Technology Centre told BBC News about the James Webb Space Telescope, “…that’s telling you that the discoveries are just sitting out there waiting to be made.”

There are infinite discoveries to be made in an organisation’s data – whether you know it or not. There are new lessons to learn, new ways of working to uncover and boundless benefits not only for your own business, but for your clients too.

Knowing your organisation’s history and story through its data can change the way it operates forever. And there is no better way of knowing this than through Solidatus.

Discover your data solar system and the possibilities that lie beyond with Solidatus. Request a free demo today.

Latest news

22.05 000023 blog image

At a recent keynote speech in London, Solidatus Co-CEO and data lineage Philip Dutton said that “the demon of complexity is the thing we’re all struggling with.” But do the trappings of pivoting towards a more data-driven business really have to be so complex?

The answer to that, quite simply, is no. They don’t. In fact, having the right data blueprint can not only eliminate complexities but also support and accelerate growth. And at the A-Team Data Management Summit, Philip looked deeper into this.

Noting that we may not have control over certain events in the world that can change the very fabric of how we operate as businesses, countries and even people, we can position ourselves to be adaptable to change and even lay foundations for innovation.

In his keynote, Philip discusses:

  • how agility, innovation and disruption can be achieved
  • why this is so important as the digital world continues to transform at a rapid pace, and;
  • who can provide this kind of intelligible, future-proof blueprint

Watch on-demand now to hear his insights:

Play Video

Latest news

22.05 000022 waterfall blog image 1

“If we do our jobs right, we can save the planet.”

Our Co-CEO Philip Miller gave us some serious food for thought at the recent A-Team ESG Data & Tech Summit in London.

In his keynote speech, ‘7 things you need to know about ESG’, Philip talked us through the challenges of ESG as it stands today, how it is constantly developing and why it’s imperative organizations create and implement the right strategy to ensure future compliance.

He raised some insightful points for the audience to take away with them:

  • We need to live and breathe a data-driven culture in businesses, so that ESG becomes part of a holistic company mindset – making it everyone’s responsibility
  • Organizations have to publish a strategy that holds up to public scrutiny as the court of public opinion will come to bear on ESG initiatives
  • As our CDO, Lorraine Waters, said earlier during a panel discussion at the same event, “it’s difficult to model the ever-evolving regulations without the right tools”. The Solidatus solution can give you the control points to get good returns while helping you save the planet

Hear more from Philip by watching his keynote speech on-demand now:

Play Video

Latest news

hand bag scaled

Meeting the requirements of Environmental Social Governance (ESG) criteria marks a commitment to building a more sustainable organisation and ultimately a climate positive company. But with so many standards, disclosures and rating methods, alongside the new data sources required to meet them, how do you understand how data requirements intersect whilst maintaining high-level control?

Using Solidatus, you create living blueprints that map how your data flows, and how it’s affected, as it moves around your systems – both now and at other points in time. A Solidatus data lineage blueprint brings together ESG principles, company priorities, assessment methodologies and data sources and metrics to give a full end-to-end picture of the impact of ESG initiatives.

To comply with regulations relevant to its manufacturing activities, a global luxury fashion house with a cosmetics line has been required since 2020 to produce both its Climate Report, and report on its carbon footprint.

For each report, it has been collating information about people, processes and systems in a series of large Excel spreadsheets – these are challenging to analyse, and difficult to interpret or use to define action points.

The company needed to find a way to streamline and automate its processes, and track progress throughout the year, so it could make adjustments if it was in danger of missing its targets. This is why it came to work with our partner, Ortecha, to help automate its Climate Report, part of its ESG (Environmental, Social, and Governance) reporting suite.

Read the full case study to understand the technical steps Ortecha carried out to structure and explore the contents of the company’s many Excel spreadsheets.

Ortecha used our powerful data lineage functionality to structure and explore the contents of the company’s many Excel spreadsheets

Using our multidimensional data mapping functionality, Ortecha was able to show the company the hidden connections in the data used for their Climate Report and the key data points that could be fed into an automated process. A clear analysis of the company’s Climate Report data was produced. This resulted in:

  • the automation of key data points
  • increased frequency of the Climate Report publication
  • improved efficiency of the report production
  • reduced risk of missing a target and the associated reputational damage
  • a road map for implementing improvements.

Repeating the analysis for the company’s other reporting challenges presents further opportunities for consolidation and automation.

Screenshot 2023 06 01 at 5.05.59 PM

Luxury fashion house automates ESG reporting

A global luxury fashion house with a cosmetics line is using Solidatus to map its environmental impact and automate its climate report.

Latest news

Sand white HIGH RES scaled

From 29 – 30 September, A-Team Insight brought together the US data management community to explore the latest challenges, opportunities and data innovations facing sell side and buy side financial institutions.

On day two of the event, Solidatus Co-CEO, Philip Dutton, joined data experts Dennis Slaterry from EDMworks, Andrew Foster from Deutsche Bank, Olga Maydanchik from Voya Financial and Harpreet Singh from Luxoft on a panel titled: “The power of data lineage to deliver compliance and business insight”. A wide range of key challenges were covered from managing ever increasing data volumes to the mainstream adoption of cloud.

The panel started with each member stating their definitions of data lineage, with Philip adding an important point: the power of lineage transcends technical flows.

As a panel, we’re trying to educate the community that lineage enables a lot more than just the technical pipes where the data flows – it brings in huge amounts of impact analysis and transformational change to an organisation. There is an additional layer of value which can be utilised when we start to think about lineage in a different way than just the purely technical sense.

Play Video

The term automation means many things to many people – Philip believed that its definitions in the market are misguided.

Unfortunately, automation takes time, it takes understanding, and it takes a lot of effort. It really comes down to the ‘why’ – what’s the value that we’re generating? The state of automation that most people are looking to achieve is if a system changes, tell me about it. The problem in global investment banks and complex organisations is that if you’re hearing about a change after it’s happened in production, you’ve got a bigger problem in the organisation. We need to move away from this reactive approach to a more proactive lineage approach – we should be thinking about planning and execution, with the automation as the validation that what we’ve done is what we plan to do.

The creation and retention of data has increased exponentially in recent years, with a growing remote workforce catalysing the production and addition of more cloud applications to companies’ data landscapes. Philip made the point that the journey to the cloud is like any other transformational change companies enact.

You need to know where you’re starting from and where you want to end up – an organisation looks at their current state on-premise and plans to move these elements into the cloud, and typically, people don’t pick up their whole organisation and start moving all the data into the cloud at once; it’s this journey that takes time and part of that is designing a cloud migration strategy to ensure you don’t breach regulations like cross-border data sharing, or that you sufficiently structure the migration so you’re not moving huge volumes of data up to the cloud and then back down into on-prem, generating huge costs. Lineage allows us to simulate what that future state of an organisation looks like while remaining in the safety of a confined space.”

Before the session ended, the panel discussed the relevance on AI and Machine learning within metadata management – Philip observed that much of what the vendors are pushing in the market is overhyped mainly because we work in metadata, and not data. The record sets for metadata management are so shallow that you can never train an AI to really deal with it. When you’re working with the actual underlying data and trying to infer lineage based on that data, there’s a much better case for using an AI or an ML – it’s important to understand the distinction between the two.

Play Video

Solidatus for the cloud – Migrate, Optimise and Transform

Businesses are turning to the cloud for reliability, scalability, flexibility and consistent processing – but, with great transformation comes complexity and risk. With Solidatus, prepare your migration strategy by identifying mission–critical data and eliminating redundant information – enrich this data with current regulations, service agreements and business intelligence allowing for in-depth impact analysis before each asset is moved into the cloud.

Latest news

Anti money laundering HIGH RES scaled

“I see my team as both my colleagues within Solidatus, but also our customers and the community we work with … I love to see how of our customers are innovating using our capabilities to deliver some amazing outcomes and solutions.”

In the second of four segments with CDO Mag, Solidatus Chief Data Officer Lorraine Waters continues her conversation with Ben DeBow, CEO & Founder of Fortified Data. Together, they discussed the following key points: 

  • How data lineage is enabling cloud services, with Solidatus’ cataloguing capabilities allowing organisations to safely move data to the cloud
  • Solidatus’ strong collaborative capabilities enables businesses to federate the work that needs to be done as they develop their model and blue print
  • Solidatus makes it simple to give and manage access to crowdsource content
  • Our solution doesn’t need to be managed by a central single CDO team exclusively – colleagues and teams from across a business can be engaged to validate the data in a particular system, for a particular policy or regulation
Play Video

Latest news