How to think about Data in terms of strategical assets

From my years of experience in the data ecosystem, I observed 2 main patterns with how business treat data:

• Core Function
• Supporting Function

A core function IMO needs to fulfil the following:

• Works proactively on executing a roadmap that emerged from solving a core strategic objective for the company
• Has a dedicated strategic role attached to it to ensure the proper execution of the roadmap (Head of Data Analytics or Data Science for instance). This person then delegates to Lead(s) within the team who focus more on the tactical details of the execution
• A core function also creates assets and leverages them more and more with time

A supporting function doesn’t have one or multiple of the points mentioned above, but contributes to the ones from other teams

Most data-informed companies choose to make put data in a tight spot of reactive support while expecting it to deliver more and more with time without a deliberate serious decision to bring it more to the centre as a core function. Only those who make this transition become really data-driven.

The rest of the article is about data assets that generally characterise data-driven organisations.

Generally these are the assets a data function can build and own (Ordered in terms of priority as each one is the foundation for the next one):

Asset 1: Solid Core Data Modelling

Source of Truth for core business logic. This is an engineering asset and should be built and maintained by engineers.

Some pitfalls to avoid:

• Having this built and managed by analysts -> TechDebt becomes unmanageable (fyi: Most analyst didn’t hear about TechDebt, Architecture, Design to begin with…)
• Not paying attention to this till everything (usually scrappy sql scripts/analyses that feed into a BI tool) breaks beyond the point of painful bandaging -> When this happens, full refactoring that requires months of effort is usually needed
• Not following what is happening around you and updating -> This is especially important for ‘new’ frameworks and tools like dbt and cloud-focused modern data stack where things move fast and even concepts change super quickly

Asset 2: Functional reliable and malleable Analytics Modelling

This is Analytics or BI (Business Intelligence). The asset here is the consolidated data-proven practices around agility, effectiveness and reliability of the deliverables. None of this can be attained without the previous point especially if we take into account the more fluid nature of this aspect (metrics and how we present them can evolve quickly but business logic behind them usually evolves slower if any). This asset can be built by ensuring a good connection with Business and then leveraging the Core Data Models (could be done by strong data analysts or business savvy agile engineers)

Some pitfalls to avoid:

• Applying stiff slow modelling to this -> Engineers that don’t have agile business understanding and skills might fall into this trap. Remember analytics is more fluid and elusive than one thinks. Flexibility in modelling and delivery is important with an active feedback loop to iterate and correct as needed
• Confusing this asset for the first one -> Usually management falls into this trap to cut costs and move quickly confusing the ability to write SQL to how it should be applied: SQL could be applied for proper data modelling (first asset) or analytics business-facing modelling and visualisations. Some could do both, most can’t.
• Intermingling the core models with the analytics models -> This is easily avoidable with a proper architecture like: Sources > Staging > Core > Business Marts. The first 3 layers are the core ones (see previous point), the last is the Analytics one. If you have a semantics layer, it fits here as well

Asset 3: Creative, agile Data Science solutions

This is more of a cultural asset that builds on top of the previous 2 assets. It works best when standard metrics and analytics are well mastered and built on top of a rich solid data model that allows capable data scientists to go beyond the standards and get new solutions.

This asset calls for agile effective creative-thinking Data Scientists (not all analysts can be data scientists and most data scientists produce waste due to reduced effectiveness and lack of creativity

Some pitfalls to avoid:

• Crappy complex or irreproducible explorative work that can’t be transformed to production > Many data scientists fall into this trap. I personally think most Data work HAS to be aiming for automation and systematisation (if you need someone to run it manually all the time, let business people do it!). This doesn’t mean, that you need to sacrifice exploration, on the contrary you need to always prioritise it at the beginning of any ‘unknown’ or risky endeavor. However, plan to move beyond as soon as you start the dots connecting and the added value concretising -> This is where this asset connects with the next one
• Analysis paralysis -> As mentioned above, don’t do analysis for the sake analysis. Do it for the bigger purpose to serve the business and ultimately automate. You want to move yourself from the loop for a system to get created. Use iterations and MVPs to do this in a more agile way. You will need to manage the timing of the iterations and scope of each though.
• Expecting any analyst to be able to do this -> Not all can, this generally requires a deeper applied/practical understand of analytics, data science, stats, proba and some ML is also helpful. If someone can’t explain what is the difference between signal and noise, they 100% won’t do well with this aspect, as cutting through the noise to find the signal is a key skill here. How one does, is the creative part.

There are more, but those are quite frequent to encounter

Asset 4: Automated Data Science & Machine Learning — Data Science at scale

Not many organisation can get here as this requires a good maturity, culture, strategy and execution of the previous 3, which as you anyone who went through this knows REQUIRES time, energy and money investment. This is where companies that has Data at a core function aim for by choice. At this level of deliberate maturity, you get to building systems at scale and linking them to the bigger engineering systems built by the Dev Teams as things are needed to function beyond Ops (previous 3 assets are enough Analytics Operations) and impact the product by becoming part of it.

This one calls for Data Scientists who are also engineers (ML Engineers fit here)

Some pitfalls to avoid:

• Accelerating to this part before properly investing in the others -> This leads to highly complex TechDebt as this will involve Engineering Debt and Data Science Debt, both are complex to handle on each own
• Not paying the DataOps/MLOps price: Monitoring such systems becomes a key element of the system. Remember this is a ‘probabilistic’ system and anyone who works with probability knows that you can’t put proba in a box. You need to play but its rules and monitor how things change and keeps moving. Don’t get fooled by how most of Engineering is deterministic, ML/DS is 100% not
• Disconnecting this from the rest of the Dev org: While the other 3 could be disconnected and work in a siloed way (in the cloud data warehouse for instance) — something I don’t recommend to begin with as it limits the team — , this one HAS to be seamlessly connected with the other parts especially backend

Finally, the most pitfall of all to avoid is to assuming that such assets will build themselves reactively. That never happens and I am willing to bet my whole career on that. The reason for that is that data, its place and its utilisation in business is still highly messed up (teenage-hood maturity IMO for most sectors, except probably those that started super early as their core business model relied on that like insurances). Not all organisations need to go the full way. How much you want to go is a function of your strategy with data. If you don’t have one, start there!