Case study · Western Uusimaa Wellbeing Services County · 2024 →
From fragmented data production to a unified data platform
Western Uusimaa Wellbeing Services County moved its data production to software engineering methods. The work began in 2024 and continues.
Western Uusimaa was technically ahead before the work began. That is exactly why its problem was harder than in most places: the county had come far enough that the next bottleneck was no longer the platform. It was the way of working on top of it.
The basics
- Customer
- Western Uusimaa Wellbeing Services County
- Sector
- Public health and social care, wellbeing services county
- Duration
- 2024 → continuing
- Invinite's role
- Data production architecture and method, built together with the county's data engineering team
- Environment
- Azure Data Lake, Databricks Lakehouse
- Methods
- PySpark and Python, unit-tested transformation functions, CI/CD, data contracts
Starting point
In 2024, soon after it began operating, Western Uusimaa made a decision few of its peers made that early: it bypassed the traditional data warehouse altogether and moved straight to an Azure Data Lake and Databricks Lakehouse architecture. By early 2025 the first reports were in production.
The county had, however, identified the limit it was approaching. A lakehouse cannot be built on the terms of individual reporting applications. Every new report was its own definition, a technical implementation shaped by whoever built it, and its own truth, and each one depended on the person who had made it. What was needed was a refined data foundation that serves several purposes reliably. The data engineering team's methods needed automated testing and quality assurance in particular.
Why this is a harder problem than it looks
Replacing a data warehouse with a lakehouse is a procurement. The problem that comes after it is not. When every data product defines its own truth, an error does not show up as an error. It shows up as two different figures for the same thing. It is found in the meeting where the figure was meant to be the basis for a decision. The correction is manual work, manual work does not scale, and the more data products there are, the less anyone dares to touch them.
There is no tool that solves this. It is a question of method, and a method changes only if the people working with it change how they work. That is why this was not a delivery project.
What was done
Four changes, each of which can be checked in the code.
From SQL to PySpark and Python
The single most significant change. It makes data engineering software development rather than query writing, and with it every software engineering tool becomes available as it is.
A SQL script cannot be unit tested in any meaningful way. A function can.
From long pipelines to small transformation functions
Long, hand-validated data pipelines were split into transformation functions that a developer can test locally before anything is run.
An error is found on the developer's machine in minutes, not in production months later.
Automated testing and deployment in the data lifecycle
Testing and deployment were automated (DevOps and CI/CD). Quality management moved out of human memory and into a pipeline that runs every time.
Quality that depends on someone remembering to check is not quality. It is luck.
Data contracts
Contracts are a technical way of recording a description of what a data model contains: they describe, technically and logically, the core data that corresponds to real-world phenomena. They carry the essential metadata, state which stage of the lifecycle the data is in, and define quality rules and ownership. They travel with the code from development into production. This is the point where quality validation becomes automatic across every architectural layer. It is also the same structure that later makes AI reliable: a language model is given an unambiguous explanation of what each data structure means in operational terms. The data contracts were implemented using a widely used open public standard (ODCS).
What changed
A way of working that can be repeated
A documented way of working emerged that does not rest on one team or one person: coding practices, data lake and data architecture, and roles.
A controlled release cycle
Releases are more controlled and can be made more often. Every release is tested with automated quality and integration tests, and its effects on data products are understood.
The architecture keeps developing
A working architecture lets the data lake be used more widely: beyond reporting, for AI development, agile and reliable analytics, and as a data source for external applications. At the same time, the county has moved towards democratising data.
The customer's view
Moving to the Data as Software model has fundamentally changed the way we approach data production. We want to understand data and produce knowledge. Two years of close collaboration have given us the opportunity to treat data as a core product, and it has improved both the speed and the reliability of our data pipelines and our data products. For the wellbeing services county this means a more accurate, knowledge-based understanding, which ultimately supports better decision-making and better care for our residents.
Why this was not a delivery project
The method did not come from Invinite bringing a ready-made model. It came from Western Uusimaa having a clear direction for its platform and an exceptional ability to assess very complex data and code architecture proposals. We designed the technical architecture to solve Western Uusimaa's own problems, and they assessed it critically. A method taken this far would not have developed without that reciprocity.
The way of working was built along the way, not in advance. The documentation, the workflows and what was learned belong to both teams. They are not handed over at the end, because they have been written together all along.
What this makes possible next
A disciplined data foundation is a precondition for AI, not an alternative to it. Data contracts are decisive here. It matters that data leadership recognises you cannot jump straight to the end. At Western Uusimaa the order of progress has two stages.
AI-assisted development
The agent understands the shared concepts, how each operational area works, the data foundation and the stages of the development process. This lets it make the work of data engineers, BI developers, analysts and operational management considerably more efficient in development.
Tools for everyone who uses data
Once the structure, the data content and the AI capabilities have proved reliable in development, the use of AI can be extended to everyone who uses the data. The precondition is that the underlying data stays well defined, of good quality and under managed access rights.
Why we succeeded
Developing a new method and putting it into use at the same time worked partly because the customer is exceptionally capable. Western Uusimaa had defined its intent well and was able to assess architecture proposals critically. In an organisation with no technical capability of its own to take a position, the same method would have led either to slower progress or to the supplier making decisions alone. Now that the method is mature and the key decisions have been made, the model can be adopted more widely in other organisations too.