Metrics Observability: Power to the (Business) People
Companies of various sizes rely on metrics to quantify business performance and assess the success or failure of different initiatives, and while the modern data stack paradigm allowed for automation and scalability, access to data in an adequate (read business people friendly) format remains a struggle for many organizations. This is where the semantic layer came into play.
A couple of great articles on the definition of the semantic layer here and here.
Metrics are an essential component of the semantic layer as they translate data into indicators that have a business meaning and that can be created and traced like any other data object.
The metrics layer acts as a translator between data in the warehouse and the business, which - in theory - is an excellent step toward data democratization. But what happens when business users start to question the meaning and accuracy of the metrics? Someone from finance might think ARR should be computed differently compared to someone from sales, someone from HR might think employee retention is higher than what they see in a given dashboard, the CFO could say NRR numbers are off… The possibilities and hence nightmares for the data team are endless.
This is why we are excited to introduce the concept of Metrics Observability!
By extending the principles of our Full Data Stack Observability framework to the semantic layer, users of Sifflet can now access and discover business metrics through the Sifflet UI, get a comprehensive idea of their health status, and follow the lineage to see how the metrics are computed and how they interact with other data assets. Business users will be able to discover and trust existing and new business KPIs and drive decision-making using reliable metrics.
A metric is an aggregation of time series over one or more tables that represent a business KPI.
Users can log into the UI and either define metrics, document (using SQL), or discover existing metrics via the Sifflet catalog, see the full-stack (ingestion to consumption) column-level lineage to other data assets but also to other metrics. Thanks to our ML-based monitoring, business and data users can get alerted in case of a data quality anomaly and conduct proper root cause analysis and incident management reporting. Similar to our Full Data Stack Observability approach for technical data assets.
In an upcoming release, we will expand to broader BI and transformation tools (currently only available on Looker) and metric stores/semantic layers. We will also expand the breadth of our ML-based monitoring to group business metrics that belong to the same business domain, and capitalise on the lineage to ensure consistency.
This feature is a way for data consumers and business users to achieve trust in their metrics and explore the full potential of the semantic layer. Reach out for more details email@example.com.