Data Observability, Five Years In: Why the Old Playbook Doesn’t Work Anymore

Introduction: The End of the Beginning

Five years ago, data observability burst onto the scene as the missing puzzle piece in the modern data stack. It promised to bring clarity, stability, and trust to increasingly complex data environments.

At the time, the mission was clear: automate and scale data quality. Tools emerged to help detect schema drift, freshness issues, and volume anomalies — essentially turning manual testing into something more scalable and programmatic. It was the right answer to the problems of that era.

Fast-forward to today, and the space is crowded with tools, frameworks, and philosophies. But many are still stuck in the past — overly focused on technical infrastructure, limited in scope, and detached from business value. Meanwhile, the world has changed. AI is transforming how teams consume and act on data, and data observability needs to catch up.

This post is a look back — and a look forward. How did we get here? Where did we go wrong? And what comes next?

The First Wave: Data Infrastructure-Centric Observability

The first generation of data observability platforms was designed for data engineers. Heavily influenced by APM & infrastructure monitoring, their goal was to monitor data platforms like you’d monitor servers: alerts for when things broke, metrics for system health, and lineage graphs to debug where failures occurred.

Many companies emerged to fill this gap. They helped teams get visibility into broken pipelines and reduced time to detection.

But they also created a new kind of noise: shallow alerts without context. Who was impacted? How critical was the data? Was it worth waking someone up for?

These tools treated data incidents like infrastructure outages — but data isn’t infrastructure. It’s a product. And data quality isn’t just about uptime — it’s about trust, usability, and business impact.

The Shift Left Movement: Developer-First Observability

As the category matured, a new philosophy gained traction: shift observability left. Another inspiration from the DevOps world… This approach embedded data checks earlier in the lifecycle — during dbt runs, CI/CD pipelines, or through data contracts.

Observability-as-code made perfect sense for teams with strong engineering practices. It caught issues early, brought data quality into the development workflow, and treated data like software.

But it also narrowed the aperture of observability to technical users and technical issues. It reinforced the idea that data quality is a developer problem — not a company-wide concern.

Shift-left observability doesn’t help a marketing analyst trying to explain a dashboard anomaly, or a product leader wondering whether a KPI drop is real or a data bug. It rarely connects issues to business outcomes or ownership outside the data team.

The Lineage + Catalog Convergence (and Confusion)

In parallel, many platforms — including ours at Sifflet — began layering in lineage, discovery, and metadata. The idea was to give teams more context: not just what broke, but why and where.

This led to a growing overlap between observability and catalogs. Tools started looking similar, competing for the same budget, and confusing customers about who does what.

This convergence wasn't necessarily wrong — but it lacked opinionation. It prioritized breadth over depth. And it often failed to translate metadata into meaning.

Just knowing the lineage graph doesn’t help if you can’t trace the business impact.

Datadog x Metaplane: Validation, But Not the Whole Picture

Datadog’s acquisition of Metaplane marked a major milestone: it validated that data observability is now part of the broader observability landscape. Monitoring data is as critical as monitoring infrastructure or apps.

But it also reinforced a technical worldview: data observability as an extension of APM. If your infrastructure is up, your tables are fresh and your schemas haven’t changed, the thinking goes, your data must be fine.

But that’s not how trust works. A pipeline can be perfectly healthy and still deliver the wrong numbers. A drop in ingestion volume could be perfectly expected when provided with the right business context. And more importantly, Infrastructure-level checks may tell you that something broke, but they rarely tell you: Who is affected. Why it matters. What to prioritize and How to fix it.

This approach leads to alert fatigue, wasted cycles on false positives, and — most importantly — a lack of trust in data from business users.

Uptime is not the same as reliability. Treating Data Observability as a part of infrastructure monitoring is a step backward. It reinforces the outdated idea that data is just another IT asset. That the job ends when the pipeline runs. That if the schema is intact and the job succeeded, the data must be “fine.”

But we’ve spent the last decade proving that’s not true. Data has evolved from backend plumbing to a strategic product. It drives decisions, powers AI models, and connects directly to revenue and customer experience. Treating it like infrastructure not only flattens its role — it undermines the entire progress made in the Data & AI infrastructure space over the past decade where as an industry we rallied for a future where Data is a product, not just a pipeline;trust is built through context and consumption, not just uptime; And AI readiness depends on quality, lineage, and business semantics, not just freshness. We can’t reduce data observability to a set of technical monitors.

‍

Sifflet’s View: Context-Aware, Agentic Observability

At Sifflet, we believe the next phase of observability must go beyond pipelines and dashboards. It must be contextual, collaborative, and intelligent by design.

We take every data asset and place it in both its technical and business context. That means:

• Technically: metadata, lineage, query logs, monitors, dependencies.
  
  
• Functionally: ownership, usage patterns, semantic inputs, documentation, business KPIs.
  
  

We help teams detect incidents and triage them based on business impact. We don’t just raise alerts — we tell you what’s broken, who cares, and what to do next.

We’re also building toward a future where observability systems can operate with — or even as — agents. That means surfacing insights, automating triage, resolving known issues, and escalating what truly matters.

Observability isn’t just about detecting problems. It’s about operating with intelligence and intent.

The Future: Intelligent, Impact-Driven, and Collaborative

Data observability has matured — but the old playbook no longer works. What we need now is:

• Business-aware prioritization: Not all incidents are equal. Observability should know the difference.
  
  
• Cross-functional collaboration: Analysts, product managers, and execs need to trust and act on data. Everyone is a data user, and therefore everyone deserves to know and understand data health.
  
  
• Agentic intelligence: AI and metadata should help automate and accelerate root cause analysis, not just flag issues.
  
  

The next generation of data observability platforms will go from passive monitors to active copilots.

At Sifflet, that’s exactly what we’re building. Because reliable data isn’t just about tables and triggers — it’s about trust.

‍