I wanted to share this article we put together that illustrates how real data teams are benefiting from data observability. Data observability, an organization’s ability to fully understand the health and quality of the data in their systems, has become one of the hottest technologies in modern data engineering. In…

Data teams are more important than ever before — but they need to get closer to the business. Here’s how we can right the ship. — Over the past decade, data teams have been simultaneously underwater and riding a wave. We’ve been building modern data stacks, migrating to Snowflake like our lives depended on it, investing in headless BI, and growing our teams faster than you can say reverse ETL. Yet, much of the time we…

The post-modern data stack is coming. Are we ready? — If you don’t like change, data engineering is not for you. Little in this space has escaped reinvention. The most prominent, recent examples are Snowflake and Databricks disrupting the concept of the database and ushering in the modern data stack era. As part of this movement, Fivetran and dbt fundamentally…

Some organizations approach data quality like a crash diet. Instead, try these 6 steps for sustainable data quality at scale. — I like to think of operationalizing data reliability within the context of physical fitness. You can get in shape with hard work, but staying in shape requires good habits. And above all else it’s a mindset and a lifestyle. You also have to sweat. At least a little bit. No…

And how to avoid falling into the same traps — Everywhere we go in the cloud data space today, we’re hearing one message loud and clear: “you should be thinking about data governance”. It’s a sentiment that we wholeheartedly endorse, but we like to take it a little bit further — you should be thinking about data governance differently. In…

The data gatekeeper is dead, long live the…oh no what have we done? — A silent alarm rings in my head whenever I hear someone utter the phrase, “data is everyone’s responsibility.” You can just as easily translate this to mean that “data is no one’s responsibility,” too. This, readers, is what I call the “data tragedy of the commons.” Here’s why. The term…

Even with the most well-designed data platforms, systems will break. Without embracing risk, you’re playing with fire. — Say it with me: broken data is inevitable. It doesn’t care about how proactive you are at writing dbt tests, how perfectly your data is modeled, or how robust your architecture is. The possibility of a major data incident (Null value? Errant schema change? Failed model?) that reverberates across the…

Or, defining what success looks like for data leaders — As analytics professionals, we deal in data: serving ad-hoc reports on a minute’s notice, pulling queries for executives, and generally forecasting company performance across a variety of metrics. But how can we be truly successful if we don’t measure our own performance, too? In this article, we discuss six important…

Self-serve systems are a big priority for data leaders and a principle of nearly every modern data strategy, but what exactly does it mean? — Data self-service, the ability for stakeholders in the organization to answer their own business questions with data, is a top initiative for nearly every data leader I’ve spoken to this year. It’s so foundational to creating a data-driven organization, that most of the questions surrounding it focus on the “when”…