Or, defining what success looks like for data leaders — As analytics professionals, we deal in data: serving ad-hoc reports on a minute’s notice, pulling queries for executives, and generally forecasting company performance across a variety of metrics. But how can we be truly successful if we don’t measure our own performance, too? In this article, we discuss six important…

Self-serve systems are a big priority for data leaders and a principle of nearly every modern data strategy, but what exactly does it mean? — Data self-service, the ability for stakeholders in the organization to answer their own business questions with data, is a top initiative for nearly every data leader I’ve spoken to this year. It’s so foundational to creating a data-driven organization, that most of the questions surrounding it focus on the “when”…

What are data contracts and do they make sense for your organization? — Forward: A Data Engineering Movement Afoot? We ran into Andrew, a team lead and senior engineer at GoCardless, at our London IMPACT event. He talked about implementing data contracts and how GoCardless has eschewed what has started to solidify as the industry standard ELT modern data warehouse approach. This approach typically involves complex pipelines, large…

Innuendo aside, “big” data is getting smaller and faster. Data leaders need to adjust to the new paradigm. — At some point in the last two decades, the size of our data became inextricably linked to our ego. The bigger the better. We watched enviously as FAANG companies talked about optimizing hundreds of petabyes in their data lakes or data warehouses. We imagined what it would be like to…

When it comes to building a category, it’s all about your customers. Here’s our vision for the future of reliable data. Four years ago, I was leading a data team at Gainsight, a customer success company, responsible for generating the analytics that powered our executive dashboards. …

The data steward has an impossible job. Here’s why and what we can do to empower them. — In his seminal 2017 blog post, The Downfall of the Data Engineer, Maxime Beauchemin wrote that the data engineer had the worst seat at the table. Data technology and teams have changed tremendously since that time, and now the Preset CEO and creator of Apache Airflow and Apache Superset has…

Data lineage should be less like a treasure map and more like Google Maps — Data lineage isn’t new, but automation has finally made it accessible and scalable — to a certain extent. In the old days (way back in the mid-2010s), lineage happened through a lot of manual work. This involved identifying data assets, tracking them to their ingestion sources, documenting those sources, mapping…

The data warehouse is the foundation of the modern data stack, so it caught our attention when we saw Convoy head of data Chad Sanderson declare, “the data warehouse is broken” on LinkedIn. Of course, Chad isn’t referring to the technology, but how it’s being used. As he sees it…

On average, companies experience one data issue for every 15 tables in their warehouse. Here are 8 reasons why and what you can do. — Say it with me: your data will never be perfect. Any team striving for completely accurate data will be sorely disappointed. Unit tests, anomaly detection, and cataloging are important steps, but technology alone will not solve your data quality problem. Like any entropic system, data breaks. And as we’ve learned…