Improving data reliability + Building a Culture of Experimentation + The dbt IDE + More (PAN #24)

Edition #24 - December 2, 2019 Originally sent via Mailchimp

Good morning product owners 👋

For our American friends, I hope you all enjoyed thanksgiving 🦃 Now please get back to work as it’s been awfully quiet for us outside the US.

In this edition, I’m happy to share an interview I did with Mark Rittman on his most excellent Drill to Detail podcast. That was a first for me and it shows, to say the least. I treat the audience to an 8 minute intro to how I got into data analytics, a thick french accent, nervous laughter and occasional nonsense. I hope you will enjoy it 😃

That said, I was lucky enough to be treated to a Sunday roast when I visited Mark a few weeks back in London. We had actually recorded that interview a few months ago and it had confronted me with things I needed to improve on my side. So continuing that conversation was immensely valuable. Mark is really a great, knowledgeable and generous guy and that whole experience has been really enriching.

Here’s me eating weird British food.

With that, on with the 24th edition of the Product Analytics newsletter!

Olivier

@olivierdupuis

Top Pick

What has been my highlight?

Improving data reliability - Andrea Kopitz, Envoy (SF dbt Meetup, November 2019)

YouTube.com by @andrea_kopitz and @envoy

The folks at Fishtown Analytics dropped a ton of videos on YouTube to binge on 🍿. There’s one that stands out (which is something considering the high quality of all those talks) and it’s a talk given by Andrea Kopitz at San Francisco’s dbt meetup.

There are just so many great ideas in this talk, but it’s their overall approach to evaluating, improving and monitoring their company’s data quality that resonates. Sending a data survey to employees for example is just so obvious, but I honestly had never thought about that. It gives you brutal feedback about how end users feel about the confidence they have in your data and practices. It then aligns the team as to what to improve, how and maintaining that high quality.

I followed up with Arvind Ramesh who is also part of the data team at Envoy and he generously shared a link to their data survey. He gave me the go ahead to share that with you as well. So here goes:

https://signwithenvoy.typeform.com/to/mPLn7v

Product News

What has been happening in the product analytics market?

Introducing: The dbt IDE

GetDbt.com by @DrewBanin

Here’s another milestone in dbt’s journey (I know, I know, another dbt story 🤷‍♂️) - the introduction of their IDE. This doesn’t come as a surprise, as they had already shared their roadmap with the release of dbt 0.14 back in July.

Before I get into the IDE, maybe I should explain a bit why this is an interesting story. First off, because dbt is changing how we do analytics. By simplifying the transformation process, making it more robust through devops best practices, and by building on a SQL foundation that allows modularity and eases integration with other parts of a data stack.

The second reason why this news is important is that it’s in clear alignment with a trend to democratize data engineering. It’s about making the whole data life cycle accessible to analysts. I think this is also where Meltano is heading.

Back to the IDE. Well, it’s what you would expect from any IDE (getting you up and running quickly, removing complexity, accelerating development, etc.). Except that it runs directly within dbt Cloud, which allows you to code and execute in almost real-time. It’s about opening up to users who are not your hardcore data engineers, but to people who are interested in interrogating the data, who will find that dbt’s IDE gives them even more flexibility to answer their questions on their own.

For extra fun, you can watch dbt’s office hours where Drew presented the IDE.

Strategy

What profound ideas should orient your product analytics journey?

How Imperfect Food Built A Culture Of Experimentation And Increased Customer Retention

Segment.com

This article is pretty straightforward when it comes to experimenting with a product’s UI. If you haven’t yet adopted an experimentation process, it’s worth the read.

But what’s novel about this article is when it discusses the process of experimenting with internal processes. Can’t say I had heard about that before.

In this cade study, they talk about Imperfect Food’s experimentation with the customer care team. The experiment they ran might not be groundbreaking (providing real-time info about a customer’s purchase), but the underlying idea is still worth exploring further.

Best Practices

How should you tackle your product analytics journey?

Clickstream Analytics: Gamechanger for eCommerce

RudderLabs.com

In itself, RubberLabs is an interesting story as it presents itself as an open-source alternative to Segment. I’m not familiar enough with them to evaluate that claim, but it seems to me that Snowplow Analytics might already be that open-source alternative to Segment 🤨

But anywho, back to our story… clickstream analytics! What the hell is this you ask?

“To put very simply – a “clickstream” is a sequence (“stream”) of events that represent user actions (“clicks”) on a website or a mobile application. However, in practice, the ambit of clickstream extends beyond clicks to include product impressions, purchases and any such events that might be of relevance to the business.”

So, it’s essentially all events generated by a user relevant to your product. Nothing new here. But what’s interesting is what follows. The author proposes interesting ways to analyse that data in order to improve conversions, such as this one:

“Recurrent Neural Networks or Long-Short Term Memory algorithms can be used to generate a model which in turn can predict the next page in a sequence that would culminate in a purchase.”

Looking for neat ways to analyse your clickstream data? There a good pool of ideas in here.