Subscribe to our Data Product newsletter.


Predicting the Future + Data testing done right + Lifecycle analysis + More (PAN #26)

Edition #26 - January 20, 2020 Originally sent via Mailchimp

Good morning product friends đź‘‹

There used to be a segment in Conan O’Brian’s show back in the 90s where he went on to do ridiculously funny predictions about the year 2000.

They were absurd and that’s pretty much how I always felt about predictions. That said, they can also be fun and sometimes, just sometimes, someone might be on to something.

And that might just be the case with the BI operational system referred to in our top pick below. To be truly data driven, a business needs to have a very tight loop from raw data all the way towards actions. And the industry is continuously evolving in that direction.

I like that concept of BI Operational System. There’s certainly a lot more to explore on that subject and we’ll circle back to it in the future.

In the meanwhile, let me be the last one to wish you a happy new year. And with that, on with the 26th edition of the Product Analytics newsletter!



Top Pick

What has been my highlight? by @Sisense

What would a new decade be without predictions? Sisense delivered a good piece with predictions for the 20s and the 30s (those of the 21st century btw).

Here’s one prediction that struck a chord as it goes against the modular trend we’re seeing right now.

What we started to see in 2019 was a motion to combine several of the more popular point solutions into larger stacks. Google acquired Looker, Salesforce acquired Tableau, and Sisense merged with Periscope Data. This is a trend that will continue for the next few years until a few major players emerge as viable end-to-end solutions.As the industry continues bundling, organizations will buy one of the new monoliths; it will no longer be a tenable solution to customize a BI stack with a combination of preferred point solutions.

That said, the authors predict that it’s a pendulum swing as some new technology will eventually emerge that will break those silos again, and we’ll swing back to a more modular stack eventually.

Here’s another one on the rise of the BI operating system.

As developers create more experiences related to a company’s data flow, they will begin to enrich the system with analytic apps that reduce the time needed to take actions based on data — actually closing the BI loop. The flow will look like this: Data > Insight > Action > Data. This will transform the traditional dashboard into an operating system that continually drives the business.

It seems to me that this idea of a BI operational system goes hand in hand with the democratization of BI tools, which we’ve talked about a few times before (article and newsletter).

There is some amazing engineering being done in the data space right now and businesses are being built around that idea of making those awesome tools available to analysts and business people directly. That tightens the Data > Insight > Action > Data loop which is referred above.

What do you all think? Are we really on a road towards monolithic stacks owned by mammoths again? And is data engineering to become so transparent to end users that answering questions will just be a matter of having the right analytical app in place that plugs into the right sources?

Data Strategy

Growing your product with the help of data.

How to get to a multi-touch attribution model that works for you by Erika Wolfe

I haven’t shared many (in fact probably none) marketing analytics pieces on that newsletter, because I feel it’s a bit outside the scope of product analytics. But it’s probably more due to a too restrictive definition on my part - I tend to equate product analytics with registered users.

In fact, a registered user’s journey would be incomplete without taking into account how that user first got his/her foot through the door. And that’s where you eventually get into the messy, uh fascinating, world of attribution models.

At a high level, the aim of attribution is to understand the real return on ad spend (ROAS), and a data-informed, multi-touch attribution model provides that kind of insight.

I chose to share that article because it goes further than a single-source attribution, which is more often than not unrealistic. Multi-touch attribution takes into account that an individual may be exposed to a product through various channels before taking the plunge. Our models should reflect that complexity.

This is only a primer on that vast world of analysis, but it’s a great first step if that is an important issue for you.


Factory operations to transform data into analytics.

Data testing: advice from analysts and data engineers, How great data teams test their data and Office Hours: Testing Special by @janessalantz, Erin Vaughan and @clairebcarroll

Here’s a pretty decent dump of great stuff from the good folks at Fishtown Analytics. They’re great follow-ups to a story we ran 2 newsletters ago on data surveys. It all comes back to quality assurance and increasing trust by your end users.

It starts off with advises from analysts and data engineers on how to approach data testing. I like this quote: “Tim Finkel calls this “building the immune system.””. And also Claus Herther’s breakdown of how he writes tests at each stage of transformation.

Follows a piece on how teams are tackling testing as an organization. If you’re wondering how to start on your data testing journey, there are worst ways to proceed than to read advices from very smart people in that field.

And finally, for the true geeks amongst you, here’s a recording of the December 2019 dbt office hours, with a special focus on testing. Having dealt with TDD as a web app developer in another life, I have a sweet spot for dtspec by Sterling Paramore, which is explained in great details here - Testing data transformations.

Data Analysis

Deriving insights from your product’s data.

Growth through segmentation: lifecycle analysis to understand your users by Moinak Bandyopadhyay

Not all users are created equal. Segmenting users into different groups and tracking their movement between different engagement states is a great way to get a deeper understanding of your users and find opportunities for further growth.

I spent a lot of time combing through that article as I had a small lifecycle analysis project on my plate (which I will talk more of soon). Although this is a Mixpanel piece, the concepts and ideas goes beyond what you can do within their tool. So if you’re looking for a good introduction on the subject, have a read.

Market News

What’s happening the product analytics market.

Drill to Detail Ep.76 ‘Segment, Ecosystems and Customer Data Platforms’ by @markrittman

Finally, here’s a great interview that finished the year for the Drill to Details podcast, where Mark Rittman spoke with Calvin French-Owen, co-founder and CTO of Segment. If you want the grand tour of what their platform is, current innovations and future directions, it’s worth the listen.