Product ownership is about precision. How clearly you understand your user’s struggle. And how you position your product in that struggle’s market.
I’ve founded this small consulting agency because product analytics is exciting. It’s a world filled with very creative, smart and generous people. So much so that it’s a bit hard to keep track of all that’s happening. Which is why I started the Product Analytics newsletter, as an exercise to continuously learn about that space and share my observations.
The audience I was targeting at first were product owners as they are the majority of clients I work with. But as the audience of the newsletter grew, so did other type of readers joined the fun.
And now I’m left wondering how well my newsletter is filling its mission:
This newsletter’s mission is to share data analytics’ best practices and new ideas with product owners and analysts, so they can incrementally and intelligently improve their product.
Overview of Product Performance
Trying to answer that question is like taking a plunge, because we start from very high-level metrics which leads us to dig a little bit deeper, ask new questions and go a level deeper again. Over and over.
For example, based on high-level, aggregated metrics, the newsletter had been doing somewhat ok from the start.
A slightly declining but still “healthy” open rate…
And a wildly varying click rate…
Nothing earth-shatering, but I guess normal for a new newsletter. That said, it wasn’t exactly what I was hoping for. I had little echo from readers and my intuition was telling me I was missing the mark.
My audience was growing, but it seemed to me that it was becoming more heterogeneous. And I wasn’t sure how that fitted in my initial strategy. Could it explain my declining open rate?
I wanted to go deeper than those metrics, segment users and start exploring those group’s behaviours.
Deep Dive With User Lifecycles
Going down that route requires granular data. And there’s no way around it, you need to own that data. I won’t get into the data engineering aspect of that problem, as you can read how I tackled this here: How To Gain Insights On Your Newsletter’s Development With Product Analytics
Ok! So we have the data and we modeled it to get a “lifecycle status” associated to all subscribers per newsletter edition, as well as “status movements” for even more insights on behaviours (I’m not going into how I modeled those, but do write me if you’d like to learn more). Let’s move forward with the analysis.
I first segmented our users based on their professional roles. Here’s their distribution and the evolution of that distribution.
One first insight is that my audience is composed of different product roles, but the growing one are data professionals (analysts, engineers, etc.). They work closely with product owners and are certainly a good vector through which to fill this newsletter’s mission.
I then started doing lifecycle analysis for my users.
Mapping out user journeys such as the ones below for product owners…
Aggregating journey movements…
Going through those graphs, segmenting per user roles, It became obvious that data professionals are a growing segment of my newsletter’s readership and they are an engaged one. It makes sense as this newsletter does get technical and they are a natural bridge to product owners.
Without alienating product owners, I started thinking about how I could restructure my newsletter to provide even more value to data professionals as it essentially benefits product owners in the end. I did start making subtle changes such as sharing some more technical pieces and introducing sections in the newsletter that I know will interest data professionals more than product owners.
Tracking the Impact
As I mentioned, I started making subtle changes and, although too early to celebrate, I’m seeing positive trends in the data.
The open rate has stoped declining, now trending close to 60%, which I feel is pretty decent.
Same with click rate. I like how it’s trending, but let’s see how it goes.
One metric I like to keep track of is how many times a single email has been viewed by all users who opened it at least once. That gives me an idea of interest and sharing. It’s far from a perfect metric as results could be due to multiple factors, but if you take it with a grain of salt, it can be informative. That metric has also been on a healthy path.
Not Being Everything to Everyone
If there’s anything I learned from reading a few Seth Godin books, it’s that you can’t and shouldn’t aim to be everything to everyone. Building a business nowadays is about having a very precise mission that naturally creates a following. And that’s certainly something I want to do with my business and newsletter.
Lifecycle analysis allowed me to better understand who is taking interest in that mission and how I can better communicate with them to amplify my impact.
How else are those insights influencing my own product development journey? Well, for once, this blog post is a result of it – data professionals probably will be interested in some of the analysis I myself do in regards to the newsletter. But there is still a clear link with product ownership and how data helps grow a product.
Could I go further? Maybe start sharing some of the tools I use when doing product analytics? I’ve also been thinking about doing live pairing with fellow data professionals to tackle common situations.
As a product owner myself, that whole journey led to a simple realization that the audience that is being built around that newsletter is not exclusive to others who have a similar role as I have – it’s more inclusive than I had planned for.
But that still leaves open an important question for me: should I still aim to engage directly with product owners or are data professionals a more natural bridge in the pursuit of my mission?
And even more importantly, how can I keep improving the newsletter’s format and content to fulfill my mission even better? That “views-per-open” metric above is probably my North Star at this point as it gives me an (although far from perfect) idea of how much that newsletter is worth multiple reads and worth sharing.