Choosing a Product Analytics Tool

You might say CDP, I say it comes down to event tracking.

Last updated: 10/03/2022

Every company’s got users. And users do things on a site that a large team of engineers has been paid to build based on designs. How do you know if the designs implemented are actually good? How do you know if all that time, money, and overall effort resulted in a product that is actually beneficial to the business?

That’s product analytics: the discipline of understanding how users engage with your product. The “analytics” portion can contain a lot of different data, and that’s where confusion ensues. Mostly, it’s front-end tracking data—in short, tracking what pages users view and buttons they click. This will be the primary focus of this post.

Why would you need product analytics, you might ask? Granular event tracking data can give you the following tidbits:

• Funnel analysis and drop-offs. Looking at percent of users that reach pages A, B, C, then D in a flow and comparing those percentages will show where users get stuck. If from page A 95% reach B, 90% reach C, but only 60% reach D, there’s clearly too much friction between pages C and D.

• Customized user experiences. A set of actions and then no conversion indicates a marketing opportunity. Consider a user that checked out an FAQ page about purchasing but didn’t purchase a product—if this event data lives in the marketing platform, automated campaigns can re-engage this user to potentially convert.

Photo by Arno Senoner

While some of this data might live in the existing production database that’s required to run any application, backend data doesn’t give the whole picture nor is it easily accessible to analytics and business teams.

Customer data platforms (CDPs) claim to solve the accessibility problem. It’s a confusing term, I know. One way to define a CDP is a platform that centralizes user behavior (usually in the form of events) and sends it to other tools enabling customized user experiences. For instance, sending that button click event to the marketing platform directly.

Some have argued the data warehouse itself should be the CDP, while others have taken event platforms and rebranded as CDPs with existing functionality. While there’s disagreement as to what a CDP should look like, there’s agreement around one thing: event tracking data is at the center of product analytics.

---

I struggle to find the best place to have conversations at the intersection of marketing, data, and product. If you agree and want to have a dedicated Slack community to do so, express interest here:

---

Features of event tracking products

When you buy a dining room table it doesn’t only come down to the functionality of the table, but also how many people it can sit, the design of any matching chairs, price, and more. Similarly with event tracking, it’s more than just firing some events somewhere (don’t haphazardly do that, it leads to major tech debt and chaos).

Tracking implementation

While product managers and marketers may care less, engineers will care about implementation specifics deeply. The most common ways to integrate an event tracking tool into an application are by using an SDK, Pixel, or Webhooks. All have SDKs, and some have one of the other two. Depending on the existing application stack, your engineers will have an opinion so make sure to check with them.

💡 Tip: I lean towards tools that support a variety of both web and mobile SDKs and at least a pixel or webhooks as well to increase options.

Native integrations

The tools that market themselves as CDPs will have the option of intaking data from or sending data to other tools as a part of the platform. Advantages may include: sending data to the warehouse directly without need for an ETL tool; sending event data from the application to a SaaS tool to trigger a custom experience; or sending an event from one SaaS tool to another for yet another custom experience. Allowing the analytics team to model event tracking data from a warehouse will lead to a different approach than enabling the operations team to send events as triggers between a handful of SaaS tools.

💡 Tip: Make a list of the highest priority initiatives, who will tackle them, and what tools already exist in your stack. Ensure these are supported by your choice of vendor.

Analysis without paralysis

Who’s doing the actual funnel analysis of event data and driving the decisions? If it’s the product team, they should have the tools to easily understand user behavior within the event tracking tool itself; for an analytics team, data should easily make its way into a warehouse. I consider “analysis” to be available within the tool if exporting to a data warehouse isn’t needed to extract insights.

💡 Tip: Assign an owner to analyzing event tracking data and ensure they have tools accessible to them to visualize it.

Pricing

Some tools are closed-source only, some have fully-open source offerings, and others have limited usage free tiers based on the number of events or users. As we all know—open source is usually free like a puppy so involves more engineering effort but can be attractive in highly regulated industries. However, for small businesses a freemium plan isn’t really that limiting and works great.

💡 Tip: Get your budget and compliance requirements list ready, as event tracking can be heavily regulated in regions like Europe.

Vendors

Without further ado, here’s my take on the vendors in the space. Instead of grouping them into categories as they all market themselves as being an event tracking system, a CDP, and an analytics platform as well, I’ll highlight features that may help you decide based on the points I’ve outlined above.

While the tools listed may market themselves differently, as a consultant, I wouldn’t recommend allotting budget for more than one of these tools as they have too much overlap to produce enough ROI in tandem. Additionally, the vast majority of these tools (all except Segment) don’t offer native integrations between them. So, how do you choose?

Evaluating vendors comes down to: the skillset of the team owning the onboarding of the event orchestration tool (analytics vs engineering vs product), budget considering volume, and the most important purpose of bringing on event tracking.

Update 10/03/22: Analysis only includes in-tool visualization, not transformation. Posthog and Heap added.

A quick note on process: I purposely do not reach out to vendors and talk to sales. This is based on both personal experience and documentation, which I rely on being updated and easy to search through.

Amplitude

Choose if: Product managers want to explore event data directly with the least amount of friction.

• Tracking: SDKs in a variety of languages, native pixel implementation.

• Integrations: Amplitude recently rebranded as a CDP. Sources/destinations to common SaaS tools and warehouses are in beta and growing.

• Analysis: This is where Amplitude shines. Support for custom dashboards directly in the Amplitude UI.

• Pricing: Free up to 10M events/month.

• Other notes: Has proprietary identity resolution (consolidating a user that logs in on multiple devices as one person).

Segment

Choose if: Sending events realtime between a diverse set of sources and destinations needs to be done seamlessly in one click.

• Tracking: SDKs in a variety of languages, webhooks.

• Integrations: This is where Segment shines. Integration list is large and growing. Supports SaaS tools, third-party pixels, ETL, webhooks, and more.

• Analysis: Limited to error alerting and counts of events.

• Pricing: Free up to 1k visitors/month.

• Other notes: A “traditional” CDP.

Rudderstack

Choose if: Event transformation between source and destination is a top priority.

• Tracking: SDKs in a variety of languages, webhooks.

• Integrations: Integration list is large and growing. Supports SaaS tools, third-party pixels, webhooks, and more.

• Analysis: Limited to editing transformations, not data analysis.

• Pricing: Free up to 5M events/month.

• Other notes: This is where Rudderstack shines. Not in the form of in-app dashboards, but the ability to transform events between source and destination. This adds a lot of customization ability for bizops CDP owners. Additionally, they have a proprietary identity resolution (consolidating a user that logs in on multiple devices as one person) similar to other tools like Amplitude.

Mixpanel

Choose if: You need to integrate with an existing CDP and/or if analysis in the form of audience segmentation is a top priority.

• Tracking: SDKs in a variety of languages, native pixel implementation, other event tracking systems (ex. Segment, Rudderstack), the data warehouse as a source.

• Integrations: Integrates mostly with other CDPs and the data warehouse.

• Analysis: This is where Mixpanel shines. Interactive reports, dashboards, and segmentation with a marketing focus.

• Pricing: Free up to 100k monthly tracked users.

• Other notes: Mixpanel is best fit for a team with a lot of engineering talent to support analytical product managers and marketers. Mixpanel allows organizations to take the weight off of analytics teams.

mParticle

Choose if: Running data quality and governance checks on events directly is a top priority.

• Tracking: SDKs in a variety of languages.

• Integrations: Integration list with other SaaS tools is long and growing. Writes directly to customer’s data warehouse.

• Analysis: Limited to error alerting and counts of events.

• Pricing: ? (I genuinely just don’t know)

• Other notes: This is where mParticle shines. Without involving third-party tools, mParticle stresses data quality and governance by providing testing and data cleaning capabilities using the tool itself.

Snowplow

Choose if: You need to use open-source with a lot of customization.

• Tracking: This is where Snowplow shines. SDKs, based on an open-source core. Users can choose to self-host the Snowplow platform and/or build custom solutions.

• Integrations: Integration list focuses on cloud tools like GCP/AWS and the warehouse, with some SaaS tools.

• Analysis: Limited to error alerting and counts of events.

• Pricing: Open source, or hosted: ? (I genuinely just don’t know)

Posthog

Choose if: You need to customize deployment or integrations and want a strong emphasis on experimentation.

• Tracking: SDKs in a variety of languages and webhooks.

• Integrations: Integration list focuses on in-house events and data warehouses, with limited SaaS integrations.

• Analysis: This is where Posthog shines. With strong emphasis on experimentation and insights, Posthog combines this with sought-after capabilities like feature-flags and session recording.

• Pricing: Free up to 1M events/mo.

Heap

Choose if: You require a combination of customized analytics and limited technical requirements for tracking all events.

• Tracking: They market an auto-track feature which doesn’t rely on specific events firing. However, QA and schema contracts should be thought about carefully if using this feature.

• Integrations: Integration list is large and growing, including SaaS tools and warehouses.

• Analysis: This is where Heap shines. Extensive ability to build custom dashboards, segmentation, and templates for product launches.

• Pricing: Free up to 10k sessions/mo.

Notable mentions

While it’s not my preferred approach and doesn’t fully fit under the category, many teams can get started by triggering events from Google Tag Manager by configuring tags based on templates and exposing a user ID in the data layer. While easy to implement initially, keep an eye on ease of use and quality as this solution doesn’t scale so well.

Lastly, of course there’s Google Analytics. This is the absolute easiest to get started with, but isn’t quite event tracking. It offers funnel analysis but at an aggregated level, without the ability to customize user experiences by using that data in other tools. It’s a must have but doesn’t replace the above listed tools as it’s the least granular and most opaque. This, however, is going to change with GA4. If you’re migrating to GA4 already, consider those capabilities when/if choosing additional tooling.

Final thoughts

Phew, that was a lot! Front-end event tracking is complicated, and CDPs can quickly grow in complexity. Whether you center your CDP with an event-tracking tool or in the warehouse, make sure the data in it is well governed and understood.

Thanks for reading! I talk all things product, marketing, data and startups so please reach out. I’d love to hear from you.