The Future of CDPs: Packaged and Composable

The conversation dominating the customer data space right now is the idea of ‘composable’ vs ‘packaged’ CDPs.

Glenn Vanderlinden from Human37 had a great post recently on Arpit Choudhury’s blog, Data Beats, that is worth a read. 

At the core of the discussion is the idea of bundling and unbundling…We all know the quote from Jim Barkadale, “there’s two ways to make money on the internet, bundling and unbundling.” So I’m not surprised that this seems like a never ending debate. 

The question is just because you can unbundle a CDP, should you?

For those unfamiliar, there is a convergence between two data architectures - one that takes an opinionated approach to solve challenges specific to customer data, and one that takes a more of a generalized approach that can accommodate the various types of data created within an org, including but not limited to customer data. The first is anchored by the CDP and the latter, the cloud data warehouse. 

I personally find such debates somewhat silly, and find that the world tends to operate in less extremes. Admittedly, my views have changed quite a bit over the past couple years and I do believe that the CDP can be made better by Cloud Data Warehouses like Snowflake and BigQuery. 

Jerry Chen from Greylock wrote about the different layers in the enterprise stack a few years back in a great post called The New Moats. In the post he discusses the three layers that any enterprise has in their various tech stacks. They are:

1. Systems of record- a source of “truth” where valuable information resides. There are usually multiple SORs throughout an organization given the unique nature of, and required treatment of each of the various sets of information. 
2. Systems of intelligence- platforms where data can be unified across various applications and ML/AI can be applied to unlock valuable insights. 
3. Systems of engagement- the tools where rich interactions between businesses and humans can be managed. 

I’d argue that there is a fourth category:

4. Systems of movement (or integration). A system of movement helps orchestrate data interchange on a many-to-one-to-many basis, serving as middleware between sources and destinations with value added capabilities in transit. If we envision these systems as a stack, I’d argue the system of movement connects to all layers. 

The reason systems of movement are valuable is that in any highly networked system, the quality of interactions between the components is what dictates success or failure, and having a specialized vendor to own the connection between components adds a safeguard against data chaos. 

Will the CDP unbundle isn’t necessarily the right question. 

I think it’s clear that the Data Warehouse is a great choice as the System of Record for many organizations, and that the opportunity for a data layer like a CDP will need to focus less on management of data, and more on providing insights and enriching the data with additional customer context in transit. In other words, in some data architectures, CDPs will cede data management capabilities to cloud data warehouses, focusing on value-add activities during the data movement process. Separating data management and data movement should be viewed as a natural evolution. 

I believe the real question is not ‘if’ but, how much of the CDP will become unbundled?

The answer is that it’s a matter of marginal utility, not just total utility. The equation for success is the following:

The incremental value offered by multiple specialized players must be greater than the increase in total cost of ownership due to incremental complexity (in workflows, integration, and maintenance).

Unbundling should be viewed as accretive until the point where you start to experience diminishing returns, represented by this graph below, where the total number of units are represented by the number of disparate vendors.

I think it’s reasonable to argue that there is incremental value by unbundling the data layer into systems focused on data management, and those focused on data movement, but it becomes questionable to unbundle the system of movement further, and would advocate for an end-to-end system of data movement.

A false dichotomy.

The idea of composability feels liberating - “do it on your terms and be in control” or often is expressed as “own your data!” It’s a great story, and we agree with the concepts. But the dichotomy between composable and packaged may be false.

Over the past couple years, some vendors have referred to “composable” as an architecture that utilizes disparate tools which are componentized, and can be integrated in order to replicate what an end-to-end solution offers. These vendors have argued for disparate tool selection because each of them wants the market opportunity, but is this good for the customer? We must acknowledge the value of componentized services, such as SOA, microservices, etc. but we must also acknowledge that the value accrues from the "componentized" aspect, not the "disparate."

In our experience, teams often want to minimize the number of vendors, but they want the power and flexibility that composability offers. Componentization affords the business more control and transparency, with the ability to replace (via build or buy) just the parts they want to innovate — whereas disparate vendors create incremental costs, and overhead.

Recall the equation from above, the value from componentization must be greater than the increase in total cost of ownership due to disparate vendors. 

We also take exception with the implied negative space. Composable is flexible, so packaged must be rigid. Composable is transparent so packaged must be opaque. Composable assists with data ownership so packaged inhibits it. 

I’d agree that rigidity is often limiting, flexibility is preferred, and data ownership is important. I’d also agree that transparency is necessary, and opacity is concerning. 

But remember, transparency is not limited to any company type or category, however; some CDPs are more transparent and flexible than others, while some reverse ETL vendors are more transparent and flexible than others. 

At mParticle, we have open sourced our native data collection SDKs since day 1. We built our identity resolution capabilities to be completely configurable and programmatically accessible. We offer real-time APIs for every part of our platform, so the data can be accessed in any way teams may prefer. We publish a change log monthly.  We offer custom based roles so teams have total control over how the platform is used by who down to screen level access. Our value based pricing is the only solution that allows teams to designate event classes for greater cost efficiencies. We consider this to be a good indication of transparency and flexibility. Greater flexibility in choice at the application layer was the initial premise of the CDP after all. 

But what if teams didn’t need to choose between composable and packaged? For argument’s sake, if composable and packaged are considered polar opposite concepts, each with their sets of pros and cons, then perhaps ‘configurable’ is the midway point between the two. A midway point that balances transparency, flexibility, and control of componentization with the operational scalability of an end-to-end solution.

Where do we go from here?

A packaged CDP can, and in some cases should, leverage the powerful data management capabilities offered by the Cloud Data Warehouse. With the announcement of WarehouseSync, we are fully embracing the data warehouse ecosystem. We will be aggressively leaning into this exciting trend. When we purchased Indicative in the beginning of 2022, one aspect that was really exciting about it was that they had built natively on top of the Data Warehouse. 

By combining the best of all worlds, you can easily ensure that one source of truth powers the entire application layer including the systems of engagement, intelligence, etc. In this construct, the cloud data warehouse may serve as the system of record, and the CDP — the system of movement.

With capabilities such as role based permissions, flexibility in the pricing model, and deep integrations into the data warehouse ecosystem — configurability can be easily achieved. A configurable approach will not only get you on a fast path to value realization but also ensure a low total cost of ownership as well. 

Data is ultimately a team sport. The work done by data engineering teams should be leveraged by the business rather than isolated from it. By combining the best aspects of packaged and composable, we will see a new wave of innovation — one that drives the industry forward.