Enhancements to mParticle’s developer tools make it easier to collect data and ensure quality at the source

Customer data is only valuable if it is accurate. Low quality data in your internal pipelines and downstream tools can result in a host of negative outcomes––like inaccurate messaging, misguided business decisions, and engineering time wasted on tedious debugging. This is why any team that collects and leverages customer data has a vested interest in protecting the quality of their data pipelines. 

Ensuring cross-platform data quality, especially for brands that collect data across many different customer touchpoints, is easier said than done. It is no small feat to guarantee accuracy and consistency of all demographic and behavioral data collected across sources like web, iOS and Android applications, OTT, point-of-sale systems, and others. While the consequences of mistakes in this process impact teams across the organization, the responsibility of avoiding and remediating them falls on the shoulders of engineers. 

At mParticle, we understand that maintaining data quality is not only critical to data health across an organization, but essential to the developer experience. When engineering teams are able to prevent data quality issues before they arise, they are also avoiding hours of time-consuming debugging work that comes from bad data entering internal systems. This is why we treat data quality as a pillar of the developer experience, and invest in tools and workflows that make it easy for engineers to seamlessly and accurately translate their team’s data plan into data collection code. 

Today, we are excited to announce the general availability of three developer tools designed to help engineers seamlessly and accurately collect data across platforms: SmartType, the Data Planning Snippet Tool, and the Data Planning API. 

What’s the best way to avoid errors when you write event collection code? How about letting an automated code generation tool translate your data plan directly into working events for multiple platforms, and provide intelligent code completion directly in the IDE? Smartype does exactly this. 

The first step in leveraging Smartype to develop your data collection code is to import the latest version of your mParticle data plan into your project. There are multiple ways to do this, but using the mParticle CLI gives you the flexibility to automate this action in ways that work with your CI/CD process. 

Here’s how this works:

Step 1: Download the mParticle CLI

Assuming you have Node downloaded, you can download the mParticle CLI by running: npm install -g @mparticle/cli.

Step 2: Download your Data Plan JSON

With the mParticle CLI installed, you can import your Data Plan JSON into your project. First, obtain the following information:

• Data Plan ID
• Data Plan Version
• Workspace ID
• Client ID (mParticle access token)
• Client Secret (mParticle access token)

Then run the following CLI command: 

mp planning:data-plan-versions:fetch --dataPlanId=$DATA_PLAN_ID -- versionNumber=$DATA_PLAN_VERSION --workspaceId=$WORKSPACE_ID --clientId=$CLIENT_ID --clientSecret=$CLIENT_SECRET --outFile=$OUT_FILE

You may want to consider using the API Credentials interface, which makes it easy to create and manage your mParticle access tokens for Data Planning. 

Once your Data Plan JSON is has been downloaded, here is how you can use Smartype to create data collection code directly without needing to import your data plan into your project at all: 

• Download the latest executable Smartype jar CLI. 
• In the directory where this .jar file is saved, run Smrtype init: java -jar <smartype_filename.jar> init
• The init command will prompt you with several questions, including the path to your Data Plan JSON, to create a smartype.config.json file
• Once Smartype has created a smartype.config.json file, run Smartype generate: java -jar <smartype_filename.jar> generate

After you run the smartype generate command, Smartype leverages Kotlin Multiplatform to convert the properties, values, and data types that are found in your data plan into invocable methods for your platform of choice (Web, iOS, or Android). This blog post and accompanying video covers how to use these methods in a web project to implement event collection without needing to manually translate any aspect of your data plan into working code. 

Data plans are constantly growing and evolving, which means that implementing and maintaining them is an ongoing process for engineering teams. This is why we provide developers with multiple tools and workflows for translating data plans into code. While Smartype is powerful and flexible, certain implementation use cases––like updating a single event as opposed to a full data plan version, for example––may be better served with a simple UI-based tool rather than a full-fledged CLI. 

This is where our Data Planning Snippets tool comes in. This tool has similar functionality to Smartype in that it translates your data plan as a JSON file directly into working code for Web, iOS or Android. Rather than doing this within a command line tool, however, the Data Planning Snippets Tool exposes a simple two-paned UI that lets you paste your data plan as JSON on one side, and instantly see production-ready code on the code on the other that you can copy directly into your projects. 

The Data Planning Snippets Tool is particularly powerful when used as part of a cross-functional workflow in which both technical and non-technical teams leverage mParticle’s Data Plan Builder to create and establish a source of truth for their data plans. A simple Google Sheet, the Data Plan Builder allows teams to easily define data plans based on industry-specific templates, and automatically translate their table-based data plan into JSON schema by running a script:

This article describes a data planning workflow centered around the Data Plan Builder in greater detail. 

While the Data Planning Snippets Tool has been available to mParticle customers in beta status since last year, we are now excited to announce the general availability of this powerful feature for helping engineering teams ensure data quality with ease. 

By allowing you to import the latest version of your Data Plan into your projects directly from your code, and performing CRUD operations on Data Plans without entering the UI, the Data Planning API enables you to fully automate the process of including Data Plans into your app development CICD pipeline and automate the data quality process.

We recently announced the general availability of this API, and with that release came powerful new data quality features for developers, including:

• A Postman collection that allows you to access all endpoints on the Data Planning API, and easily perform create, read, update and delete operations on your data plans
• Sample recipes for Node and Python that make it easy to get started automate the deployment and management of Data Plans, and incorporating them into their CICD pipeline
• An Open API spec for creating helper SDKs
• Updates to our Web, iOS and Android sample applications that let you see mParticle data plans in fully-functioning production code

Smartype and the Data Planning Snippet Tool give engineering teams powerful and flexible options when it comes to maintaining data quality in their event collection process. These are certainly not the only parts of the mParticle developer toolkit that help engineers implement data collection with confidence. Here are some other features and workflows that help engineers protect data quality: 

• Live Stream: This feature of the mParticle UI shows events entering the system in real time, and immediately surfaces inconsistencies between expected and collected data events. 
• Smartype Hubs: Use GitHub actions to integrate data plan imports into your CI/CD process, and never implement event collection based on an outdated version of your data plan.