mParticle launches open-source data validation tools
Toolset includes new APIs and Smartype, a new product to help engineers ensure proper event collection at run time, now in beta.
NEW YORK, NY—(May 13, 2020) – mParticle, the Customer Data Platform (CDP) of choice for multi-channel consumer brands, today announced the beta release of a new open-source developer toolset to give engineering teams instant data quality protection and feedback in their integrated development environments (IDE).
This toolset also features Smartype, which translates any JSON based data model into strongly-typed code. Additionally, this toolset pairs mParticle's Data Planning API, the command-line interface (CLI), and static code analysis (linting), to help developers validate their data model throughout their software development lifecycle.
New Data Planning API, and Smartype
Earlier this year, mParticle released the next generation of its Data Master product, allowing engineering, product, and marketing teams to collaboratively design and enforce a consistent data model. But once a development team has created a data plan, developers still need an easy and foolproof way to implement the plan without having to leave their editor to cross-reference a web UI—a process that can be slow, error-prone, and doesn’t provide change management.
The newly available Data Planning API allows engineering teams to use Data Master programmatically through an HTTP API. With the Data Planning API, developers can easily store data plans in their source code, and use their own SDLC and approval processes to define the data model to best suit their needs. The Data Planning API can create, read, update, and delete Data Plans, as well as integrate mParticle’s suite of Data Planning tools to perform compile-time and runtime data quality verification.
Together, the Data Planning API and Smartype allow developers to translate a Data Plan (or more generally, Data Model) into usable code to improve implementations and minimize data quality issues by improving instrumentation quality and consistency.
Smartype addresses common data quality issues through these features:
- Data Plans as code
All constants called for in the Data Plan (event names, attribute names, enum values) available as a machine-readable JSON Schema - Data Planning API
Programmatically perform all CRUD operations on Data Plans to decrease the time-to-data-quality and time to implement - Compile-time feedback
Feedback developers can use to conform to a Data Plan - thereby limiting the likelihood of an implementation issue. - Generate typesafe libraries
Programmatically generate a strongly typed client library across a variety of languages from a centrally defined Data Model. - Multiplatform code sharing
Built on Kotlin so that you can share your common Data Plan implementations across platforms (Android, iOS, Web, and server-side environments). - Managed within codebase
Any changes to Data Plans result in a regenerated type-safe library that is always in-sync with the latest Data Plan.
Smartype makes it easier for development teams to define a data model, collaborate against a data model programmatically, implement it correctly at compile-time, report validations in time to fix them, and transform data points in the plan on an ongoing basis.
CLI and Linting
In addition to the Data Planning API and Smartype, the new developer toolset also includes mParticle CLI and Linting. For developers, the new CLI offers an easy way to interface with the mParticle platform through the command-line instead of the user interface.
Using the CLI allows developers to create, maintain, update, and even delete data plans with version control provided by a git repository. The CLI also enables developers to use linting tools to statically lint code against a data plan as they develop to ensure adherence to their data plan and that only high-quality data is logged to their workspace. The lint tool can also be used within CI to ensure validity throughout the entire development lifecycle.
For additional details on Developer Tools, including Smartype, visit: https://www.mparticle.com/blog/smartype, or go to https://www.github.com/mparticle/smartype to view the open-source code.
About mParticle
mParticle believes that better customer experiences begin with better data. Its Customer Data Platform helps engineers, product managers, and marketers at companies like Spotify, Paypal, NBCUniversal, Starbucks, and Airbnb improve data quality and simplify integrations across the entire marketing stack. Founded in 2013, mParticle is headquartered in New York City with offices in San Francisco, Seattle, and London and manages more than 500 billion API calls monthly. For more information, please visit www.mparticle.com.
