Smartype Generate: Translate any JSON schema into data collection libraries for web, iOS and Android

When product managers, marketers, or other non-technical stakeholders decide which customer data points their apps should track, an all-too-typical workflow is to create a spreadsheet or some other visual representation of the desired additions or changes to the data plan, and hand this off to the engineering team. The developers are then tasked with referring to this static resource as the source of truth as they code the data collection libraries for each individual platform on which their apps and websites reside. 

This process is highly error-prone (to say nothing of its repetition), even for the most detailed-oriented developers. It is easy to imagine how CustomerID in one library could become Customer_ID in another. If errors like this go unnoticed and move into downstream systems, the list of costly consequences includes inaccurate messaging to customers, wasted spend on vendors, misguided feature changes, and ineffective paid advertising efforts, to name just a few. Over time, these inconsistencies can also make it impossible to tie customer events to a single profile. Assuming the incorrect data is spotted (the best case scenario), marketers have to cleanse the data in the tracking system, if it can be salvaged at all, and developers need to correct inconsistent attribute names, taking valuable time away from building and shipping new features. 

Necessity, they say, is the mother of invention, and the need to solve this persistent data quality problem is what prompted mParticle to create Smartype––a robust data quality feature that translates the properties, values, and types that comprise any JSON schema into usable, statically-typed code in multiple target languages. While Smartype is part of an ecosystem of developer tools originally intended for engineers working with mParticle data plans, its value is by no means limited to these teams. As a platform-agnostic tool, Smartype can help any engineering team streamline the process of implementing and maintaining data collection libraries, and ensure quality and consistency in the data collected across platforms. 

In the rest of this post, we’ll look at how to use Smartype to create data collection libraries for web, iOS and Android apps without writing a single line of code. We’ll also take a look at a practical example of how Smartype can help you maintain event data quality in a web application.

To get started with Smartype, download the latest executable jar CLI from the Github releases page and import it into your project. Then, save a JSON schema representing the data object you would like to implement to your codebase, like the one below representing a view_product event:

In the same directory where the .jar file is saved, run Smrtype init: java -jar <smartype_filename.jar> init. This will ask you a series of questions in the command line about the type of libraries you would like to generate, the file paths of the input JSON, and where the generated libraries should be created. Once you answer these questions, init will create a smartype.config.json file in your project.

Now, we are ready to run Smartype generate: java -jar <smartype_filename.jar> generate. This will read your configuration file and generate libraries for the platforms that you have specified. Behind the scenes, Smartype generate converts these JSON schema into Kotlin data classes, then using Kotlin Multiplatform to convert these classes into various platform-specific languages. Leveraging Kotlin allows Smartype to translate a JSON schema that defines expected declarations mapped to actual declarations in platform-specific code.

When Smartype generate has finished running, you will be able to see the code Smartype has created for all three platforms in your smartype-dist directory (assuming you have not specified a separate output directory in your configuration file). For Android apps, Smartype has created an aar file that you can  include in your projects. For iOS, Smartype built a “fat” dynamic framework ready for production use. For the web, Smartype builds a set of JavaScript and Typescript files that you can import directly into your application’s codebase. 

Let’s consider a use case for Smartype in a sample web application. With our view_product object represented as a JavaScript class, we can use this object to track which products our visitors are viewing in our UI, and maintain type consistency with our data model when we integrate this object into our production code. 

After we import Smartype, we will create an instance of the SmartypeApi object, which surfaces our viewProduct object as well as other methods to add and remove receivers for our data. After calling viewProduct, again we can see that our IDE surfaces all of the properties and data types we specified in our JSON schema as the list of arguments that viewProduct can accept:

As you can see, Smartype doesn’t just bring data models into the codebase in the form of individualized data collection libraries. It also displays all events in the data model directly in the autocomplete function of any IDE, allowing developers to easily visualize their complete data model without having to leave the editor. In the case of our web application, this means that when we attach this type-safe message object to a user event in our UI, we can be confident that the data we send to our receivers is consistent with our data plan. 

In a later post, we will look at how to use Smartype to send data to Firebase as a custom receiver. Until then, read more about the benefits of Smartype, and watch a LinkedIn Live video on how Smartype works in an Android project and extends the functionality of the mParticle Data Planning API.