Improve customer experiences with A/B testing

A travel booking company was looking to increase sales made through their mobile app, as well as encourage customers to open the app more frequently. To achieve this, marketers and product managers knew they needed to make purchasing on the app as seamless as possible and so they wanted to test how different screen layouts would affect purchase rates. But to maximize the effectiveness of these changes and minimize the amount of time and resources, they needed to be able to quickly test variations across their entire customer database for statistical significance and act on those learnings without forcing users to update the app every time they made a change. This workflow will show you how to use mParticle and Optimizely to power your experimentation and provide your marketing and product teams with a way to easily test, learn, and deploy winning digital experiences in real time.

• Personalize content experiences, including homepage
• Test new product features
• Experiment with messaging, layout, and placement
• Improve customer experiences and retention
• Test variants quickly, without additional engineering/design time

Optimizely is an optimization platform that allows you to test how new product features, layouts, messaging, and personalization efforts affect how customers experience your site or mobile app. Different variants of your experiment, like the color or placement of a checkout button for example, are shown to randomly selected sample groups of real customers. As customers interact with your site, these actions and movements can be tracked and reported back to your Optimizely dashboard where you can measure which variation of your changes seem to be most effective.

mParticle makes it easy to add Optimizely to your tech stack without adding additional third-party code to your app, lowering the risk of dependencies that could result in a sub-optimal end-user experience. When integrated, mParticle's API collects users’ in-app and web behaviors, like product views and location data, forwards them to Optimizely automatically. As marketers experiment with front-end web changes, like copy, page layouts, and ad or product placement variants, additional contextual data collected from second- and third-party sources by mParticle can be used in Optimizely to determine what each user receives based on user attributes and previous actions or engagements.

The beauty of using mParticle and Optimizely to build an experimentation platform is that, while normally you would have to design and release changes to see how they would impact your business goals or product roadmap, you can instead run multiple experiments at once to save you time and resources while ensuring that regular service continues for customers as you run experiments.

Prerequisites: An Optimizely account, your Optimizely experiment set up in a project, and your project ID.

For web, you’ll want to embed the mParticle JavaScript SDK into your standalone web app/

1. Create an API key by going to Setup, then clicking inputs in the mParticle dashboard. Select Web to generate your API key and secret.

2. In the development version of your web app, insert the the code snippet, found in our repo here, and add in the API key you just generated.

3. Check your Live Stream to ensure data is being sent to mParticle. If data does not populate in the Live Stream, refresh and check that you’ve entered the API key correctly, and that your config object contains isDevelopmentMode:true.

For IOS or Android:
1. Generate a Key and Secret to enable mParticle to collect data from your mobile app by going to Inputs and selecting your app's platform, then clicking "Issue Keys."

2. Depending on your app's platform and your preferred method, you'll have to add your SDK dependencies to either your CocoaPods Podfile or Cartfile for iOS or to your build.gradle for Android via Maven Central or jCenter.

3. Follow the instructions to initialize the SDK for your platform detailed here for iOS or here for Android and input your generated Key and Secret.

1. Add the Optimizely integration from the directory and choose event output to set up the integration. This will send raw user event data to Optimizely. Then, click Go to Setup.

2. In Event configuration, you’ll be prompted to name your configuration and enter your Optimizely Product ID.

For Web: When you add your Optimizely Product ID, the mParticle Web SDK will automatically load the Optimizely Web Client SDK for your project.

For mobile: Add the Optimizely Kit to your iOS or Android app by adding this snippet to your Cocapods Podfile or build.gradle.

3. You will then be taken to the Connections page, where you will choose you input (iOS in this case) and then choose your Optimizely configuration as the output and enter your event and file intervals.

4. From here you can add rules to dictate what kind of data is forwarded to Optimizely. You can also use the User Sampling feature to limit the amount of data forward to help control costs.

Once you have integrated Optimizely, you can use this reference code to activate your experiment. Note—You must use same user ID that the mParticle integration is configured to use, so that events are associated with the correct user.

By default, mParticle will use the device application stamp if present. You can reference this page for additional supported identity types. If you’ve configured a different ID than device application stamp, be sure to use that ID (if present) to activate experiments. You can also refer to Optimizely's docs for further information on activating your experiments here.

As customers see and interact with different variants of your experiment, mParticle will forward your data and you should see it begin populating within your Results page in Optimizely.

From here on, the sky's the limit! Beyond testing whether specific CTAs, app layouts, or button placements, you can use this experimentation platform to personalize content among many other use cases. Optimizely has a great set of popular experiments that you can reference for inspiration here.

If you'd like to learn more about how mParticle and Optimizely can help you leverage user behavior data from both mobile and web to create personalized digital experiences based on data insights, get in touch or try this workflow by clicking the button below!

Optimizely docs

Integration page

Additional information on setting up experiments

Testing ideas from Optimizely