Theme Scientist is an A/B testing app for Shopify.
Highlights:
- Theme and Product A/B testing
- Tracking pixel to record stats of each version and show the results in the app dashboard
- Create a test by selecting your 2 versions to test + the time interval, and the app will automatically swap versions and track the stats
- Analyze the success of your test with metrics like Conversion Rate, Orders, Sales, Adds to Cart, Page Views
- Theme Scheduler feature to publish a theme at a specific time in the future
👉 Get Theme Scientist on the Shopify app store.
HOW THEME A/B TESTING WORKS
- Duplicate your live theme then make the adjustments you want to test on the new theme.
- In the app, select the 2 themes you’re testing, choose the time interval you want them to swap, then create the test.
- When the test starts Theme Scientist will publish theme A and begin swapping the 2 themes at regular intervals, tracking stats, and showing the results on the theme test details page.
HOW PRODUCT A/B TESTING WORKS
- You’ll select a product to test.
- Select the attributes you want to test and edit them on the B version.
- Choose the time interval you want them to swap, then create the test.
- When the test starts Theme Scientist will publish version A and begin swapping the 2 versions at regular intervals, tracking stats, and showing the results on the product test details page.
We made it super easy so you just set up the test then kick back and wait for the stats to come in.
Tips for an accurate test
- Test 1 element or section at a time.
- Don’t make edits to a theme or product during testing. If it’s necessary, make the same changes to both versions to preserve the integrity of the test.
- The more traffic you have and the longer your tests run, the more accurate the results.
- Aim for a statistical significance of 95% for best results.
- To accelerate results, run extra traffic to the page(s) you’re testing.
- Spread your email blasts and marketing promotions between the 2 intervals, so send half on day 1 and half on day 2 if you’re doing daily swaps.
- If you set your test interval to daily, a good minimum time frame is 2 weeks so each version sees 1 day of the week.
A/B testing gives you data-driven results to make smarter decisions.