Split testing
Use A/B testing to improve flow metrics.
Last updated
Was this helpful?
Use A/B testing to improve flow metrics.
Last updated
Was this helpful?
To get a quick start on traffic splitting within Formsort itself, you can assign weights to individual variants and direct your responders to the flow URL, which looks like https://<client-id>.formsort.app/flow/<flow-name>. This will serve a pseudo-random variant according to the weights given.
Assigning weight to deployed variants will distribute flow visitors according to the weight given to the variant. For instance, say a flow has two variants - variant A and variant B - and both have a weight of 50. When the flow URL is loaded, half of the responders will be sent to variant A and the other half to variant B.
In the Domains tab of your Studio, pick a flow from the flow to load for this Hostname option. Do not specify a variant.
This is the same process as using a flow URL to serve weighted variants, only with an extra layer of addresses. The custom domain points to the flow URL, and Formsort handles splitting incoming traffic.
Another, more comprehensive pattern is to add unique variant URLs to an upstream testing tool such as Optimizely. These purpose-built tools are much better at handling traffic splits and can aggregate experiment data that spans across the user journey - either upstream, downstream, or inside of Formsort.
For the most part, yes. Deploying a superceding update will:
ensure new users have the most recent variant revision served and,
push users that have previously interacted with the variant onto the most recent variant revision.
The users that will not get pushed to the "superceded" revision are those that are returning after visiting a different variant.
Let's say you you're ready to move on from variant A, so you:
Push a superceding update to variant B.
Point all your links and custom domains to variant B.
Remove the weighting from variant A.
However, there are users have visited variant A already!
The returning "A" variant users will still have variant A served when they return to finish their form, even if you've removed weigthing from variant A. This is because the UUID for the variant is still cached in the returning user's browser.
At this point, it's time to deprecate variant A entirely:
Once A is archived, no users will be able to access it, and returning A variant users will be directed to variant B!
Only new traffic is split among variants according to weight. Once the user has interacted with a specific variant, that will be stored in their browser.
Once you have your variants appropriately weighted and your configured, you can point a flow to your domain and have random variants served to new traffic.
This pattern does not require weight to be added to individual variants, since traffic should be split upstream of Formsort. The only thing you'll need is the variants you want to use, and the URLs for them!
In the process of testing different variants, there will come a point where you'll want to move off a particular variant entirely. The thing to be aware of is that returning responders will still get directed to the variants they have previously interacted with (due to ), so it's likely your analytics will pick up events on variants that you thought you deprecated long ago. This can be a problem!
The solution here is to the variants you want to deprecate, and set a redirect to URL. This pulls old or unwanted variants out of circulation for all traffic, and ensures that returning visitors are pushed to the variant you want them to see.
But wait, won't push all users onto the most recent variant deployment?
variant A.
During archiving, set the redirect to URL to either variant B, , or your Custom Domain (which in turn may point to the flow URL).
