How the kind of offer on the homepage impacts name conversion Experiment ID: #11954

Boys Town

Experiment Summary

Timeframe: 2/7/2019 - 2/22/2019

On Boys Town’s homepage, they offer people the opportunity to sign-up for their eNewsletter. With the significant volume of traffic on their home page, we hypothesized a different kind of offer might increase name conversion. To test this hypothesis, we proposed testing a content specific offer instead of the eNewsletter.

Research Question

Which offer will convert website traffic into names for the organization more effectively?


C: Control
T1: Tantrums offer


Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.40% $0.00
T1: Tantrums offer 1.1% 171.6% 95.7% $0.00

This experiment has a required sample size of 1,218 in order to be valid. Since the experiment had a total sample size of 2,553, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 171.6% increase in conversion rate
× 0% increase in average gift

Key Learnings

The content specific offer increased name acquisition by 171%. While the treatment may only be something a smaller segment of the homepage traffic might be interested in based on the content being offered, the value of this offer out-weighed that of the control.  The treatment also communicated value to the end reader in the call-to-action. The call-to-action for the treatment was “Get this free eBook” where the control was “Sign up Now”. The control communicates that there is work required to get the offer. 

Experiment Documented by...

Courtney Gaines

Courtney is a Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.