How adding a donors first name to the instant donation page can increase conversion rate | NextAfter
Hoover Institution

How adding a donors first name to the instant donation page can increase conversion rate

Experiment ID: #46813

Hoover Institution

Experiment Summary

Timeframe: 01/28/2021 - 05/01/2021

In this experiment, we added a donor’s first name to the instant donation page once they downloaded the eBook it said “Thank you, First name” while the control just said, “Thank you, friend”. This experiment was run specifically on Facebook and we had a 50/50 split on who saw the control vs the treatment.

The main metric we were looking to track on this experiment was the donor conversion rate. Previously with Hoover we ran this experiment and saw a lift but we didn’t launch it across the board so we wanted to test it again as they had also recently changed their donation platform.

 

 

Research Question

We believe that by adding the first name for those that visit the page will achieve an increase in both donations and revenue.

Design

C: Control
T1: First Name

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 8.4% $0.00
T1: First Name 9.4% 12.2% 67.7% $0.00

This experiment has a required sample size of 5,932 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

We had to stop this experiment early so it did not receive the necessary traffic it needed to validate (67%LoC) but with it seeing a 12% increase in conversion rate and a 43% increase in revenue (99% LoC) we will continue to test this across multiple Hoover offers.

People enjoy seeing their name and it adds that personal aspect and helps reinforce that they completed the signup.

 

Notable changes to other metrics due to the experiment:

  • Revenue experienced a 43.8% increase with a 99% level of confidence.

Observations of the impact on other visitor segments in the experiment:

  • Organic Search visitors had an 8.3% increase in donations with an 86% level of confidence.
  • Paid Search visitors had a 14.3% increase in donations with an 87% level of confidence.
  • 55-64 visitors had a 322.3% increase in donations with a 99% level of confidence.


Experiment Documented by NextAfter

Question about experiment #46813

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.