How a more personalized value proposition affects remarketing conversion rate Experiment ID: #7376

Hillsdale College

Founded in 1844, Hillsdale College is an independent liberal arts college with a student body of about 1,400. Hillsdale’s educational mission rests upon two principles: academic excellence and institutional independence. The College does not accept federal or state taxpayer subsidies for any of its operations.

Experiment Summary

Timeframe: 8/29/2017 - 9/10/2017

Hillsdale College decided to launch remarketing ads on Facebook to users who visited the signup page for their acclaimed “Constitution 101” course. They deliberated about the copy on these remarketing ads—should it re-state the original value proposition of the course, or should it encourage them to finish completion of the course?

Since this was a testable hypothesis, they created two ads. The first told the user that their course was still available, and encouraged them to finish signup. The second used a standard value proposition that encouraged them to grow their understanding of the Constitution through the course, but didn’t show any indicator that they had shown previous interest.

Research Question

How will a more personalized value proposition affect email acquisition rate?


C: ad3
T1: ad2 - activate


Treatment Name Conv. Rate Relative Difference Confidence
C: ad3 0.18%
T1: ad2 - activate 0.90% 393.4% 99.9%

This experiment has a required sample size of 1,240 in order to be valid. Since the experiment had a total sample size of 6,124, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 393.4% increase in conversion rate
× 0% increase in average gift

Key Learnings

The treatment produced a 393% lift in conversion rate, which reveals a few key learnings about user behavior and remarketing ads. First it shows that reminding people that they failed to complete signup might make them feel guilty and less likely to sign up. Second, it reveals that this is remarketing, and could possibly indicate a “creepiness” factor that they are being followed around the web.

But most importantly, it shows the power of removing the true value proposition from an ad—even an remarketing ad—and the concurrent effect that it has on motivation. Even if a user has already absorbed the value proposition from a previous ad, remarketing ads must contain a strong value proposition as well if they are to complete conversion the second time around.

Share this research with a colleague

Our mission is to help elevate the field of fundraising by openly sharing our research and inspiring a wider community of testing and optimization. If you have found our research to be helpful, insightful, or even just interesting—please share it with a fellow fundraiser.

Get the Detailed Case Study

Get a copy of the slides that highlight the different elements of the experiment, the variables that were changed, and the key takeaways and learnings that you can apply to your own organization.

Pick your format of choice:

Experiment Documented by...

Jeff Giddens

Jeff is a Senior Vice President at NextAfter. If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.