Americans for Prosperity

Direct Mail Style Headings and Boldings

Experiment ID: #94536

Americans for Prosperity

Experiment Summary

Timeframe: 05/19/2022 - 06/26/2022

AFP’s donation pages were relying on high school essay style formatting–a bold header and standard punctuation and formatting throughout. We know that direct mail sees an increase in response with emphasized material and we tested that here in digital. The idea is to have the bolded and/or emphasized text tell the story themselves and function as a TLDR that can either 1) motivate someone to donate immediately, or 2) motivate them to read the rest of the donation page copy and then donate.

Research Question

We believe that direct mail style headings and bolding for donation page visitors will achieve increased conversion rate to donor.

Design

C: Control
T1: Variant 1 - DM Style Headings and Bolding

Results

  Treatment Name Conv. Rate Relative Difference Confidence Average Gift
C: Control 0.06% $0.00
T1: Variant 1 - DM Style Headings and Bolding 0.24% 289.0% 81.4% $0.00

This experiment has a required sample size of 3,690 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

We closed the experiment at 289% lift and 81% stat sig b/c this donation page followed an iVol and was orphaned by a MarComms change to the iVol.

This experiment confirmed we should use DM-style formatting to better communicate with potential donors and increase conversion rate.

Going forward, new pages will be built with this learning in mind and we will go back and update current donation pages.


Experiment Documented by NextAfter

Question about experiment #94536

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.