Americans for Prosperity

How posing questions in your email impacts clickthrough rate

Experiment ID: #35128

Americans for Prosperity

Experiment Summary

Ended On: 08/24/2020

As a part of promoting a new white paper for their subscribers, Americans for Prosperity was looking to distribute a relevant and new white paper produced by their team.

In reviewing the control experience, we wondered if posing questions to inspire the reader to create demand for the white paper would increase email clickthrough rate.

Research Question

How does posing questions that are only answered by clicking to download and read the linked promotion impact clickthrough rate?

Design

C: Positioned as a statement
T1: Positioned with Intrigue

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Positioned as a statement 2.4%
T1: Positioned with Intrigue 4.3%76.2% 96.1%

This experiment has a required sample size of 718 in order to be valid. Since the experiment had a total sample size of 1,607, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    76.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

With a 96.1% level of confidence, we observed a 76.2% increase in clickthrough rate.

The hypothesis that if we use intriguing questions to create demand that is only fulfilled after the reader clicks to received/download the materials promoted within an email seems to hold true.

If the goal of your email is not to cultivate or maximize readership (but rather to produce the highest number of clicks), then the goal should be to inspire the highest number of clicks by creating demand that is only fulfilled after they take the action you want them to (to click your email links).


Experiment Documented by Greg Colunga
Greg Colunga is Executive Vice President at NextAfter.

Question about experiment #35128

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.