Texas State Historical Association

How adding video to an donation page affects revenue

Experiment ID: #11696

Texas State Historical Association

Experiment Summary

Timeframe: 08/26/2015 - 09/01/2015

As part of their fiscal year end campaign, the Texas State Historical Association (TSHA) wanted to showcase the educational content that they produce for use in Texas classrooms. The vast majority of the emails used in the campaign were relatively long stories that linked to a landing page with images to reinforce the copy.

We had a hypothesis that a video could tell the story better than simply text alone. We also know from previous experiments that a video is able to drive significantly more traffic to a landing page. Based upon this hypothesis, we created a treatment that began at the email and linked to a landing page with a video that then transitioned into the donation form.  We also linked to the non-video landing page in both emails for those links that made direct asks for a donation.

Research Question

Does the usage of a video in an appeal increase donor conversion or revenue?


C: No Video
T1: Video


 Treatment NameRevenue per VisitorRelative DifferenceConfidenceAverage Gift
C: No Video $0.12$41.67
T1: Video $0.02-81.0% 99.9%$35.00

This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    0% increase in traffic
× 77.4% decrease in conversion rate
× 16.0% decrease in average gift
81.0% decrease in revenue

Key Learnings

By adding an image of the video in the email, we were able to increase the unique visits to the donation page by 71.4% for the video treatment.  However, these visitors were motivated to watch the video instead of giving a gift. As a result, the increased traffic had a significantly lower response rate.

Due to the significantly lower response rate, the email with video links resulted in an 81% decrease in overall revenue.

Experiment Documented by NextAfter

Question about experiment #11696

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.