How adding video to an donation page affects revenue
Texas State Historical Association
Experiment Summary
Timeframe: 08/26/2015 - 09/01/2015
As part of their fiscal year end campaign, the Texas State Historical Association (TSHA) wanted to showcase the educational content that they produce for use in Texas classrooms. The vast majority of the emails used in the campaign were relatively long stories that linked to a landing page with images to reinforce the copy.
We had a hypothesis that a video could tell the story better than simply text alone. We also know from previous experiments that a video is able to drive significantly more traffic to a landing page. Based upon this hypothesis, we created a treatment that began at the email and linked to a landing page with a video that then transitioned into the donation form. We also linked to the non-video landing page in both emails for those links that made direct asks for a donation.
Research Question
Does the usage of a video in an appeal increase donor conversion or revenue?
Design
Results
| Treatment Name | Revenue per Visitor | Relative Difference | Confidence | Average Gift | |
|---|---|---|---|---|---|
| C: | No Video | $0.12 | $41.67 | ||
| T1: | Video | $0.02 | -81.0% | 99.9% | $35.00 |
This experiment was validated using 3rd party testing tools. Based upon those calculations, a significant level of confidence was met so these experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 77.4% decrease in conversion rate
× 16.0% decrease in average gift
81.0% decrease in revenue
Key Learnings
By adding an image of the video in the email, we were able to increase the unique visits to the donation page by 71.4% for the video treatment. However, these visitors were motivated to watch the video instead of giving a gift. As a result, the increased traffic had a significantly lower response rate.
Due to the significantly lower response rate, the email with video links resulted in an 81% decrease in overall revenue.