How adding a popularity number impacted email clicks and donations on a NonDonor audience
Experiment ID: #192767
Hoover Institution
Experiment Summary
Timeframe: 11/07/2024 - 11/13/2024
The Hoover Institution is a public policy research center and think tank located at Stanford University in California, United States.
Hoover recirculates a new content offer every Thursday by sending out an email to their Donor and NonDonor audiences. For this specific recirculation of one of Hoover’s most popular offers, Victory and the Savior Generals. This experiment was for their non-donor audience.
We wanted to update the copy to have these updates:
- Shorter
- Mentions well-known and respected Hoover fellow, Victor Davis Hanson, at the beginning of email
- And have the callout “Already downloaded by more than 30,000 people in the last 6 months, you don’t want to miss your chance to learn from history’s most influential leaders from one of our country’s most esteemed historians.”
Research Question
We believe that refreshing the email copy for Hoover’s non-donor audience will achieve an increase in clicks and conversions because they see that other people wanted this with the “30,000 people” callout.
Design
C: Version A
Array
(
[0] => Array
(
[ID] => 68206
[id] => 68206
[title] => Version_A.png
[filename] => Version_A-19.png
[filesize] => 123921
[url] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[link] => https://www.nextafter.com/experiments/how-adding-a-popularity-number-impacted-email-clicks-and-donations-on-a-nondonor-audience/attachment/version_a-png-110/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_a-png-110
[status] => inherit
[uploaded_to] => 68196
[date] => 2024-11-13 18:03:46
[modified] => 2024-11-13 18:03:46
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 469
[height] => 933
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_A-19-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Version_A-19-151x300.png
[medium-width] => 151
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[medium_large-width] => 469
[medium_large-height] => 933
[large] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[large-width] => 469
[large-height] => 933
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[1536x1536-width] => 469
[1536x1536-height] => 933
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[2048x2048-width] => 469
[2048x2048-height] => 933
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Version_A-19-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_A-19-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Version_A-19-25x50.png
[admin-width] => 25
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[full-width] => 469
[full-height] => 933
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Version_A-19-372x740.png
[thumb_372-width] => 372
[thumb_372-height] => 740
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_A-19-314x625.png
[single_post_thumbnail-width] => 314
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Version_A-19-16x32.png
[thumb_32_32-width] => 16
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Version_A-19-24x48.png
[thumb_48_48-width] => 24
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Version_A-19-32x64.png
[thumb_64_64-width] => 32
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Version_A-19-64x128.png
[thumb_128_128-width] => 64
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Version_A-19-34x67.png
[thumb_122_67-width] => 34
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Version_A-19-77x153.png
[thumb_153_153-width] => 77
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Version_A-19-82x163.png
[thumb_130_163-width] => 82
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Version_A-19-89x177.png
[thumb_232_177-width] => 89
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Version_A-19-327x650.png
[thumb_272_325-width] => 327
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Version_A-19-174x346.png
[thumb_271_346-width] => 174
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Version_A-19-136x271.png
[thumb_271_271-width] => 136
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Version_A-19-86x172.png
[thumb_272_172-width] => 86
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Version_A-19-232x461.png
[thumb_372_461-width] => 232
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Version_A-19-251x500.png
[thumb_372_500-width] => 251
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Version_A-19-136x270.png
[thumb_385_270-width] => 136
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Version_A-19-121x241.png
[thumb_416_241-width] => 121
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Version_A-19-206x410.png
[thumb_448_410-width] => 206
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Version_A-19-412x820.png
[thumb_448_410_retina-width] => 412
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Version_A-19-192x382.png
[thumb_574_382-width] => 192
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Version_A-19-199x395.png
[thumb_776_395-width] => 199
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[gform-image-choice-sm-width] => 151
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[gform-image-choice-md-width] => 201
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[gform-image-choice-lg-width] => 302
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 68207
[id] => 68207
[title] => Version_B.png
[filename] => Version_B-17.png
[filesize] => 99044
[url] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[link] => https://www.nextafter.com/experiments/how-adding-a-popularity-number-impacted-email-clicks-and-donations-on-a-nondonor-audience/attachment/version_b-png-100/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_b-png-100
[status] => inherit
[uploaded_to] => 68196
[date] => 2024-11-13 18:03:48
[modified] => 2024-11-13 18:03:48
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 532
[height] => 769
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_B-17-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Version_B-17-208x300.png
[medium-width] => 208
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[medium_large-width] => 532
[medium_large-height] => 769
[large] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[large-width] => 532
[large-height] => 769
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[1536x1536-width] => 532
[1536x1536-height] => 769
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[2048x2048-width] => 532
[2048x2048-height] => 769
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Version_B-17-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_B-17-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Version_B-17-35x50.png
[admin-width] => 35
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[full-width] => 532
[full-height] => 769
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Version_B-17-372x538.png
[thumb_372-width] => 372
[thumb_372-height] => 538
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_B-17-432x625.png
[single_post_thumbnail-width] => 432
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Version_B-17-22x32.png
[thumb_32_32-width] => 22
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Version_B-17-33x48.png
[thumb_48_48-width] => 33
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Version_B-17-44x64.png
[thumb_64_64-width] => 44
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Version_B-17-89x128.png
[thumb_128_128-width] => 89
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Version_B-17-46x67.png
[thumb_122_67-width] => 46
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Version_B-17-106x153.png
[thumb_153_153-width] => 106
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Version_B-17-113x163.png
[thumb_130_163-width] => 113
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Version_B-17-122x177.png
[thumb_232_177-width] => 122
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Version_B-17-450x650.png
[thumb_272_325-width] => 450
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Version_B-17-239x346.png
[thumb_271_346-width] => 239
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Version_B-17-187x271.png
[thumb_271_271-width] => 187
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Version_B-17-119x172.png
[thumb_272_172-width] => 119
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Version_B-17-319x461.png
[thumb_372_461-width] => 319
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Version_B-17-346x500.png
[thumb_372_500-width] => 346
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Version_B-17-187x270.png
[thumb_385_270-width] => 187
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Version_B-17-167x241.png
[thumb_416_241-width] => 167
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Version_B-17-284x410.png
[thumb_448_410-width] => 284
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[thumb_448_410_retina-width] => 532
[thumb_448_410_retina-height] => 769
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Version_B-17-264x382.png
[thumb_574_382-width] => 264
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Version_B-17-273x395.png
[thumb_776_395-width] => 273
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[gform-image-choice-sm-width] => 208
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[gform-image-choice-md-width] => 277
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[gform-image-choice-lg-width] => 415
[gform-image-choice-lg-height] => 600
)
)
)
T1: Version B
Array
(
[0] => Array
(
[ID] => 68206
[id] => 68206
[title] => Version_A.png
[filename] => Version_A-19.png
[filesize] => 123921
[url] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[link] => https://www.nextafter.com/experiments/how-adding-a-popularity-number-impacted-email-clicks-and-donations-on-a-nondonor-audience/attachment/version_a-png-110/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_a-png-110
[status] => inherit
[uploaded_to] => 68196
[date] => 2024-11-13 18:03:46
[modified] => 2024-11-13 18:03:46
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 469
[height] => 933
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_A-19-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Version_A-19-151x300.png
[medium-width] => 151
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[medium_large-width] => 469
[medium_large-height] => 933
[large] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[large-width] => 469
[large-height] => 933
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[1536x1536-width] => 469
[1536x1536-height] => 933
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[2048x2048-width] => 469
[2048x2048-height] => 933
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Version_A-19-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_A-19-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Version_A-19-25x50.png
[admin-width] => 25
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[full-width] => 469
[full-height] => 933
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Version_A-19-372x740.png
[thumb_372-width] => 372
[thumb_372-height] => 740
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_A-19-314x625.png
[single_post_thumbnail-width] => 314
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Version_A-19-16x32.png
[thumb_32_32-width] => 16
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Version_A-19-24x48.png
[thumb_48_48-width] => 24
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Version_A-19-32x64.png
[thumb_64_64-width] => 32
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Version_A-19-64x128.png
[thumb_128_128-width] => 64
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Version_A-19-34x67.png
[thumb_122_67-width] => 34
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Version_A-19-77x153.png
[thumb_153_153-width] => 77
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Version_A-19-82x163.png
[thumb_130_163-width] => 82
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Version_A-19-89x177.png
[thumb_232_177-width] => 89
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Version_A-19-327x650.png
[thumb_272_325-width] => 327
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Version_A-19-174x346.png
[thumb_271_346-width] => 174
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Version_A-19-136x271.png
[thumb_271_271-width] => 136
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Version_A-19-86x172.png
[thumb_272_172-width] => 86
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Version_A-19-232x461.png
[thumb_372_461-width] => 232
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Version_A-19-251x500.png
[thumb_372_500-width] => 251
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Version_A-19-136x270.png
[thumb_385_270-width] => 136
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Version_A-19-121x241.png
[thumb_416_241-width] => 121
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Version_A-19-206x410.png
[thumb_448_410-width] => 206
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Version_A-19-412x820.png
[thumb_448_410_retina-width] => 412
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Version_A-19-192x382.png
[thumb_574_382-width] => 192
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Version_A-19-199x395.png
[thumb_776_395-width] => 199
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[gform-image-choice-sm-width] => 151
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[gform-image-choice-md-width] => 201
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Version_A-19.png
[gform-image-choice-lg-width] => 302
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 68207
[id] => 68207
[title] => Version_B.png
[filename] => Version_B-17.png
[filesize] => 99044
[url] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[link] => https://www.nextafter.com/experiments/how-adding-a-popularity-number-impacted-email-clicks-and-donations-on-a-nondonor-audience/attachment/version_b-png-100/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => version_b-png-100
[status] => inherit
[uploaded_to] => 68196
[date] => 2024-11-13 18:03:48
[modified] => 2024-11-13 18:03:48
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 532
[height] => 769
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_B-17-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Version_B-17-208x300.png
[medium-width] => 208
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[medium_large-width] => 532
[medium_large-height] => 769
[large] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[large-width] => 532
[large-height] => 769
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[1536x1536-width] => 532
[1536x1536-height] => 769
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[2048x2048-width] => 532
[2048x2048-height] => 769
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Version_B-17-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_B-17-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Version_B-17-35x50.png
[admin-width] => 35
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[full-width] => 532
[full-height] => 769
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Version_B-17-372x538.png
[thumb_372-width] => 372
[thumb_372-height] => 538
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Version_B-17-432x625.png
[single_post_thumbnail-width] => 432
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Version_B-17-22x32.png
[thumb_32_32-width] => 22
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Version_B-17-33x48.png
[thumb_48_48-width] => 33
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Version_B-17-44x64.png
[thumb_64_64-width] => 44
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Version_B-17-89x128.png
[thumb_128_128-width] => 89
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Version_B-17-46x67.png
[thumb_122_67-width] => 46
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Version_B-17-106x153.png
[thumb_153_153-width] => 106
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Version_B-17-113x163.png
[thumb_130_163-width] => 113
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Version_B-17-122x177.png
[thumb_232_177-width] => 122
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Version_B-17-450x650.png
[thumb_272_325-width] => 450
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Version_B-17-239x346.png
[thumb_271_346-width] => 239
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Version_B-17-187x271.png
[thumb_271_271-width] => 187
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Version_B-17-119x172.png
[thumb_272_172-width] => 119
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Version_B-17-319x461.png
[thumb_372_461-width] => 319
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Version_B-17-346x500.png
[thumb_372_500-width] => 346
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Version_B-17-187x270.png
[thumb_385_270-width] => 187
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Version_B-17-167x241.png
[thumb_416_241-width] => 167
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Version_B-17-284x410.png
[thumb_448_410-width] => 284
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[thumb_448_410_retina-width] => 532
[thumb_448_410_retina-height] => 769
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Version_B-17-264x382.png
[thumb_574_382-width] => 264
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Version_B-17-273x395.png
[thumb_776_395-width] => 273
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[gform-image-choice-sm-width] => 208
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[gform-image-choice-md-width] => 277
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Version_B-17.png
[gform-image-choice-lg-width] => 415
[gform-image-choice-lg-height] => 600
)
)
)
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Version A | 1.0% | ||
| T1: | Version B | 1.1% | 11.2% | 98.6% |
This experiment has a required sample size of 63,667 in order to be valid. Since the experiment had a total sample size of 201,744, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
11.2% increase in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learnings from this experiment are: 1. Updating the email copy to be shorter and mentioning Hoover fellow Victor Davis Hanson at the beginning proved to be effective in driving engagement. This indicates that concise and personalized content resonates well with the non-donor audience. 2. The callout highlighting the popularity of the offer with "Already downloaded by more than 30,000 people in the last 6 months" likely created a sense of urgency and social proof, contributing to the increase in clicks and conversions. 3. The treatment showed a significant 11.2% increase in Clicks for All Traffic, a 19% increase in Opens, and a 117% decrease in Clicks to Opens, all with high levels of statistical confidence. However, with the donations Version B showed a 30% decrease in number of donations with a 79% level of confidence. With total revenue being: Version A: $1415 with 30 gifts Version B: $890 with 21 gifts To leverage these learnings in future experimentation, Hoover Institution should consider: 1. Continuing to focus on crafting concise and personalized content that resonates with the audience. 2. Utilizing social proof and urgency in callouts to drive engagement and conversions. By applying these learnings, Hoover Institution can optimize their email marketing strategy to better engage their non-donor audience and drive conversions effectively.