How including a button in an appeal email affects donor conversion rates for members
Experiment ID: #198831
The Heritage Foundation
Experiment Summary
Ended On: 07/31/2025
The Heritage Foundation conducted this experiment as part of their President’s Challenge matching gift campaign. This campaign was designed to drive donor engagement by offering a limited-time opportunity to double the impact of a donor’s gift through a matching challenge. The email in question was the first send in the campaign sequence, making it a high-leverage moment for influencing donor behavior and maximizing initial response rates.
Given the high stakes of the campaign’s early momentum and the importance of clearly communicating the match opportunity, the Heritage Foundation sought to test whether different call-to-action (CTA) formats would affect click-through and conversion rates.
The test was deployed via email, targeting the Heritage Foundation’s member list.
Both versions of the email—control and treatment—were linked to the same donation landing page but differed in their CTA presentation.
The experiment was driven by a hypothesis rooted in behavioral and conversion rate optimization principles: a visually prominent, action-oriented button might outperform a plain-text hyperlink in driving user action.
Key motivations included:
- Improving clarity and visibility of the CTA: While the raw URL in the control version is technically functional, it lacks the immediate visual weight and action language of a button. The team suspected that this might cause the CTA to be overlooked or feel less urgent.
- Reducing cognitive friction: A clear, concise button like “Double My Gift” directly mirrors the donor’s intent and may simplify decision-making.
- Encouraging mobile-friendly design: Buttons often perform better on mobile devices where touch interaction is key, and plain-text links can be harder to tap.
This test is a good example of using micro-optimizations in email fundraising design to explore how seemingly small changes can have an outsized effect on user behavior and conversion rates.
Research Question
We believe that using a visually prominent, action-oriented button that says “Double My Gift” will increase donor conversion rate because it draws more attention than a plain-text hyperlink, creates a clearer and more compelling call to action, and reduces friction by making the next step more obvious and easier to complete.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 720121
[id] => 720121
[title] => Control.png
[filename] => Control-168.png
[filesize] => 106615
[url] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[link] => https://www.nextafter.com/experiments/how-including-a-button-in-an-appeal-email-affects-donor-conversion-rates-for-members/attachment/control-png-10909/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-10909
[status] => inherit
[uploaded_to] => 720118
[date] => 2025-08-01 15:34:36
[modified] => 2025-08-01 15:34:36
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1019
[height] => 585
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-168-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-168-300x172.png
[medium-width] => 300
[medium-height] => 172
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-168-768x441.png
[medium_large-width] => 640
[medium_large-height] => 368
[large] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[large-width] => 640
[large-height] => 367
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[1536x1536-width] => 1019
[1536x1536-height] => 585
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[2048x2048-width] => 1019
[2048x2048-height] => 585
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-168-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-168-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-168-50x29.png
[admin-width] => 50
[admin-height] => 29
[full] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[full-width] => 1019
[full-height] => 585
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-168-372x214.png
[thumb_372-width] => 372
[thumb_372-height] => 214
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-168-980x563.png
[single_post_thumbnail-width] => 980
[single_post_thumbnail-height] => 563
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-168-32x18.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 18
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-168-48x28.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 28
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-168-64x37.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 37
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-168-128x73.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 73
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-168-117x67.png
[thumb_122_67-width] => 117
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-168-153x88.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 88
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-168-130x75.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 75
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-168-232x133.png
[thumb_232_177-width] => 232
[thumb_232_177-height] => 133
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-168-544x312.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 312
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-168-271x156.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 156
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-168-271x156.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 156
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-168-272x156.png
[thumb_272_172-width] => 272
[thumb_272_172-height] => 156
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-168-372x214.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 214
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-168-372x214.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 214
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-168-385x221.png
[thumb_385_270-width] => 385
[thumb_385_270-height] => 221
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-168-416x239.png
[thumb_416_241-width] => 416
[thumb_416_241-height] => 239
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-168-448x257.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 257
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-168-896x514.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 514
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-168-574x330.png
[thumb_574_382-width] => 574
[thumb_574_382-height] => 330
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-168-688x395.png
[thumb_776_395-width] => 688
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[gform-image-choice-sm-width] => 300
[gform-image-choice-sm-height] => 172
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[gform-image-choice-md-width] => 400
[gform-image-choice-md-height] => 230
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[gform-image-choice-lg-width] => 600
[gform-image-choice-lg-height] => 344
)
)
[1] => Array
(
[ID] => 720122
[id] => 720122
[title] => Treatment_#1.png
[filename] => Treatment_1-125.png
[filesize] => 100102
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[link] => https://www.nextafter.com/experiments/how-including-a-button-in-an-appeal-email-affects-donor-conversion-rates-for-members/attachment/treatment_1-png-4070/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-4070
[status] => inherit
[uploaded_to] => 720118
[date] => 2025-08-01 15:34:44
[modified] => 2025-08-01 15:34:44
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 759
[height] => 644
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-300x255.png
[medium-width] => 300
[medium-height] => 255
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[medium_large-width] => 640
[medium_large-height] => 543
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[large-width] => 640
[large-height] => 543
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[1536x1536-width] => 759
[1536x1536-height] => 644
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[2048x2048-width] => 759
[2048x2048-height] => 644
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-50x42.png
[admin-width] => 50
[admin-height] => 42
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[full-width] => 759
[full-height] => 644
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-372x316.png
[thumb_372-width] => 372
[thumb_372-height] => 316
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-737x625.png
[single_post_thumbnail-width] => 737
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-32x27.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 27
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-48x41.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 41
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-64x54.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 54
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-128x109.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 109
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-79x67.png
[thumb_122_67-width] => 79
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-153x130.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 130
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-130x110.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 110
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-209x177.png
[thumb_232_177-width] => 209
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-544x462.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 462
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-271x230.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 230
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-271x230.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 230
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-203x172.png
[thumb_272_172-width] => 203
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-372x316.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 316
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-372x316.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 316
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-318x270.png
[thumb_385_270-width] => 318
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-284x241.png
[thumb_416_241-width] => 284
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-448x380.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 380
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[thumb_448_410_retina-width] => 759
[thumb_448_410_retina-height] => 644
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-450x382.png
[thumb_574_382-width] => 450
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-466x395.png
[thumb_776_395-width] => 466
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[gform-image-choice-sm-width] => 300
[gform-image-choice-sm-height] => 255
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[gform-image-choice-md-width] => 400
[gform-image-choice-md-height] => 339
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[gform-image-choice-lg-width] => 600
[gform-image-choice-lg-height] => 509
)
)
)
T1: Treatment #1
Array
(
[0] => Array
(
[ID] => 720121
[id] => 720121
[title] => Control.png
[filename] => Control-168.png
[filesize] => 106615
[url] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[link] => https://www.nextafter.com/experiments/how-including-a-button-in-an-appeal-email-affects-donor-conversion-rates-for-members/attachment/control-png-10909/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-10909
[status] => inherit
[uploaded_to] => 720118
[date] => 2025-08-01 15:34:36
[modified] => 2025-08-01 15:34:36
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1019
[height] => 585
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-168-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-168-300x172.png
[medium-width] => 300
[medium-height] => 172
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-168-768x441.png
[medium_large-width] => 640
[medium_large-height] => 368
[large] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[large-width] => 640
[large-height] => 367
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[1536x1536-width] => 1019
[1536x1536-height] => 585
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[2048x2048-width] => 1019
[2048x2048-height] => 585
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-168-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-168-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-168-50x29.png
[admin-width] => 50
[admin-height] => 29
[full] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[full-width] => 1019
[full-height] => 585
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-168-372x214.png
[thumb_372-width] => 372
[thumb_372-height] => 214
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-168-980x563.png
[single_post_thumbnail-width] => 980
[single_post_thumbnail-height] => 563
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-168-32x18.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 18
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-168-48x28.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 28
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-168-64x37.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 37
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-168-128x73.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 73
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-168-117x67.png
[thumb_122_67-width] => 117
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-168-153x88.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 88
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-168-130x75.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 75
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-168-232x133.png
[thumb_232_177-width] => 232
[thumb_232_177-height] => 133
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-168-544x312.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 312
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-168-271x156.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 156
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-168-271x156.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 156
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-168-272x156.png
[thumb_272_172-width] => 272
[thumb_272_172-height] => 156
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-168-372x214.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 214
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-168-372x214.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 214
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-168-385x221.png
[thumb_385_270-width] => 385
[thumb_385_270-height] => 221
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-168-416x239.png
[thumb_416_241-width] => 416
[thumb_416_241-height] => 239
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-168-448x257.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 257
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-168-896x514.png
[thumb_448_410_retina-width] => 896
[thumb_448_410_retina-height] => 514
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-168-574x330.png
[thumb_574_382-width] => 574
[thumb_574_382-height] => 330
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-168-688x395.png
[thumb_776_395-width] => 688
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[gform-image-choice-sm-width] => 300
[gform-image-choice-sm-height] => 172
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[gform-image-choice-md-width] => 400
[gform-image-choice-md-height] => 230
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-168.png
[gform-image-choice-lg-width] => 600
[gform-image-choice-lg-height] => 344
)
)
[1] => Array
(
[ID] => 720122
[id] => 720122
[title] => Treatment_#1.png
[filename] => Treatment_1-125.png
[filesize] => 100102
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[link] => https://www.nextafter.com/experiments/how-including-a-button-in-an-appeal-email-affects-donor-conversion-rates-for-members/attachment/treatment_1-png-4070/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-4070
[status] => inherit
[uploaded_to] => 720118
[date] => 2025-08-01 15:34:44
[modified] => 2025-08-01 15:34:44
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 759
[height] => 644
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-300x255.png
[medium-width] => 300
[medium-height] => 255
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[medium_large-width] => 640
[medium_large-height] => 543
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[large-width] => 640
[large-height] => 543
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[1536x1536-width] => 759
[1536x1536-height] => 644
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[2048x2048-width] => 759
[2048x2048-height] => 644
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-50x42.png
[admin-width] => 50
[admin-height] => 42
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[full-width] => 759
[full-height] => 644
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-372x316.png
[thumb_372-width] => 372
[thumb_372-height] => 316
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-737x625.png
[single_post_thumbnail-width] => 737
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-32x27.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 27
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-48x41.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 41
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-64x54.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 54
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-128x109.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 109
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-79x67.png
[thumb_122_67-width] => 79
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-153x130.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 130
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-130x110.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 110
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-209x177.png
[thumb_232_177-width] => 209
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-544x462.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 462
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-271x230.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 230
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-271x230.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 230
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-203x172.png
[thumb_272_172-width] => 203
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-372x316.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 316
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-372x316.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 316
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-318x270.png
[thumb_385_270-width] => 318
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-284x241.png
[thumb_416_241-width] => 284
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-448x380.png
[thumb_448_410-width] => 448
[thumb_448_410-height] => 380
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[thumb_448_410_retina-width] => 759
[thumb_448_410_retina-height] => 644
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-450x382.png
[thumb_574_382-width] => 450
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125-466x395.png
[thumb_776_395-width] => 466
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[gform-image-choice-sm-width] => 300
[gform-image-choice-sm-height] => 255
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[gform-image-choice-md-width] => 400
[gform-image-choice-md-height] => 339
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-125.png
[gform-image-choice-lg-width] => 600
[gform-image-choice-lg-height] => 509
)
)
)
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 19.5% | ||
| T1: | Treatment #1 | 10.2% | -47.7% | 99.7% |
This experiment has a required sample size of 109 in order to be valid. Since the experiment had a total sample size of 535, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
Ă— 47.7% decrease in conversion rate
Ă— 0% increase in average gift
Key Learnings
The key learnings from this experiment are that the visually prominent, action-oriented button with the text "Double My Gift" did not perform as expected. In fact, it resulted in a 47.7% decrease in donations compared to the plain-text hyperlink used in the control version of the email. This outcome was statistically significant with a confidence level of 99.7%. In future experimentation, it is important to remember that assumptions based on design principles and best practices may not always hold true for every audience or context. It is crucial to test different elements and approaches to see what resonates best with donors. Additionally, it is essential to prioritize evidence-based decision-making over assumptions, even if they seem logical on the surface. This result highlights the importance of testing variations and being open to unexpected outcomes to truly understand donor behavior and optimize conversion rates effectively.