How deeper survey engagement affects donor conversion
Experiment ID: #7120
The Heritage Foundation
Experiment Summary
Ended On: 10/02/2018
The Heritage Foundation was running their Annual Member Survey, which asked members for their opinions on key policy initiatives and then gave them the opportunity to give. The initial treatment for the survey came from the direct mail piece, which was a very basic survey that just asked members to check boxes with the policy initiatives they thought were important. This might have worked in direct mail, but digitally, this seemed very underwhelming to the Heritage team. They crafted a second survey that was much longer—asking multiple, nuanced questions for each policy area and leaving room for open-ended responses. They knew that this was risky—common wisdom says that many people don’t know what to write in open-ended responses—but they thought that deeper engagement would result in higher conversion.
Research Question
How will a longer survey that requires more time and attention from each respondent affect donor conversion?
Design
C: Control
Array
(
[0] => Array
(
[ID] => 42524
[id] => 42524
[title] => Control.jpg
[filename] => Control-1912-scaled.jpg
[filesize] => 640047
[url] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[link] => https://www.nextafter.com/experiments/how-deeper-survey-engagement-affects-donor-conversion/attachment/control-jpg-2226/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-jpg-2226
[status] => inherit
[uploaded_to] => 15342
[date] => 2022-10-25 21:09:14
[modified] => 2022-10-25 21:09:14
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1377
[height] => 2560
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-1912-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-1912-161x300.jpg
[medium-width] => 161
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-1912-768x1428.jpg
[medium_large-width] => 640
[medium_large-height] => 1190
[large] => https://www.nextafter.com/wp-content/uploads/Control-1912-551x1024.jpg
[large-width] => 551
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-1912-826x1536.jpg
[1536x1536-width] => 826
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-1912-1102x2048.jpg
[2048x2048-width] => 1102
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[tp-image-grid-width] => 377
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-1912-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-1912-27x50.jpg
[admin-width] => 27
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[full-width] => 1377
[full-height] => 2560
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-1912-372x691.jpg
[thumb_372-width] => 372
[thumb_372-height] => 691
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-1912-336x625.jpg
[single_post_thumbnail-width] => 336
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-1912-17x32.jpg
[thumb_32_32-width] => 17
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-1912-26x48.jpg
[thumb_48_48-width] => 26
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-1912-34x64.jpg
[thumb_64_64-width] => 34
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-1912-69x128.jpg
[thumb_128_128-width] => 69
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-1912-36x67.jpg
[thumb_122_67-width] => 36
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-1912-82x153.jpg
[thumb_153_153-width] => 82
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-1912-88x163.jpg
[thumb_130_163-width] => 88
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-1912-95x177.jpg
[thumb_232_177-width] => 95
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-1912-350x650.jpg
[thumb_272_325-width] => 350
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-1912-186x346.jpg
[thumb_271_346-width] => 186
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-1912-146x271.jpg
[thumb_271_271-width] => 146
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-1912-93x172.jpg
[thumb_272_172-width] => 93
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-1912-248x461.jpg
[thumb_372_461-width] => 248
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-1912-269x500.jpg
[thumb_372_500-width] => 269
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-1912-145x270.jpg
[thumb_385_270-width] => 145
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-1912-130x241.jpg
[thumb_416_241-width] => 130
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-1912-221x410.jpg
[thumb_448_410-width] => 221
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-1912-441x820.jpg
[thumb_448_410_retina-width] => 441
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-1912-206x382.jpg
[thumb_574_382-width] => 206
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-1912-212x395.jpg
[thumb_776_395-width] => 212
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[gform-image-choice-sm-width] => 161
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[gform-image-choice-md-width] => 215
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[gform-image-choice-lg-width] => 323
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 42527
[id] => 42527
[title] => Treatment_#1.jpg
[filename] => Treatment_1-826-scaled.jpg
[filesize] => 386615
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[link] => https://www.nextafter.com/experiments/how-deeper-survey-engagement-affects-donor-conversion/attachment/treatment_1-jpg-966/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-jpg-966
[status] => inherit
[uploaded_to] => 15342
[date] => 2022-10-25 21:09:48
[modified] => 2022-10-25 21:09:48
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 859
[height] => 2560
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-101x300.jpg
[medium-width] => 101
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-768x2288.jpg
[medium_large-width] => 640
[medium_large-height] => 1907
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-344x1024.jpg
[large-width] => 344
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-516x1536.jpg
[1536x1536-width] => 516
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-688x2048.jpg
[2048x2048-width] => 688
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[tp-image-grid-width] => 235
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-17x50.jpg
[admin-width] => 17
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[full-width] => 859
[full-height] => 2560
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-372x1108.jpg
[thumb_372-width] => 372
[thumb_372-height] => 1108
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-210x625.jpg
[single_post_thumbnail-width] => 210
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-11x32.jpg
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-16x48.jpg
[thumb_48_48-width] => 16
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-21x64.jpg
[thumb_64_64-width] => 21
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-43x128.jpg
[thumb_128_128-width] => 43
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-22x67.jpg
[thumb_122_67-width] => 22
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-51x153.jpg
[thumb_153_153-width] => 51
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-55x163.jpg
[thumb_130_163-width] => 55
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-59x177.jpg
[thumb_232_177-width] => 59
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-218x650.jpg
[thumb_272_325-width] => 218
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-116x346.jpg
[thumb_271_346-width] => 116
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-91x271.jpg
[thumb_271_271-width] => 91
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-58x172.jpg
[thumb_272_172-width] => 58
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-155x461.jpg
[thumb_372_461-width] => 155
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-168x500.jpg
[thumb_372_500-width] => 168
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-91x270.jpg
[thumb_385_270-width] => 91
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-81x241.jpg
[thumb_416_241-width] => 81
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-138x410.jpg
[thumb_448_410-width] => 138
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-275x820.jpg
[thumb_448_410_retina-width] => 275
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-128x382.jpg
[thumb_574_382-width] => 128
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-133x395.jpg
[thumb_776_395-width] => 133
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[gform-image-choice-sm-width] => 101
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[gform-image-choice-md-width] => 134
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[gform-image-choice-lg-width] => 201
[gform-image-choice-lg-height] => 600
)
)
)
T1: Treatment #1
Array
(
[0] => Array
(
[ID] => 42524
[id] => 42524
[title] => Control.jpg
[filename] => Control-1912-scaled.jpg
[filesize] => 640047
[url] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[link] => https://www.nextafter.com/experiments/how-deeper-survey-engagement-affects-donor-conversion/attachment/control-jpg-2226/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-jpg-2226
[status] => inherit
[uploaded_to] => 15342
[date] => 2022-10-25 21:09:14
[modified] => 2022-10-25 21:09:14
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1377
[height] => 2560
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-1912-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-1912-161x300.jpg
[medium-width] => 161
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-1912-768x1428.jpg
[medium_large-width] => 640
[medium_large-height] => 1190
[large] => https://www.nextafter.com/wp-content/uploads/Control-1912-551x1024.jpg
[large-width] => 551
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-1912-826x1536.jpg
[1536x1536-width] => 826
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-1912-1102x2048.jpg
[2048x2048-width] => 1102
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[tp-image-grid-width] => 377
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-1912-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-1912-27x50.jpg
[admin-width] => 27
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[full-width] => 1377
[full-height] => 2560
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-1912-372x691.jpg
[thumb_372-width] => 372
[thumb_372-height] => 691
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-1912-336x625.jpg
[single_post_thumbnail-width] => 336
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-1912-17x32.jpg
[thumb_32_32-width] => 17
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-1912-26x48.jpg
[thumb_48_48-width] => 26
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-1912-34x64.jpg
[thumb_64_64-width] => 34
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-1912-69x128.jpg
[thumb_128_128-width] => 69
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-1912-36x67.jpg
[thumb_122_67-width] => 36
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-1912-82x153.jpg
[thumb_153_153-width] => 82
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-1912-88x163.jpg
[thumb_130_163-width] => 88
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-1912-95x177.jpg
[thumb_232_177-width] => 95
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-1912-350x650.jpg
[thumb_272_325-width] => 350
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-1912-186x346.jpg
[thumb_271_346-width] => 186
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-1912-146x271.jpg
[thumb_271_271-width] => 146
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-1912-93x172.jpg
[thumb_272_172-width] => 93
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-1912-248x461.jpg
[thumb_372_461-width] => 248
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-1912-269x500.jpg
[thumb_372_500-width] => 269
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-1912-145x270.jpg
[thumb_385_270-width] => 145
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-1912-130x241.jpg
[thumb_416_241-width] => 130
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-1912-221x410.jpg
[thumb_448_410-width] => 221
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-1912-441x820.jpg
[thumb_448_410_retina-width] => 441
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-1912-206x382.jpg
[thumb_574_382-width] => 206
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-1912-212x395.jpg
[thumb_776_395-width] => 212
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[gform-image-choice-sm-width] => 161
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[gform-image-choice-md-width] => 215
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-1912-scaled.jpg
[gform-image-choice-lg-width] => 323
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 42527
[id] => 42527
[title] => Treatment_#1.jpg
[filename] => Treatment_1-826-scaled.jpg
[filesize] => 386615
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[link] => https://www.nextafter.com/experiments/how-deeper-survey-engagement-affects-donor-conversion/attachment/treatment_1-jpg-966/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-jpg-966
[status] => inherit
[uploaded_to] => 15342
[date] => 2022-10-25 21:09:48
[modified] => 2022-10-25 21:09:48
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 859
[height] => 2560
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-150x150.jpg
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-101x300.jpg
[medium-width] => 101
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-768x2288.jpg
[medium_large-width] => 640
[medium_large-height] => 1907
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-344x1024.jpg
[large-width] => 344
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-516x1536.jpg
[1536x1536-width] => 516
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-688x2048.jpg
[2048x2048-width] => 688
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[tp-image-grid-width] => 235
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-150x150.jpg
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-17x50.jpg
[admin-width] => 17
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[full-width] => 859
[full-height] => 2560
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-372x1108.jpg
[thumb_372-width] => 372
[thumb_372-height] => 1108
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-210x625.jpg
[single_post_thumbnail-width] => 210
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-11x32.jpg
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-16x48.jpg
[thumb_48_48-width] => 16
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-21x64.jpg
[thumb_64_64-width] => 21
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-43x128.jpg
[thumb_128_128-width] => 43
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-22x67.jpg
[thumb_122_67-width] => 22
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-51x153.jpg
[thumb_153_153-width] => 51
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-55x163.jpg
[thumb_130_163-width] => 55
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-59x177.jpg
[thumb_232_177-width] => 59
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-218x650.jpg
[thumb_272_325-width] => 218
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-116x346.jpg
[thumb_271_346-width] => 116
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-91x271.jpg
[thumb_271_271-width] => 91
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-58x172.jpg
[thumb_272_172-width] => 58
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-155x461.jpg
[thumb_372_461-width] => 155
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-168x500.jpg
[thumb_372_500-width] => 168
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-91x270.jpg
[thumb_385_270-width] => 91
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-81x241.jpg
[thumb_416_241-width] => 81
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-138x410.jpg
[thumb_448_410-width] => 138
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-275x820.jpg
[thumb_448_410_retina-width] => 275
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-128x382.jpg
[thumb_574_382-width] => 128
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-133x395.jpg
[thumb_776_395-width] => 133
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[gform-image-choice-sm-width] => 101
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[gform-image-choice-md-width] => 134
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-826-scaled.jpg
[gform-image-choice-lg-width] => 201
[gform-image-choice-lg-height] => 600
)
)
)
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 1.3% | ||
| T1: | Treatment #1 | 2.4% | 81.9% | 99.4% |
This experiment has a required sample size of 1,287 in order to be valid. Since the experiment had a total sample size of 4,657, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
Ă— 81.9% increase in conversion rate
Ă— 0% increase in average gift
Key Learnings
Interestingly, the longer and more engaging survey produced a 9% increase in survey completions, which might have led to an incremental increase in donations just based on volume. But when the team looked at the analytics, the treatment survey (longer, more engaging) produced an 81.9% increase in donations. This indicates that the more a survey requires, the more invested the user might be. And deeper investment, in this case, led to increased response rates in terms of donor conversions. This brought up one significant question—what if Heritage tried this in the mail?