How simplifying a petition impacted petition signature rate
Experiment ID: #56299
Americans for Prosperity
Experiment Summary
Ended On: 04/01/2021
As a part of an open letter (petition) promotional campaign that Americans for Prosperity was running, they wanted to see if they could experiment with increasing email signup rates on the massive increases in click-through traffic received from their simplified ad approach.
As expected, far more people were clicking, but their email signup rate was dropping because the pre-click and post-click experiences weren’t directly congruent if the end user did not read the corresponding ad text to know that if they clicked, they would be asked to sign a petition.
Overall volume of emails was much higher with the simplified ad experience, but the organization wanted to see if they could capture a higher percentage of this overall increase in visitors to the open letter (petition).
Therefore, we attempted to simplify the open letter by removing some additional detail from the page, thereby shortening the letter altogether to see if we could remove friction in the email signup process.
Research Question
We believe that simplifying the petition for simplified ad clickers will achieve an increase in email signup rate.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 46693
[id] => 46693
[title] => Control.png
[filename] => Control-3470.png
[filesize] => 2584261
[url] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[link] => https://www.nextafter.com/experiments/how-simplifying-a-petition-impacted-petition-signature-rate/attachment/control-png-3994/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-3994
[status] => inherit
[uploaded_to] => 18357
[date] => 2022-11-02 18:35:43
[modified] => 2022-11-02 18:35:43
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1626
[height] => 5404
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-3470-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-3470-90x300.png
[medium-width] => 90
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-3470-768x2552.png
[medium_large-width] => 640
[medium_large-height] => 2127
[large] => https://www.nextafter.com/wp-content/uploads/Control-3470-308x1024.png
[large-width] => 308
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-3470-462x1536.png
[1536x1536-width] => 462
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-3470-616x2048.png
[2048x2048-width] => 616
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[tp-image-grid-width] => 211
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-3470-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-3470-15x50.png
[admin-width] => 15
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[full-width] => 1626
[full-height] => 5404
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-3470-372x1236.png
[thumb_372-width] => 372
[thumb_372-height] => 1236
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-3470-188x625.png
[single_post_thumbnail-width] => 188
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-3470-10x32.png
[thumb_32_32-width] => 10
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-3470-14x48.png
[thumb_48_48-width] => 14
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-3470-19x64.png
[thumb_64_64-width] => 19
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-3470-39x128.png
[thumb_128_128-width] => 39
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-3470-20x67.png
[thumb_122_67-width] => 20
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-3470-46x153.png
[thumb_153_153-width] => 46
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-3470-49x163.png
[thumb_130_163-width] => 49
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-3470-53x177.png
[thumb_232_177-width] => 53
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-3470-196x650.png
[thumb_272_325-width] => 196
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-3470-104x346.png
[thumb_271_346-width] => 104
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-3470-82x271.png
[thumb_271_271-width] => 82
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-3470-52x172.png
[thumb_272_172-width] => 52
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-3470-139x461.png
[thumb_372_461-width] => 139
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-3470-150x500.png
[thumb_372_500-width] => 150
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-3470-81x270.png
[thumb_385_270-width] => 81
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-3470-73x241.png
[thumb_416_241-width] => 73
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-3470-123x410.png
[thumb_448_410-width] => 123
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-3470-247x820.png
[thumb_448_410_retina-width] => 247
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-3470-115x382.png
[thumb_574_382-width] => 115
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-3470-119x395.png
[thumb_776_395-width] => 119
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[gform-image-choice-sm-width] => 90
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[gform-image-choice-md-width] => 120
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[gform-image-choice-lg-width] => 181
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 32553
[id] => 32553
[title] => Treatment_#1.png
[filename] => Treatment_1-631.png
[filesize] => 2308931
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[link] => https://www.nextafter.com/experiments/how-simplifying-a-petition-impacted-petition-signature-rate/attachment/treatment_1-png-708/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-708
[status] => inherit
[uploaded_to] => 18357
[date] => 2022-10-07 21:20:12
[modified] => 2022-10-07 21:20:12
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1496
[height] => 4976
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-90x300.png
[medium-width] => 90
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-768x2555.png
[medium_large-width] => 640
[medium_large-height] => 2129
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-308x1024.png
[large-width] => 308
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-462x1536.png
[1536x1536-width] => 462
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-616x2048.png
[2048x2048-width] => 616
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[tp-image-grid-width] => 210
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-15x50.png
[admin-width] => 15
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[full-width] => 1496
[full-height] => 4976
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-372x1237.png
[thumb_372-width] => 372
[thumb_372-height] => 1237
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-188x625.png
[single_post_thumbnail-width] => 188
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-10x32.png
[thumb_32_32-width] => 10
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-14x48.png
[thumb_48_48-width] => 14
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-19x64.png
[thumb_64_64-width] => 19
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-38x128.png
[thumb_128_128-width] => 38
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-20x67.png
[thumb_122_67-width] => 20
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-46x153.png
[thumb_153_153-width] => 46
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-49x163.png
[thumb_130_163-width] => 49
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-53x177.png
[thumb_232_177-width] => 53
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-195x650.png
[thumb_272_325-width] => 195
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-104x346.png
[thumb_271_346-width] => 104
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-81x271.png
[thumb_271_271-width] => 81
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-52x172.png
[thumb_272_172-width] => 52
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-139x461.png
[thumb_372_461-width] => 139
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-150x500.png
[thumb_372_500-width] => 150
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-81x270.png
[thumb_385_270-width] => 81
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-72x241.png
[thumb_416_241-width] => 72
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-123x410.png
[thumb_448_410-width] => 123
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-247x820.png
[thumb_448_410_retina-width] => 247
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-115x382.png
[thumb_574_382-width] => 115
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-119x395.png
[thumb_776_395-width] => 119
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[gform-image-choice-sm-width] => 90
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[gform-image-choice-md-width] => 120
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[gform-image-choice-lg-width] => 180
[gform-image-choice-lg-height] => 600
)
)
)
T1: Treatment #1
Array
(
[0] => Array
(
[ID] => 46693
[id] => 46693
[title] => Control.png
[filename] => Control-3470.png
[filesize] => 2584261
[url] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[link] => https://www.nextafter.com/experiments/how-simplifying-a-petition-impacted-petition-signature-rate/attachment/control-png-3994/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-3994
[status] => inherit
[uploaded_to] => 18357
[date] => 2022-11-02 18:35:43
[modified] => 2022-11-02 18:35:43
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1626
[height] => 5404
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-3470-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-3470-90x300.png
[medium-width] => 90
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-3470-768x2552.png
[medium_large-width] => 640
[medium_large-height] => 2127
[large] => https://www.nextafter.com/wp-content/uploads/Control-3470-308x1024.png
[large-width] => 308
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-3470-462x1536.png
[1536x1536-width] => 462
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-3470-616x2048.png
[2048x2048-width] => 616
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[tp-image-grid-width] => 211
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-3470-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-3470-15x50.png
[admin-width] => 15
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[full-width] => 1626
[full-height] => 5404
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-3470-372x1236.png
[thumb_372-width] => 372
[thumb_372-height] => 1236
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-3470-188x625.png
[single_post_thumbnail-width] => 188
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-3470-10x32.png
[thumb_32_32-width] => 10
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-3470-14x48.png
[thumb_48_48-width] => 14
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-3470-19x64.png
[thumb_64_64-width] => 19
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-3470-39x128.png
[thumb_128_128-width] => 39
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-3470-20x67.png
[thumb_122_67-width] => 20
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-3470-46x153.png
[thumb_153_153-width] => 46
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-3470-49x163.png
[thumb_130_163-width] => 49
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-3470-53x177.png
[thumb_232_177-width] => 53
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-3470-196x650.png
[thumb_272_325-width] => 196
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-3470-104x346.png
[thumb_271_346-width] => 104
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-3470-82x271.png
[thumb_271_271-width] => 82
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-3470-52x172.png
[thumb_272_172-width] => 52
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-3470-139x461.png
[thumb_372_461-width] => 139
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-3470-150x500.png
[thumb_372_500-width] => 150
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-3470-81x270.png
[thumb_385_270-width] => 81
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-3470-73x241.png
[thumb_416_241-width] => 73
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-3470-123x410.png
[thumb_448_410-width] => 123
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-3470-247x820.png
[thumb_448_410_retina-width] => 247
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-3470-115x382.png
[thumb_574_382-width] => 115
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-3470-119x395.png
[thumb_776_395-width] => 119
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[gform-image-choice-sm-width] => 90
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[gform-image-choice-md-width] => 120
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-3470.png
[gform-image-choice-lg-width] => 181
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 32553
[id] => 32553
[title] => Treatment_#1.png
[filename] => Treatment_1-631.png
[filesize] => 2308931
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[link] => https://www.nextafter.com/experiments/how-simplifying-a-petition-impacted-petition-signature-rate/attachment/treatment_1-png-708/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-708
[status] => inherit
[uploaded_to] => 18357
[date] => 2022-10-07 21:20:12
[modified] => 2022-10-07 21:20:12
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 1496
[height] => 4976
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-90x300.png
[medium-width] => 90
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-768x2555.png
[medium_large-width] => 640
[medium_large-height] => 2129
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-308x1024.png
[large-width] => 308
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-462x1536.png
[1536x1536-width] => 462
[1536x1536-height] => 1536
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-616x2048.png
[2048x2048-width] => 616
[2048x2048-height] => 2048
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[tp-image-grid-width] => 210
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-15x50.png
[admin-width] => 15
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[full-width] => 1496
[full-height] => 4976
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-372x1237.png
[thumb_372-width] => 372
[thumb_372-height] => 1237
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-188x625.png
[single_post_thumbnail-width] => 188
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-10x32.png
[thumb_32_32-width] => 10
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-14x48.png
[thumb_48_48-width] => 14
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-19x64.png
[thumb_64_64-width] => 19
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-38x128.png
[thumb_128_128-width] => 38
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-20x67.png
[thumb_122_67-width] => 20
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-46x153.png
[thumb_153_153-width] => 46
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-49x163.png
[thumb_130_163-width] => 49
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-53x177.png
[thumb_232_177-width] => 53
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-195x650.png
[thumb_272_325-width] => 195
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-104x346.png
[thumb_271_346-width] => 104
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-81x271.png
[thumb_271_271-width] => 81
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-52x172.png
[thumb_272_172-width] => 52
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-139x461.png
[thumb_372_461-width] => 139
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-150x500.png
[thumb_372_500-width] => 150
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-81x270.png
[thumb_385_270-width] => 81
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-72x241.png
[thumb_416_241-width] => 72
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-123x410.png
[thumb_448_410-width] => 123
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-247x820.png
[thumb_448_410_retina-width] => 247
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-115x382.png
[thumb_574_382-width] => 115
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631-119x395.png
[thumb_776_395-width] => 119
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[gform-image-choice-sm-width] => 90
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[gform-image-choice-md-width] => 120
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-631.png
[gform-image-choice-lg-width] => 180
[gform-image-choice-lg-height] => 600
)
)
)
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 42.9% | ||
| T1: | Treatment #1 | 37.8% | -11.9% | 100.0% |
This experiment has a required sample size of 715 in order to be valid. Since the experiment had a total sample size of 11,787, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 11.9% decrease in conversion rate
× 0% increase in average gift
Key Learnings
With a 99.9% level of confidence, we observed a decrease in email signup rate of -11.9%.
Our theory is that the additional context and detail that was removed from the petition actually provided less on-page context for the higher number of visitors received from the simplified ad experience — which served as a foundation to motivate people who clicked without understanding what they were being asked to do to finish the petition signing process.
Further experimentation with creating alignment between the pre-click and post-click experience of the "simplified ad experience" is advised to focus on increasing email signup rates.