let us help
Subscribe
The Heritage Foundation

How previewing a pledge affects click-through rates on a Facebook ad

Experiment ID: #110981

The Heritage Foundation

Experiment Summary

Ended On: 10/18/2022

After turning off some underperforming Facebook ads, we wanted to introduce a new treatment to test against our previous champion. For this treatment, we chose a concept that has shown success in the past: previewing a choice from the offered asset in order to entice the viewer to engage and click through to the asset itself. For this experiment, we changed only the ad creative and ran this test for 30 days.

Research Question

We believe that previewing a pledge on a Facebook ad will increase click-through rates by engaging the viewer and prompting them to click/tap to make their selection.

Design

C: Control
Array ( [0] => Array ( [ID] => 51049 [id] => 51049 [title] => Control.png [filename] => Control-4292.png [filesize] => 159846 [url] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [link] => https://www.nextafter.com/experiments/how-previewing-a-pledge-affects-click-through-rates-on-a-facebook-ad/attachment/control-png-4994/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-4994 [status] => inherit [uploaded_to] => 39008 [date] => 2023-03-13 14:48:47 [modified] => 2023-03-13 14:48:47 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 322 [height] => 604 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-4292-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-4292-160x300.png [medium-width] => 160 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [medium_large-width] => 322 [medium_large-height] => 604 [large] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [large-width] => 322 [large-height] => 604 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [1536x1536-width] => 322 [1536x1536-height] => 604 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [2048x2048-width] => 322 [2048x2048-height] => 604 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [tp-image-grid-width] => 322 [tp-image-grid-height] => 604 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-4292-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-4292-27x50.png [admin-width] => 27 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [full-width] => 322 [full-height] => 604 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [thumb_372-width] => 322 [thumb_372-height] => 604 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [single_post_thumbnail-width] => 322 [single_post_thumbnail-height] => 604 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-4292-17x32.png [thumb_32_32-width] => 17 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-4292-26x48.png [thumb_48_48-width] => 26 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-4292-34x64.png [thumb_64_64-width] => 34 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-4292-68x128.png [thumb_128_128-width] => 68 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-4292-36x67.png [thumb_122_67-width] => 36 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-4292-82x153.png [thumb_153_153-width] => 82 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-4292-87x163.png [thumb_130_163-width] => 87 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-4292-94x177.png [thumb_232_177-width] => 94 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [thumb_272_325-width] => 322 [thumb_272_325-height] => 604 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-4292-184x346.png [thumb_271_346-width] => 184 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-4292-144x271.png [thumb_271_271-width] => 144 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-4292-92x172.png [thumb_272_172-width] => 92 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-4292-246x461.png [thumb_372_461-width] => 246 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-4292-267x500.png [thumb_372_500-width] => 267 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-4292-144x270.png [thumb_385_270-width] => 144 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-4292-128x241.png [thumb_416_241-width] => 128 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-4292-219x410.png [thumb_448_410-width] => 219 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [thumb_448_410_retina-width] => 322 [thumb_448_410_retina-height] => 604 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-4292-204x382.png [thumb_574_382-width] => 204 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-4292-211x395.png [thumb_776_395-width] => 211 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [gform-image-choice-sm-width] => 160 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [gform-image-choice-md-width] => 213 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [gform-image-choice-lg-width] => 320 [gform-image-choice-lg-height] => 600 ) ) [1] => Array ( [ID] => 51050 [id] => 51050 [title] => Treatment_#1.png [filename] => Treatment_1-1248.png [filesize] => 208326 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [link] => https://www.nextafter.com/experiments/how-previewing-a-pledge-affects-click-through-rates-on-a-facebook-ad/attachment/treatment_1-png-1485/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-1485 [status] => inherit [uploaded_to] => 39008 [date] => 2023-03-13 14:48:57 [modified] => 2023-03-13 14:48:57 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 325 [height] => 601 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-162x300.png [medium-width] => 162 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [medium_large-width] => 325 [medium_large-height] => 601 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [large-width] => 325 [large-height] => 601 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [1536x1536-width] => 325 [1536x1536-height] => 601 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [2048x2048-width] => 325 [2048x2048-height] => 601 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [tp-image-grid-width] => 325 [tp-image-grid-height] => 601 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-27x50.png [admin-width] => 27 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [full-width] => 325 [full-height] => 601 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [thumb_372-width] => 325 [thumb_372-height] => 601 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [single_post_thumbnail-width] => 325 [single_post_thumbnail-height] => 601 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-17x32.png [thumb_32_32-width] => 17 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-26x48.png [thumb_48_48-width] => 26 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-35x64.png [thumb_64_64-width] => 35 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-69x128.png [thumb_128_128-width] => 69 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-36x67.png [thumb_122_67-width] => 36 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-83x153.png [thumb_153_153-width] => 83 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-88x163.png [thumb_130_163-width] => 88 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-96x177.png [thumb_232_177-width] => 96 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [thumb_272_325-width] => 325 [thumb_272_325-height] => 601 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-187x346.png [thumb_271_346-width] => 187 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-147x271.png [thumb_271_271-width] => 147 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-93x172.png [thumb_272_172-width] => 93 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-249x461.png [thumb_372_461-width] => 249 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-270x500.png [thumb_372_500-width] => 270 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-146x270.png [thumb_385_270-width] => 146 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-130x241.png [thumb_416_241-width] => 130 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-222x410.png [thumb_448_410-width] => 222 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [thumb_448_410_retina-width] => 325 [thumb_448_410_retina-height] => 601 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-207x382.png [thumb_574_382-width] => 207 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-214x395.png [thumb_776_395-width] => 214 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [gform-image-choice-sm-width] => 162 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [gform-image-choice-md-width] => 216 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [gform-image-choice-lg-width] => 324 [gform-image-choice-lg-height] => 600 ) ) )
T1: Treatment #1
Array ( [0] => Array ( [ID] => 51049 [id] => 51049 [title] => Control.png [filename] => Control-4292.png [filesize] => 159846 [url] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [link] => https://www.nextafter.com/experiments/how-previewing-a-pledge-affects-click-through-rates-on-a-facebook-ad/attachment/control-png-4994/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-4994 [status] => inherit [uploaded_to] => 39008 [date] => 2023-03-13 14:48:47 [modified] => 2023-03-13 14:48:47 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 322 [height] => 604 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-4292-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-4292-160x300.png [medium-width] => 160 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [medium_large-width] => 322 [medium_large-height] => 604 [large] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [large-width] => 322 [large-height] => 604 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [1536x1536-width] => 322 [1536x1536-height] => 604 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [2048x2048-width] => 322 [2048x2048-height] => 604 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [tp-image-grid-width] => 322 [tp-image-grid-height] => 604 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-4292-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-4292-27x50.png [admin-width] => 27 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [full-width] => 322 [full-height] => 604 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [thumb_372-width] => 322 [thumb_372-height] => 604 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [single_post_thumbnail-width] => 322 [single_post_thumbnail-height] => 604 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-4292-17x32.png [thumb_32_32-width] => 17 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-4292-26x48.png [thumb_48_48-width] => 26 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-4292-34x64.png [thumb_64_64-width] => 34 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-4292-68x128.png [thumb_128_128-width] => 68 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-4292-36x67.png [thumb_122_67-width] => 36 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-4292-82x153.png [thumb_153_153-width] => 82 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-4292-87x163.png [thumb_130_163-width] => 87 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-4292-94x177.png [thumb_232_177-width] => 94 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [thumb_272_325-width] => 322 [thumb_272_325-height] => 604 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-4292-184x346.png [thumb_271_346-width] => 184 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-4292-144x271.png [thumb_271_271-width] => 144 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-4292-92x172.png [thumb_272_172-width] => 92 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-4292-246x461.png [thumb_372_461-width] => 246 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-4292-267x500.png [thumb_372_500-width] => 267 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-4292-144x270.png [thumb_385_270-width] => 144 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-4292-128x241.png [thumb_416_241-width] => 128 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-4292-219x410.png [thumb_448_410-width] => 219 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [thumb_448_410_retina-width] => 322 [thumb_448_410_retina-height] => 604 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-4292-204x382.png [thumb_574_382-width] => 204 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-4292-211x395.png [thumb_776_395-width] => 211 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [gform-image-choice-sm-width] => 160 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [gform-image-choice-md-width] => 213 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-4292.png [gform-image-choice-lg-width] => 320 [gform-image-choice-lg-height] => 600 ) ) [1] => Array ( [ID] => 51050 [id] => 51050 [title] => Treatment_#1.png [filename] => Treatment_1-1248.png [filesize] => 208326 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [link] => https://www.nextafter.com/experiments/how-previewing-a-pledge-affects-click-through-rates-on-a-facebook-ad/attachment/treatment_1-png-1485/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-1485 [status] => inherit [uploaded_to] => 39008 [date] => 2023-03-13 14:48:57 [modified] => 2023-03-13 14:48:57 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 325 [height] => 601 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-162x300.png [medium-width] => 162 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [medium_large-width] => 325 [medium_large-height] => 601 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [large-width] => 325 [large-height] => 601 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [1536x1536-width] => 325 [1536x1536-height] => 601 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [2048x2048-width] => 325 [2048x2048-height] => 601 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [tp-image-grid-width] => 325 [tp-image-grid-height] => 601 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-27x50.png [admin-width] => 27 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [full-width] => 325 [full-height] => 601 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [thumb_372-width] => 325 [thumb_372-height] => 601 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [single_post_thumbnail-width] => 325 [single_post_thumbnail-height] => 601 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-17x32.png [thumb_32_32-width] => 17 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-26x48.png [thumb_48_48-width] => 26 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-35x64.png [thumb_64_64-width] => 35 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-69x128.png [thumb_128_128-width] => 69 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-36x67.png [thumb_122_67-width] => 36 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-83x153.png [thumb_153_153-width] => 83 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-88x163.png [thumb_130_163-width] => 88 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-96x177.png [thumb_232_177-width] => 96 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [thumb_272_325-width] => 325 [thumb_272_325-height] => 601 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-187x346.png [thumb_271_346-width] => 187 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-147x271.png [thumb_271_271-width] => 147 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-93x172.png [thumb_272_172-width] => 93 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-249x461.png [thumb_372_461-width] => 249 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-270x500.png [thumb_372_500-width] => 270 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-146x270.png [thumb_385_270-width] => 146 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-130x241.png [thumb_416_241-width] => 130 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-222x410.png [thumb_448_410-width] => 222 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [thumb_448_410_retina-width] => 325 [thumb_448_410_retina-height] => 601 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-207x382.png [thumb_574_382-width] => 207 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248-214x395.png [thumb_776_395-width] => 214 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [gform-image-choice-sm-width] => 162 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [gform-image-choice-md-width] => 216 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-1248.png [gform-image-choice-lg-width] => 324 [gform-image-choice-lg-height] => 600 ) ) )

Results

  Treatment Name Click Rate Relative Difference Confidence
C: Control 1.7%
T1: Treatment #1 1.3% -22.7% 100.0%

This experiment has a required sample size of 7,536 in order to be valid. Since the experiment had a total sample size of 111,998, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    22.7% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

After running this experiment for 30 days, we observed a 22.7% drop in click-through-rates with a 99.9% level of confidence. When analyzing these results, we identified a couple of factors that may have contributed to this decline.

First, while such a treatment has worked well for survey offers in the past, we feel that this approach may not be as effective for a pledge because we are not "asking" them to complete any action, i.e. when previewing a survey question, the preview "asks" the viewer a question ... but in the context of a pledge, the viewer simply sees a statement with a checked box beside it. This might make it unclear to the viewer that they are being asked to engage, create friction and fail to do enough to get them to stop scrolling.

Similarly, because the treatment presents a statement with a box already checked, it may simply cue viewers to agree and move on. In the control, action is encouraged by agitating urgency around the issue, likely doing more to encourage the viewer to stop, consider the situation, and want to learn more/take some action.

Lastly, we also noticed that in an attempt to strengthen the call to action, we added more copy and reduced the font size—"sign the pledge" appears much larger and eye-catching on the control than it does on the treatment, which could, again, making it clear to the viewer that there is an action to take.

Experiment Documented by NextAfter

Question about experiment #110981

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.