Pathway to Victory

Does using Yes/NO ads to promote a new premium offer impact donor conversion rates?

Experiment ID: #163499

Pathway to Victory

Experiment Summary

Timeframe: 10/20/2023 - 11/06/2023

Pathway to Victory is the broadcast ministry arm of First Baptist Dallas, one of the largest Baptist churches in the country.

For a new resource that was created to respond to questions PTV had been getting with the chaos in Israel, we wanted to test whether using yes/no creative would make an impact on donor conversions. The yes/no concept is something PTV had used on organic posts; we had not tested this before for Facebook ads.

In this experiment, which we ran as adding an additional creative element to a campaign, not a straight A/B test, we tested the control which was an image of the book cover against the yes/no ads.

Research Question

We believe that using yes/no ads for a Facebook audience will achieve a change in conversion rates.

Design

C: Control
Array ( [0] => Array ( [ID] => 65081 [id] => 65081 [title] => Control.png [filename] => Control-12.png [filesize] => 482775 [url] => https://www.nextafter.com/wp-content/uploads/Control-12.png [link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/control-png-8716/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-8716 [status] => inherit [uploaded_to] => 65075 [date] => 2024-08-19 20:33:37 [modified] => 2024-08-19 20:33:37 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 508 [height] => 725 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-12-210x300.png [medium-width] => 210 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-12.png [medium_large-width] => 508 [medium_large-height] => 725 [large] => https://www.nextafter.com/wp-content/uploads/Control-12.png [large-width] => 508 [large-height] => 725 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-12.png [1536x1536-width] => 508 [1536x1536-height] => 725 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-12.png [2048x2048-width] => 508 [2048x2048-height] => 725 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-12-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-12-35x50.png [admin-width] => 35 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Control-12.png [full-width] => 508 [full-height] => 725 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-12-372x531.png [thumb_372-width] => 372 [thumb_372-height] => 531 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-438x625.png [single_post_thumbnail-width] => 438 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-12-22x32.png [thumb_32_32-width] => 22 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-12-34x48.png [thumb_48_48-width] => 34 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-12-45x64.png [thumb_64_64-width] => 45 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-12-90x128.png [thumb_128_128-width] => 90 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-12-47x67.png [thumb_122_67-width] => 47 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-12-107x153.png [thumb_153_153-width] => 107 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-12-114x163.png [thumb_130_163-width] => 114 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-12-124x177.png [thumb_232_177-width] => 124 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-12-455x650.png [thumb_272_325-width] => 455 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-12-242x346.png [thumb_271_346-width] => 242 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-12-190x271.png [thumb_271_271-width] => 190 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-12-121x172.png [thumb_272_172-width] => 121 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-12-323x461.png [thumb_372_461-width] => 323 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-12-350x500.png [thumb_372_500-width] => 350 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-12-189x270.png [thumb_385_270-width] => 189 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-12-169x241.png [thumb_416_241-width] => 169 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-12-287x410.png [thumb_448_410-width] => 287 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-12.png [thumb_448_410_retina-width] => 508 [thumb_448_410_retina-height] => 725 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-12-268x382.png [thumb_574_382-width] => 268 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-12-277x395.png [thumb_776_395-width] => 277 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-12.png [gform-image-choice-sm-width] => 210 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-12.png [gform-image-choice-md-width] => 280 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-12.png [gform-image-choice-lg-width] => 420 [gform-image-choice-lg-height] => 600 ) ) [1] => Array ( [ID] => 65085 [id] => 65085 [title] => Treatment_#1.png [filename] => Treatment_1-15.png [filesize] => 476929 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/treatment_1-png-2980/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-2980 [status] => inherit [uploaded_to] => 65075 [date] => 2024-08-19 20:33:52 [modified] => 2024-08-19 20:33:52 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 510 [height] => 725 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-211x300.png [medium-width] => 211 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [medium_large-width] => 510 [medium_large-height] => 725 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [large-width] => 510 [large-height] => 725 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [1536x1536-width] => 510 [1536x1536-height] => 725 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [2048x2048-width] => 510 [2048x2048-height] => 725 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-35x50.png [admin-width] => 35 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [full-width] => 510 [full-height] => 725 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-372x529.png [thumb_372-width] => 372 [thumb_372-height] => 529 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-440x625.png [single_post_thumbnail-width] => 440 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-23x32.png [thumb_32_32-width] => 23 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-34x48.png [thumb_48_48-width] => 34 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-45x64.png [thumb_64_64-width] => 45 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-90x128.png [thumb_128_128-width] => 90 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-47x67.png [thumb_122_67-width] => 47 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-108x153.png [thumb_153_153-width] => 108 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-115x163.png [thumb_130_163-width] => 115 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-125x177.png [thumb_232_177-width] => 125 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-457x650.png [thumb_272_325-width] => 457 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-243x346.png [thumb_271_346-width] => 243 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-191x271.png [thumb_271_271-width] => 191 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-121x172.png [thumb_272_172-width] => 121 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-324x461.png [thumb_372_461-width] => 324 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-352x500.png [thumb_372_500-width] => 352 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-190x270.png [thumb_385_270-width] => 190 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-170x241.png [thumb_416_241-width] => 170 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-288x410.png [thumb_448_410-width] => 288 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [thumb_448_410_retina-width] => 510 [thumb_448_410_retina-height] => 725 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-269x382.png [thumb_574_382-width] => 269 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-278x395.png [thumb_776_395-width] => 278 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [gform-image-choice-sm-width] => 211 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [gform-image-choice-md-width] => 281 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [gform-image-choice-lg-width] => 422 [gform-image-choice-lg-height] => 600 ) ) )
T1: Treatment #1
Array ( [0] => Array ( [ID] => 65081 [id] => 65081 [title] => Control.png [filename] => Control-12.png [filesize] => 482775 [url] => https://www.nextafter.com/wp-content/uploads/Control-12.png [link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/control-png-8716/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-8716 [status] => inherit [uploaded_to] => 65075 [date] => 2024-08-19 20:33:37 [modified] => 2024-08-19 20:33:37 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 508 [height] => 725 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-12-210x300.png [medium-width] => 210 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-12.png [medium_large-width] => 508 [medium_large-height] => 725 [large] => https://www.nextafter.com/wp-content/uploads/Control-12.png [large-width] => 508 [large-height] => 725 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-12.png [1536x1536-width] => 508 [1536x1536-height] => 725 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-12.png [2048x2048-width] => 508 [2048x2048-height] => 725 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-12-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-12-35x50.png [admin-width] => 35 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Control-12.png [full-width] => 508 [full-height] => 725 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-12-372x531.png [thumb_372-width] => 372 [thumb_372-height] => 531 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-438x625.png [single_post_thumbnail-width] => 438 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-12-22x32.png [thumb_32_32-width] => 22 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-12-34x48.png [thumb_48_48-width] => 34 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-12-45x64.png [thumb_64_64-width] => 45 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-12-90x128.png [thumb_128_128-width] => 90 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-12-47x67.png [thumb_122_67-width] => 47 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-12-107x153.png [thumb_153_153-width] => 107 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-12-114x163.png [thumb_130_163-width] => 114 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-12-124x177.png [thumb_232_177-width] => 124 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-12-455x650.png [thumb_272_325-width] => 455 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-12-242x346.png [thumb_271_346-width] => 242 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-12-190x271.png [thumb_271_271-width] => 190 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-12-121x172.png [thumb_272_172-width] => 121 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-12-323x461.png [thumb_372_461-width] => 323 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-12-350x500.png [thumb_372_500-width] => 350 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-12-189x270.png [thumb_385_270-width] => 189 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-12-169x241.png [thumb_416_241-width] => 169 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-12-287x410.png [thumb_448_410-width] => 287 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-12.png [thumb_448_410_retina-width] => 508 [thumb_448_410_retina-height] => 725 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-12-268x382.png [thumb_574_382-width] => 268 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-12-277x395.png [thumb_776_395-width] => 277 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-12.png [gform-image-choice-sm-width] => 210 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-12.png [gform-image-choice-md-width] => 280 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-12.png [gform-image-choice-lg-width] => 420 [gform-image-choice-lg-height] => 600 ) ) [1] => Array ( [ID] => 65085 [id] => 65085 [title] => Treatment_#1.png [filename] => Treatment_1-15.png [filesize] => 476929 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/treatment_1-png-2980/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-2980 [status] => inherit [uploaded_to] => 65075 [date] => 2024-08-19 20:33:52 [modified] => 2024-08-19 20:33:52 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 510 [height] => 725 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-211x300.png [medium-width] => 211 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [medium_large-width] => 510 [medium_large-height] => 725 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [large-width] => 510 [large-height] => 725 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [1536x1536-width] => 510 [1536x1536-height] => 725 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [2048x2048-width] => 510 [2048x2048-height] => 725 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-35x50.png [admin-width] => 35 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [full-width] => 510 [full-height] => 725 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-372x529.png [thumb_372-width] => 372 [thumb_372-height] => 529 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-440x625.png [single_post_thumbnail-width] => 440 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-23x32.png [thumb_32_32-width] => 23 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-34x48.png [thumb_48_48-width] => 34 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-45x64.png [thumb_64_64-width] => 45 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-90x128.png [thumb_128_128-width] => 90 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-47x67.png [thumb_122_67-width] => 47 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-108x153.png [thumb_153_153-width] => 108 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-115x163.png [thumb_130_163-width] => 115 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-125x177.png [thumb_232_177-width] => 125 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-457x650.png [thumb_272_325-width] => 457 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-243x346.png [thumb_271_346-width] => 243 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-191x271.png [thumb_271_271-width] => 191 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-121x172.png [thumb_272_172-width] => 121 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-324x461.png [thumb_372_461-width] => 324 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-352x500.png [thumb_372_500-width] => 352 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-190x270.png [thumb_385_270-width] => 190 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-170x241.png [thumb_416_241-width] => 170 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-288x410.png [thumb_448_410-width] => 288 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [thumb_448_410_retina-width] => 510 [thumb_448_410_retina-height] => 725 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-269x382.png [thumb_574_382-width] => 269 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-278x395.png [thumb_776_395-width] => 278 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [gform-image-choice-sm-width] => 211 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [gform-image-choice-md-width] => 281 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png [gform-image-choice-lg-width] => 422 [gform-image-choice-lg-height] => 600 ) ) )

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.04%
T1: Treatment #1 0.03% -16.0% 49.1%

This experiment has a required sample size of 821,684 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95% was not met so the experiment results are not valid.

Key Learnings

The key learning from this experiment is that using yes/no ads for a Facebook audience did not result in a significant change in conversion rates. The treatment actually saw a decrease in donations compared to the control group. It is important to note that the statistical level of confidence for this experiment is below 85%, which means the results cannot be considered valid. Therefore, these results should not be used for future decision making. While CPMs across both creative variations were the same, there was a slight decrease in CTR for the variant (1.09% compared to 0.93%). However the cost to acquire a donor on the yes/no variant was 16% higher. In future experimentation, it would be beneficial to conduct further tests using different ad formats, messaging, or targeting strategies to see if there are any other variables that could impact conversion rates. Additionally, it may be worth exploring other channels or platforms to reach the target audience. Experimentation is crucial for understanding what strategies work best for increasing conversions, and it is important to evaluate results with a statistically significant level of confidence before making any conclusive decisions.


Experiment Documented by NextAfter

Question about experiment #163499

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.