Pathway to Victory
Does using Yes/NO ads to promote a new premium offer impact donor conversion rates?
Experiment ID: #163499
Pathway to Victory
Experiment Summary
Timeframe: 10/20/2023 - 11/06/2023
Pathway to Victory is the broadcast ministry arm of First Baptist Dallas, one of the largest Baptist churches in the country.
For a new resource that was created to respond to questions PTV had been getting with the chaos in Israel, we wanted to test whether using yes/no creative would make an impact on donor conversions. The yes/no concept is something PTV had used on organic posts; we had not tested this before for Facebook ads.
In this experiment, which we ran as adding an additional creative element to a campaign, not a straight A/B test, we tested the control which was an image of the book cover against the yes/no ads.
Research Question
We believe that using yes/no ads for a Facebook audience will achieve a change in conversion rates.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 65081
[id] => 65081
[title] => Control.png
[filename] => Control-12.png
[filesize] => 482775
[url] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/control-png-8716/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-8716
[status] => inherit
[uploaded_to] => 65075
[date] => 2024-08-19 20:33:37
[modified] => 2024-08-19 20:33:37
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 508
[height] => 725
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-12-210x300.png
[medium-width] => 210
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[medium_large-width] => 508
[medium_large-height] => 725
[large] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[large-width] => 508
[large-height] => 725
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[1536x1536-width] => 508
[1536x1536-height] => 725
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[2048x2048-width] => 508
[2048x2048-height] => 725
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-12-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-12-35x50.png
[admin-width] => 35
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[full-width] => 508
[full-height] => 725
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-12-372x531.png
[thumb_372-width] => 372
[thumb_372-height] => 531
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-438x625.png
[single_post_thumbnail-width] => 438
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-12-22x32.png
[thumb_32_32-width] => 22
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-12-34x48.png
[thumb_48_48-width] => 34
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-12-45x64.png
[thumb_64_64-width] => 45
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-12-90x128.png
[thumb_128_128-width] => 90
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-12-47x67.png
[thumb_122_67-width] => 47
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-12-107x153.png
[thumb_153_153-width] => 107
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-12-114x163.png
[thumb_130_163-width] => 114
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-12-124x177.png
[thumb_232_177-width] => 124
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-12-455x650.png
[thumb_272_325-width] => 455
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-12-242x346.png
[thumb_271_346-width] => 242
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-12-190x271.png
[thumb_271_271-width] => 190
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-12-121x172.png
[thumb_272_172-width] => 121
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-12-323x461.png
[thumb_372_461-width] => 323
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-12-350x500.png
[thumb_372_500-width] => 350
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-12-189x270.png
[thumb_385_270-width] => 189
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-12-169x241.png
[thumb_416_241-width] => 169
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-12-287x410.png
[thumb_448_410-width] => 287
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[thumb_448_410_retina-width] => 508
[thumb_448_410_retina-height] => 725
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-12-268x382.png
[thumb_574_382-width] => 268
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-12-277x395.png
[thumb_776_395-width] => 277
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[gform-image-choice-sm-width] => 210
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[gform-image-choice-md-width] => 280
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[gform-image-choice-lg-width] => 420
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 65085
[id] => 65085
[title] => Treatment_#1.png
[filename] => Treatment_1-15.png
[filesize] => 476929
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/treatment_1-png-2980/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-2980
[status] => inherit
[uploaded_to] => 65075
[date] => 2024-08-19 20:33:52
[modified] => 2024-08-19 20:33:52
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 510
[height] => 725
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-211x300.png
[medium-width] => 211
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[medium_large-width] => 510
[medium_large-height] => 725
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[large-width] => 510
[large-height] => 725
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[1536x1536-width] => 510
[1536x1536-height] => 725
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[2048x2048-width] => 510
[2048x2048-height] => 725
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-35x50.png
[admin-width] => 35
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[full-width] => 510
[full-height] => 725
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-372x529.png
[thumb_372-width] => 372
[thumb_372-height] => 529
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-440x625.png
[single_post_thumbnail-width] => 440
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-23x32.png
[thumb_32_32-width] => 23
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-34x48.png
[thumb_48_48-width] => 34
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-45x64.png
[thumb_64_64-width] => 45
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-90x128.png
[thumb_128_128-width] => 90
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-47x67.png
[thumb_122_67-width] => 47
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-108x153.png
[thumb_153_153-width] => 108
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-115x163.png
[thumb_130_163-width] => 115
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-125x177.png
[thumb_232_177-width] => 125
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-457x650.png
[thumb_272_325-width] => 457
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-243x346.png
[thumb_271_346-width] => 243
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-191x271.png
[thumb_271_271-width] => 191
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-121x172.png
[thumb_272_172-width] => 121
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-324x461.png
[thumb_372_461-width] => 324
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-352x500.png
[thumb_372_500-width] => 352
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-190x270.png
[thumb_385_270-width] => 190
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-170x241.png
[thumb_416_241-width] => 170
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-288x410.png
[thumb_448_410-width] => 288
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[thumb_448_410_retina-width] => 510
[thumb_448_410_retina-height] => 725
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-269x382.png
[thumb_574_382-width] => 269
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-278x395.png
[thumb_776_395-width] => 278
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[gform-image-choice-sm-width] => 211
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[gform-image-choice-md-width] => 281
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[gform-image-choice-lg-width] => 422
[gform-image-choice-lg-height] => 600
)
)
)
T1: Treatment #1
Array
(
[0] => Array
(
[ID] => 65081
[id] => 65081
[title] => Control.png
[filename] => Control-12.png
[filesize] => 482775
[url] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/control-png-8716/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-8716
[status] => inherit
[uploaded_to] => 65075
[date] => 2024-08-19 20:33:37
[modified] => 2024-08-19 20:33:37
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 508
[height] => 725
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Control-12-210x300.png
[medium-width] => 210
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[medium_large-width] => 508
[medium_large-height] => 725
[large] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[large-width] => 508
[large-height] => 725
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[1536x1536-width] => 508
[1536x1536-height] => 725
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[2048x2048-width] => 508
[2048x2048-height] => 725
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-12-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Control-12-35x50.png
[admin-width] => 35
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[full-width] => 508
[full-height] => 725
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-12-372x531.png
[thumb_372-width] => 372
[thumb_372-height] => 531
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-12-438x625.png
[single_post_thumbnail-width] => 438
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-12-22x32.png
[thumb_32_32-width] => 22
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-12-34x48.png
[thumb_48_48-width] => 34
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-12-45x64.png
[thumb_64_64-width] => 45
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-12-90x128.png
[thumb_128_128-width] => 90
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-12-47x67.png
[thumb_122_67-width] => 47
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-12-107x153.png
[thumb_153_153-width] => 107
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-12-114x163.png
[thumb_130_163-width] => 114
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-12-124x177.png
[thumb_232_177-width] => 124
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-12-455x650.png
[thumb_272_325-width] => 455
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-12-242x346.png
[thumb_271_346-width] => 242
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-12-190x271.png
[thumb_271_271-width] => 190
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-12-121x172.png
[thumb_272_172-width] => 121
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-12-323x461.png
[thumb_372_461-width] => 323
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-12-350x500.png
[thumb_372_500-width] => 350
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-12-189x270.png
[thumb_385_270-width] => 189
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-12-169x241.png
[thumb_416_241-width] => 169
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-12-287x410.png
[thumb_448_410-width] => 287
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[thumb_448_410_retina-width] => 508
[thumb_448_410_retina-height] => 725
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-12-268x382.png
[thumb_574_382-width] => 268
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-12-277x395.png
[thumb_776_395-width] => 277
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[gform-image-choice-sm-width] => 210
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[gform-image-choice-md-width] => 280
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-12.png
[gform-image-choice-lg-width] => 420
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 65085
[id] => 65085
[title] => Treatment_#1.png
[filename] => Treatment_1-15.png
[filesize] => 476929
[url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[link] => https://www.nextafter.com/experiments/does-using-yes-no-ads-to-promote-a-new-premium-offer-impact-donor-conversion-rates/attachment/treatment_1-png-2980/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-2980
[status] => inherit
[uploaded_to] => 65075
[date] => 2024-08-19 20:33:52
[modified] => 2024-08-19 20:33:52
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 510
[height] => 725
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-211x300.png
[medium-width] => 211
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[medium_large-width] => 510
[medium_large-height] => 725
[large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[large-width] => 510
[large-height] => 725
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[1536x1536-width] => 510
[1536x1536-height] => 725
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[2048x2048-width] => 510
[2048x2048-height] => 725
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-35x50.png
[admin-width] => 35
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[full-width] => 510
[full-height] => 725
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-372x529.png
[thumb_372-width] => 372
[thumb_372-height] => 529
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-440x625.png
[single_post_thumbnail-width] => 440
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-23x32.png
[thumb_32_32-width] => 23
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-34x48.png
[thumb_48_48-width] => 34
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-45x64.png
[thumb_64_64-width] => 45
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-90x128.png
[thumb_128_128-width] => 90
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-47x67.png
[thumb_122_67-width] => 47
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-108x153.png
[thumb_153_153-width] => 108
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-115x163.png
[thumb_130_163-width] => 115
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-125x177.png
[thumb_232_177-width] => 125
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-457x650.png
[thumb_272_325-width] => 457
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-243x346.png
[thumb_271_346-width] => 243
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-191x271.png
[thumb_271_271-width] => 191
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-121x172.png
[thumb_272_172-width] => 121
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-324x461.png
[thumb_372_461-width] => 324
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-352x500.png
[thumb_372_500-width] => 352
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-190x270.png
[thumb_385_270-width] => 190
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-170x241.png
[thumb_416_241-width] => 170
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-288x410.png
[thumb_448_410-width] => 288
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[thumb_448_410_retina-width] => 510
[thumb_448_410_retina-height] => 725
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-269x382.png
[thumb_574_382-width] => 269
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15-278x395.png
[thumb_776_395-width] => 278
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[gform-image-choice-sm-width] => 211
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[gform-image-choice-md-width] => 281
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-15.png
[gform-image-choice-lg-width] => 422
[gform-image-choice-lg-height] => 600
)
)
)
Results
| |
Treatment Name |
Conv. Rate |
Relative Difference |
Confidence |
| C: |
Control
|
0.04% |
|
|
| T1: |
Treatment #1
|
0.03% |
-16.0% |
49.1%
|
This experiment has a required sample size of
821,684 in order to be valid. Unfortunately, the required sample size was not met
and a level of confidence above 95%
was not met so
the experiment results are not valid.
Key Learnings
The key learning from this experiment is that using yes/no ads for a Facebook audience did not result in a significant change in conversion rates. The treatment actually saw a decrease in donations compared to the control group.
It is important to note that the statistical level of confidence for this experiment is below 85%, which means the results cannot be considered valid. Therefore, these results should not be used for future decision making.
While CPMs across both creative variations were the same, there was a slight decrease in CTR for the variant (1.09% compared to 0.93%). However the cost to acquire a donor on the yes/no variant was 16% higher.
In future experimentation, it would be beneficial to conduct further tests using different ad formats, messaging, or targeting strategies to see if there are any other variables that could impact conversion rates. Additionally, it may be worth exploring other channels or platforms to reach the target audience. Experimentation is crucial for understanding what strategies work best for increasing conversions, and it is important to evaluate results with a statistically significant level of confidence before making any conclusive decisions.