The Heritage Foundation
How using a more provocative title for the subject of a survey impacts Facebook ad clicks
Experiment ID: #194746
The Heritage Foundation
Experiment Summary
Timeframe: 12/19/2024 - 01/08/2025
The Heritage Foundation was running a “Stop the Administrative State” survey and promoting through Facebook ads. In an attempt to increase click rates on the ads, they wanted to test whether using the more provocative term “Deep State” in place of the more straightforward “Administrative State” survey title would achieve this effect because people would be more provoked to take action against the subject when referred to as a more negative term.
The control’s Facebook ad image and copy referred to the Administrative State, while the treatment’s ad image and copy referred to the Deep State, and both received 50% of traffic. In addition, the control led to survey landing page with “Administrative State” language, and the treatment led to a survey page with “Deep State” language for continuity.
This test ran for 3 weeks.
Research Question
We believe that using an more provocative name for the Administrative State for survey Facebook ad viewers will achieve an increase in clicks on the ad because the term “Deep State” invokes stronger emotions. because the term “Deep State” invokes stronger emotions.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 69657
[id] => 69657
[title] => Control.png
[filename] => Control-23.png
[filesize] => 192320
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-facebook-ad-clicks/attachment/control-png-9726/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-9726
[status] => inherit
[uploaded_to] => 69616
[date] => 2025-01-21 16:10:18
[modified] => 2025-01-21 16:10:18
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 324
[height] => 946
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-103x300.png
[medium-width] => 103
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[medium_large-width] => 324
[medium_large-height] => 946
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[large-width] => 324
[large-height] => 946
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[1536x1536-width] => 324
[1536x1536-height] => 946
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[2048x2048-width] => 324
[2048x2048-height] => 946
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-17x50.png
[admin-width] => 17
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[full-width] => 324
[full-height] => 946
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[thumb_372-width] => 324
[thumb_372-height] => 946
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-214x625.png
[single_post_thumbnail-width] => 214
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-11x32.png
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-16x48.png
[thumb_48_48-width] => 16
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-22x64.png
[thumb_64_64-width] => 22
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-44x128.png
[thumb_128_128-width] => 44
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-23x67.png
[thumb_122_67-width] => 23
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-52x153.png
[thumb_153_153-width] => 52
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-56x163.png
[thumb_130_163-width] => 56
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-61x177.png
[thumb_232_177-width] => 61
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-223x650.png
[thumb_272_325-width] => 223
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-119x346.png
[thumb_271_346-width] => 119
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-93x271.png
[thumb_271_271-width] => 93
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-59x172.png
[thumb_272_172-width] => 59
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-158x461.png
[thumb_372_461-width] => 158
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-171x500.png
[thumb_372_500-width] => 171
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-92x270.png
[thumb_385_270-width] => 92
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-83x241.png
[thumb_416_241-width] => 83
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-140x410.png
[thumb_448_410-width] => 140
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-281x820.png
[thumb_448_410_retina-width] => 281
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-131x382.png
[thumb_574_382-width] => 131
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-135x395.png
[thumb_776_395-width] => 135
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 69660
[id] => 69660
[title] => "Deep_State".png
[filename] => Deep_State-8.png
[filesize] => 199316
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-facebook-ad-clicks/attachment/deep_state-png-9/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => deep_state-png-9
[status] => inherit
[uploaded_to] => 69616
[date] => 2025-01-21 16:10:26
[modified] => 2025-01-21 16:10:26
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 340
[height] => 955
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-107x300.png
[medium-width] => 107
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[medium_large-width] => 340
[medium_large-height] => 955
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[large-width] => 340
[large-height] => 955
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[1536x1536-width] => 340
[1536x1536-height] => 955
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[2048x2048-width] => 340
[2048x2048-height] => 955
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-18x50.png
[admin-width] => 18
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[full-width] => 340
[full-height] => 955
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[thumb_372-width] => 340
[thumb_372-height] => 955
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-223x625.png
[single_post_thumbnail-width] => 223
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-11x32.png
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-17x48.png
[thumb_48_48-width] => 17
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-23x64.png
[thumb_64_64-width] => 23
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-46x128.png
[thumb_128_128-width] => 46
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-24x67.png
[thumb_122_67-width] => 24
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-54x153.png
[thumb_153_153-width] => 54
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-58x163.png
[thumb_130_163-width] => 58
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-63x177.png
[thumb_232_177-width] => 63
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-231x650.png
[thumb_272_325-width] => 231
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-123x346.png
[thumb_271_346-width] => 123
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-96x271.png
[thumb_271_271-width] => 96
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-61x172.png
[thumb_272_172-width] => 61
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-164x461.png
[thumb_372_461-width] => 164
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-178x500.png
[thumb_372_500-width] => 178
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-96x270.png
[thumb_385_270-width] => 96
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-86x241.png
[thumb_416_241-width] => 86
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-146x410.png
[thumb_448_410-width] => 146
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-292x820.png
[thumb_448_410_retina-width] => 292
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-136x382.png
[thumb_574_382-width] => 136
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-141x395.png
[thumb_776_395-width] => 141
[thumb_776_395-height] => 395
) ) )
T1: "Deep State"
Array
(
[0] => Array
(
[ID] => 69657
[id] => 69657
[title] => Control.png
[filename] => Control-23.png
[filesize] => 192320
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-facebook-ad-clicks/attachment/control-png-9726/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-9726
[status] => inherit
[uploaded_to] => 69616
[date] => 2025-01-21 16:10:18
[modified] => 2025-01-21 16:10:18
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 324
[height] => 946
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-103x300.png
[medium-width] => 103
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[medium_large-width] => 324
[medium_large-height] => 946
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[large-width] => 324
[large-height] => 946
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[1536x1536-width] => 324
[1536x1536-height] => 946
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[2048x2048-width] => 324
[2048x2048-height] => 946
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-17x50.png
[admin-width] => 17
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[full-width] => 324
[full-height] => 946
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23.png
[thumb_372-width] => 324
[thumb_372-height] => 946
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-214x625.png
[single_post_thumbnail-width] => 214
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-11x32.png
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-16x48.png
[thumb_48_48-width] => 16
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-22x64.png
[thumb_64_64-width] => 22
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-44x128.png
[thumb_128_128-width] => 44
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-23x67.png
[thumb_122_67-width] => 23
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-52x153.png
[thumb_153_153-width] => 52
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-56x163.png
[thumb_130_163-width] => 56
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-61x177.png
[thumb_232_177-width] => 61
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-223x650.png
[thumb_272_325-width] => 223
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-119x346.png
[thumb_271_346-width] => 119
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-93x271.png
[thumb_271_271-width] => 93
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-59x172.png
[thumb_272_172-width] => 59
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-158x461.png
[thumb_372_461-width] => 158
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-171x500.png
[thumb_372_500-width] => 171
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-92x270.png
[thumb_385_270-width] => 92
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-83x241.png
[thumb_416_241-width] => 83
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-140x410.png
[thumb_448_410-width] => 140
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-281x820.png
[thumb_448_410_retina-width] => 281
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-131x382.png
[thumb_574_382-width] => 131
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-23-135x395.png
[thumb_776_395-width] => 135
[thumb_776_395-height] => 395
) ) [1] => Array
(
[ID] => 69660
[id] => 69660
[title] => "Deep_State".png
[filename] => Deep_State-8.png
[filesize] => 199316
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-facebook-ad-clicks/attachment/deep_state-png-9/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => deep_state-png-9
[status] => inherit
[uploaded_to] => 69616
[date] => 2025-01-21 16:10:26
[modified] => 2025-01-21 16:10:26
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 340
[height] => 955
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-107x300.png
[medium-width] => 107
[medium-height] => 300
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[medium_large-width] => 340
[medium_large-height] => 955
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[large-width] => 340
[large-height] => 955
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[1536x1536-width] => 340
[1536x1536-height] => 955
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[2048x2048-width] => 340
[2048x2048-height] => 955
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-18x50.png
[admin-width] => 18
[admin-height] => 50
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[full-width] => 340
[full-height] => 955
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8.png
[thumb_372-width] => 340
[thumb_372-height] => 955
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-223x625.png
[single_post_thumbnail-width] => 223
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-11x32.png
[thumb_32_32-width] => 11
[thumb_32_32-height] => 32
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-17x48.png
[thumb_48_48-width] => 17
[thumb_48_48-height] => 48
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-23x64.png
[thumb_64_64-width] => 23
[thumb_64_64-height] => 64
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-46x128.png
[thumb_128_128-width] => 46
[thumb_128_128-height] => 128
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-24x67.png
[thumb_122_67-width] => 24
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-54x153.png
[thumb_153_153-width] => 54
[thumb_153_153-height] => 153
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-58x163.png
[thumb_130_163-width] => 58
[thumb_130_163-height] => 163
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-63x177.png
[thumb_232_177-width] => 63
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-231x650.png
[thumb_272_325-width] => 231
[thumb_272_325-height] => 650
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-123x346.png
[thumb_271_346-width] => 123
[thumb_271_346-height] => 346
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-96x271.png
[thumb_271_271-width] => 96
[thumb_271_271-height] => 271
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-61x172.png
[thumb_272_172-width] => 61
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-164x461.png
[thumb_372_461-width] => 164
[thumb_372_461-height] => 461
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-178x500.png
[thumb_372_500-width] => 178
[thumb_372_500-height] => 500
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-96x270.png
[thumb_385_270-width] => 96
[thumb_385_270-height] => 270
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-86x241.png
[thumb_416_241-width] => 86
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-146x410.png
[thumb_448_410-width] => 146
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-292x820.png
[thumb_448_410_retina-width] => 292
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-136x382.png
[thumb_574_382-width] => 136
[thumb_574_382-height] => 382
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Deep_State-8-141x395.png
[thumb_776_395-width] => 141
[thumb_776_395-height] => 395
) ) )
Results
| Treatment Name | Click Rate | Relative Difference | Confidence |
---|
C: | Control | 3.3% | | |
T1: | "Deep State" | 3.3% | -1.1% | 15.9% |
This experiment has a required sample size of
1,973,450 in order to be valid. Unfortunately, the required sample size was not met and a level of confidence above 95%
was not met so
the experiment results are not valid.
Key Learnings
The key learning from this experiment is that using the term “Deep State” in place of “Administrative State” did not lead to an increase in clicks on the Facebook ad. In fact, the treatment group saw a slight 1.1% decrease in Clicks for All Traffic compared to the control group.
However, since the statistical level of confidence is only 15.9%, these results are not considered valid and should not be used for future decision making.
In addition to click rate, the treatment saw a valid 12.8% decrease in sign up rate on the landing page at a 97% level of confidence. This suggests that the term “Deep State” may have led to fewer people completing the survey.
Finally, the treatment saw a 8.5% decrease in donor acquisition rate on the post-survey landing page compared to the control, but at a 18.6% level of confidence, this result cannot be relied upon as valid.
In future experimentation, it is essential to carefully design the study with a larger sample size or run the experiment for a longer period of time to increase the statistical power. Additionally, it may be beneficial to test different variables or messaging strategies to determine what resonates best with the audience. Conducting thorough data analysis and ensuring statistical significance will help in making informed decisions based on experimental results.