The Heritage Foundation
How using a more provocative title for the subject of a survey impacts Google ad clicks
Experiment ID: #194779
The Heritage Foundation
Experiment Summary
Ended On: 01/21/2025
The Heritage Foundation was running a “Stop the Administrative State” survey and promoting through Google ads. In an attempt to increase click rates on the ads, they wanted to test whether using the more provocative term “Deep State” in place of the more straightforward “Administrative State” survey title would achieve this effect because people would be more provoked to take action against the subject when referred to as a more negative term.
The control’s Google ad image and copy options, including 8 headline and 5 description options, referred to the Administrative State, while the treatment’s ad image and copy referred to the Deep State, and both received 50% of traffic. In addition, the control led to survey landing page with “Administrative State” language, and the treatment led to a survey page with “Deep State” language for continuity.
This test ran for 3 weeks.
Research Question
We believe that using an more provocative name for the Administrative State for survey Google ad viewers will achieve an increase in clicks on the ad because the term “Deep State” invokes stronger emotions.
Design
C: Control
Array
(
[0] => Array
(
[ID] => 69670
[id] => 69670
[title] => Control.png
[filename] => Control-29.png
[filesize] => 108494
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-google-ad-clicks/attachment/control-png-9733/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-9733
[status] => inherit
[uploaded_to] => 69661
[date] => 2025-01-21 16:25:14
[modified] => 2025-01-21 16:25:14
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 313
[height] => 260
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-300x249.png
[medium-width] => 300
[medium-height] => 249
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[medium_large-width] => 313
[medium_large-height] => 260
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[large-width] => 313
[large-height] => 260
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[1536x1536-width] => 313
[1536x1536-height] => 260
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[2048x2048-width] => 313
[2048x2048-height] => 260
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-50x42.png
[admin-width] => 50
[admin-height] => 42
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[full-width] => 313
[full-height] => 260
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_372-width] => 313
[thumb_372-height] => 260
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[single_post_thumbnail-width] => 313
[single_post_thumbnail-height] => 260
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-32x27.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 27
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-48x40.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 40
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-64x53.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 53
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-128x106.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 106
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-81x67.png
[thumb_122_67-width] => 81
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-153x127.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 127
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-130x108.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 108
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-213x177.png
[thumb_232_177-width] => 213
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_272_325-width] => 313
[thumb_272_325-height] => 260
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-271x225.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 225
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-271x225.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 225
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-207x172.png
[thumb_272_172-width] => 207
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_372_461-width] => 313
[thumb_372_461-height] => 260
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_372_500-width] => 313
[thumb_372_500-height] => 260
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_385_270-width] => 313
[thumb_385_270-height] => 260
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-290x241.png
[thumb_416_241-width] => 290
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_448_410-width] => 313
[thumb_448_410-height] => 260
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_448_410_retina-width] => 313
[thumb_448_410_retina-height] => 260
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_574_382-width] => 313
[thumb_574_382-height] => 260
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_776_395-width] => 313
[thumb_776_395-height] => 260
) ) [1] => Array
(
[ID] => 69671
[id] => 69671
[title] => Treatment_#1.png
[filename] => Treatment_1-13.png
[filesize] => 104425
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-google-ad-clicks/attachment/treatment_1-png-3460/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-3460
[status] => inherit
[uploaded_to] => 69661
[date] => 2025-01-21 16:25:23
[modified] => 2025-01-21 16:25:23
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 314
[height] => 268
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-300x256.png
[medium-width] => 300
[medium-height] => 256
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[medium_large-width] => 314
[medium_large-height] => 268
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[large-width] => 314
[large-height] => 268
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[1536x1536-width] => 314
[1536x1536-height] => 268
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[2048x2048-width] => 314
[2048x2048-height] => 268
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-50x43.png
[admin-width] => 50
[admin-height] => 43
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[full-width] => 314
[full-height] => 268
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_372-width] => 314
[thumb_372-height] => 268
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[single_post_thumbnail-width] => 314
[single_post_thumbnail-height] => 268
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-32x27.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 27
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-48x41.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 41
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-64x55.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 55
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-128x109.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 109
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-79x67.png
[thumb_122_67-width] => 79
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-153x131.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 131
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-130x111.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 111
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-207x177.png
[thumb_232_177-width] => 207
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_272_325-width] => 314
[thumb_272_325-height] => 268
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-271x231.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 231
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-271x231.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 231
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-202x172.png
[thumb_272_172-width] => 202
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_372_461-width] => 314
[thumb_372_461-height] => 268
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_372_500-width] => 314
[thumb_372_500-height] => 268
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_385_270-width] => 314
[thumb_385_270-height] => 268
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-282x241.png
[thumb_416_241-width] => 282
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_448_410-width] => 314
[thumb_448_410-height] => 268
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_448_410_retina-width] => 314
[thumb_448_410_retina-height] => 268
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_574_382-width] => 314
[thumb_574_382-height] => 268
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_776_395-width] => 314
[thumb_776_395-height] => 268
) ) )
T1: Treatment #1
Array
(
[0] => Array
(
[ID] => 69670
[id] => 69670
[title] => Control.png
[filename] => Control-29.png
[filesize] => 108494
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-google-ad-clicks/attachment/control-png-9733/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-png-9733
[status] => inherit
[uploaded_to] => 69661
[date] => 2025-01-21 16:25:14
[modified] => 2025-01-21 16:25:14
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 313
[height] => 260
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-300x249.png
[medium-width] => 300
[medium-height] => 249
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[medium_large-width] => 313
[medium_large-height] => 260
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[large-width] => 313
[large-height] => 260
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[1536x1536-width] => 313
[1536x1536-height] => 260
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[2048x2048-width] => 313
[2048x2048-height] => 260
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-50x42.png
[admin-width] => 50
[admin-height] => 42
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[full-width] => 313
[full-height] => 260
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_372-width] => 313
[thumb_372-height] => 260
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[single_post_thumbnail-width] => 313
[single_post_thumbnail-height] => 260
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-32x27.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 27
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-48x40.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 40
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-64x53.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 53
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-128x106.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 106
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-81x67.png
[thumb_122_67-width] => 81
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-153x127.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 127
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-130x108.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 108
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-213x177.png
[thumb_232_177-width] => 213
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_272_325-width] => 313
[thumb_272_325-height] => 260
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-271x225.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 225
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-271x225.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 225
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-207x172.png
[thumb_272_172-width] => 207
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_372_461-width] => 313
[thumb_372_461-height] => 260
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_372_500-width] => 313
[thumb_372_500-height] => 260
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_385_270-width] => 313
[thumb_385_270-height] => 260
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29-290x241.png
[thumb_416_241-width] => 290
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_448_410-width] => 313
[thumb_448_410-height] => 260
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_448_410_retina-width] => 313
[thumb_448_410_retina-height] => 260
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_574_382-width] => 313
[thumb_574_382-height] => 260
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-29.png
[thumb_776_395-width] => 313
[thumb_776_395-height] => 260
) ) [1] => Array
(
[ID] => 69671
[id] => 69671
[title] => Treatment_#1.png
[filename] => Treatment_1-13.png
[filesize] => 104425
[url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[link] => https://www.nextafter.com/experiments/how-using-a-more-provocative-title-for-the-subject-of-a-survey-impacts-google-ad-clicks/attachment/treatment_1-png-3460/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => treatment_1-png-3460
[status] => inherit
[uploaded_to] => 69661
[date] => 2025-01-21 16:25:23
[modified] => 2025-01-21 16:25:23
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png
[width] => 314
[height] => 268
[sizes] => Array
(
[thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-300x256.png
[medium-width] => 300
[medium-height] => 256
[medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[medium_large-width] => 314
[medium_large-height] => 268
[large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[large-width] => 314
[large-height] => 268
[1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[1536x1536-width] => 314
[1536x1536-height] => 268
[2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[2048x2048-width] => 314
[2048x2048-height] => 268
[ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-600x400.png
[ab-block-post-grid-landscape-width] => 600
[ab-block-post-grid-landscape-height] => 400
[ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-600x600.png
[ab-block-post-grid-square-width] => 600
[ab-block-post-grid-square-height] => 600
[tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-700x700.png
[tp-image-grid-width] => 700
[tp-image-grid-height] => 700
[post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-50x43.png
[admin-width] => 50
[admin-height] => 43
[full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[full-width] => 314
[full-height] => 268
[thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_372-width] => 314
[thumb_372-height] => 268
[single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[single_post_thumbnail-width] => 314
[single_post_thumbnail-height] => 268
[thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-32x27.png
[thumb_32_32-width] => 32
[thumb_32_32-height] => 27
[thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-48x41.png
[thumb_48_48-width] => 48
[thumb_48_48-height] => 41
[thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-64x55.png
[thumb_64_64-width] => 64
[thumb_64_64-height] => 55
[thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-128x109.png
[thumb_128_128-width] => 128
[thumb_128_128-height] => 109
[thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-79x67.png
[thumb_122_67-width] => 79
[thumb_122_67-height] => 67
[thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-153x131.png
[thumb_153_153-width] => 153
[thumb_153_153-height] => 131
[thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-130x111.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 111
[thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-207x177.png
[thumb_232_177-width] => 207
[thumb_232_177-height] => 177
[thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_272_325-width] => 314
[thumb_272_325-height] => 268
[thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-271x231.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 231
[thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-271x231.png
[thumb_271_271-width] => 271
[thumb_271_271-height] => 231
[thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-202x172.png
[thumb_272_172-width] => 202
[thumb_272_172-height] => 172
[thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_372_461-width] => 314
[thumb_372_461-height] => 268
[thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_372_500-width] => 314
[thumb_372_500-height] => 268
[thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_385_270-width] => 314
[thumb_385_270-height] => 268
[thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13-282x241.png
[thumb_416_241-width] => 282
[thumb_416_241-height] => 241
[thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_448_410-width] => 314
[thumb_448_410-height] => 268
[thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_448_410_retina-width] => 314
[thumb_448_410_retina-height] => 268
[thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_574_382-width] => 314
[thumb_574_382-height] => 268
[thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-13.png
[thumb_776_395-width] => 314
[thumb_776_395-height] => 268
) ) )
Results
| Treatment Name | Click Rate | Relative Difference | Confidence |
---|
C: | Control | 1.8% | | |
T1: | Treatment #1 | 0.80% | -55.4% | 100.0% |
This experiment has a required sample size of
839 in order to be valid. Since the experiment had a total sample size of
1,407,300, and
the level of confidence is above 95% the experiment results are
valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average
gift).
This experiment produced the following results:
55.4% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The key learning from this experiment is that using the term “Deep State” in Google ads instead of the more straightforward “Administrative State” led to a significant decrease in clicks. Contrary to the hypothesis, the more provocative term did not invoke stronger emotions or prompt more action from viewers.
In addition to click rate, the treatment also saw a 42.3% decrease in email acquisitions on the survey at 100% level of confidence, and an invalid 2.7% decrease in donor acquisition rate (6.82% level of confidence). The email acquisition results also support the same thought that the more provocative term did not invoke stronger emotions and action from viewers.
In future experimentation, it is important to carefully consider the language and terminology used in promotional materials. While provocative terms may sometimes generate interest, they can also backfire and deter potential participants. It is essential to test different messaging strategies to see what resonates best with the target audience before implementing them widely. Additionally, it is crucial to always track and analyze the results of experiments to draw meaningful conclusions and inform future decision-making.