Hoover Institution

How changing copy to focus on a problem/solution impacted email click rates for a donor audience

Experiment ID: #192541

Hoover Institution

Experiment Summary

Timeframe: 10/24/2024 - 10/24/2024

The Hoover Institution is a public policy research center and think tank located at Stanford University in California, United States.

Hoover recirculates a new content offer every Thursday by sending out an email to their Donor and NonDonor audiences. This specific test was on their non-donor audience. For this test on one of their most popular offers, we wanted to see whether email copy focused on the problem/solution will impact email clicks.

Research Question

We believe that using a problem/solution approach for non-donors will achieve a change in email clicks.

Design

C: Control
Array ( [0] => Array ( [ID] => 67808 [id] => 67808 [title] => Control.png [filename] => Control-46.png [filesize] => 170150 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [link] => https://www.nextafter.com/experiments/how-changing-the-copy-to-focus-on-a-problem-solution-impacted-email-click-rates-for-a-donor-audience/attachment/control-png-9229/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-9229 [status] => inherit [uploaded_to] => 67730 [date] => 2024-11-01 21:31:24 [modified] => 2024-11-01 21:31:24 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 805 [height] => 1189 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-203x300.png [medium-width] => 203 [medium-height] => 300 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-768x1134.png [medium_large-width] => 640 [medium_large-height] => 945 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-693x1024.png [large-width] => 640 [large-height] => 946 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [1536x1536-width] => 805 [1536x1536-height] => 1189 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [2048x2048-width] => 805 [2048x2048-height] => 1189 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-600x400.png [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-600x600.png [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-34x50.png [admin-width] => 34 [admin-height] => 50 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [full-width] => 805 [full-height] => 1189 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-372x549.png [thumb_372-width] => 372 [thumb_372-height] => 549 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-423x625.png [single_post_thumbnail-width] => 423 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-22x32.png [thumb_32_32-width] => 22 [thumb_32_32-height] => 32 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-32x48.png [thumb_48_48-width] => 32 [thumb_48_48-height] => 48 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-43x64.png [thumb_64_64-width] => 43 [thumb_64_64-height] => 64 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-87x128.png [thumb_128_128-width] => 87 [thumb_128_128-height] => 128 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-45x67.png [thumb_122_67-width] => 45 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-104x153.png [thumb_153_153-width] => 104 [thumb_153_153-height] => 153 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-110x163.png [thumb_130_163-width] => 110 [thumb_130_163-height] => 163 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-120x177.png [thumb_232_177-width] => 120 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-440x650.png [thumb_272_325-width] => 440 [thumb_272_325-height] => 650 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-234x346.png [thumb_271_346-width] => 234 [thumb_271_346-height] => 346 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-183x271.png [thumb_271_271-width] => 183 [thumb_271_271-height] => 271 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-116x172.png [thumb_272_172-width] => 116 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-312x461.png [thumb_372_461-width] => 312 [thumb_372_461-height] => 461 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-339x500.png [thumb_372_500-width] => 339 [thumb_372_500-height] => 500 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-183x270.png [thumb_385_270-width] => 183 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-163x241.png [thumb_416_241-width] => 163 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-278x410.png [thumb_448_410-width] => 278 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-555x820.png [thumb_448_410_retina-width] => 555 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-259x382.png [thumb_574_382-width] => 259 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-267x395.png [thumb_776_395-width] => 267 [thumb_776_395-height] => 395 ) ) [1] => Array ( [ID] => 67809 [id] => 67809 [title] => Treatment_#1.png [filename] => Treatment_1-30.png [filesize] => 162811 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [link] => https://www.nextafter.com/experiments/how-changing-the-copy-to-focus-on-a-problem-solution-impacted-email-click-rates-for-a-donor-audience/attachment/treatment_1-png-3177/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-3177 [status] => inherit [uploaded_to] => 67730 [date] => 2024-11-01 21:31:46 [modified] => 2024-11-01 21:31:46 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 805 [height] => 1156 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-209x300.png [medium-width] => 209 [medium-height] => 300 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-768x1103.png [medium_large-width] => 640 [medium_large-height] => 919 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-713x1024.png [large-width] => 640 [large-height] => 919 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [1536x1536-width] => 805 [1536x1536-height] => 1156 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [2048x2048-width] => 805 [2048x2048-height] => 1156 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-600x400.png [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-600x600.png [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-35x50.png [admin-width] => 35 [admin-height] => 50 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [full-width] => 805 [full-height] => 1156 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-372x534.png [thumb_372-width] => 372 [thumb_372-height] => 534 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-435x625.png [single_post_thumbnail-width] => 435 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-22x32.png [thumb_32_32-width] => 22 [thumb_32_32-height] => 32 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-33x48.png [thumb_48_48-width] => 33 [thumb_48_48-height] => 48 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-45x64.png [thumb_64_64-width] => 45 [thumb_64_64-height] => 64 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-89x128.png [thumb_128_128-width] => 89 [thumb_128_128-height] => 128 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-47x67.png [thumb_122_67-width] => 47 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-107x153.png [thumb_153_153-width] => 107 [thumb_153_153-height] => 153 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-114x163.png [thumb_130_163-width] => 114 [thumb_130_163-height] => 163 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-123x177.png [thumb_232_177-width] => 123 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-453x650.png [thumb_272_325-width] => 453 [thumb_272_325-height] => 650 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-241x346.png [thumb_271_346-width] => 241 [thumb_271_346-height] => 346 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-189x271.png [thumb_271_271-width] => 189 [thumb_271_271-height] => 271 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-120x172.png [thumb_272_172-width] => 120 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-321x461.png [thumb_372_461-width] => 321 [thumb_372_461-height] => 461 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-348x500.png [thumb_372_500-width] => 348 [thumb_372_500-height] => 500 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-188x270.png [thumb_385_270-width] => 188 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-168x241.png [thumb_416_241-width] => 168 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-286x410.png [thumb_448_410-width] => 286 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-571x820.png [thumb_448_410_retina-width] => 571 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-266x382.png [thumb_574_382-width] => 266 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-275x395.png [thumb_776_395-width] => 275 [thumb_776_395-height] => 395 ) ) )
T1: Treatment #1
Array ( [0] => Array ( [ID] => 67808 [id] => 67808 [title] => Control.png [filename] => Control-46.png [filesize] => 170150 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [link] => https://www.nextafter.com/experiments/how-changing-the-copy-to-focus-on-a-problem-solution-impacted-email-click-rates-for-a-donor-audience/attachment/control-png-9229/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-9229 [status] => inherit [uploaded_to] => 67730 [date] => 2024-11-01 21:31:24 [modified] => 2024-11-01 21:31:24 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 805 [height] => 1189 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-203x300.png [medium-width] => 203 [medium-height] => 300 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-768x1134.png [medium_large-width] => 640 [medium_large-height] => 945 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-693x1024.png [large-width] => 640 [large-height] => 946 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [1536x1536-width] => 805 [1536x1536-height] => 1189 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [2048x2048-width] => 805 [2048x2048-height] => 1189 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-600x400.png [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-600x600.png [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-34x50.png [admin-width] => 34 [admin-height] => 50 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46.png [full-width] => 805 [full-height] => 1189 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-372x549.png [thumb_372-width] => 372 [thumb_372-height] => 549 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-423x625.png [single_post_thumbnail-width] => 423 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-22x32.png [thumb_32_32-width] => 22 [thumb_32_32-height] => 32 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-32x48.png [thumb_48_48-width] => 32 [thumb_48_48-height] => 48 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-43x64.png [thumb_64_64-width] => 43 [thumb_64_64-height] => 64 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-87x128.png [thumb_128_128-width] => 87 [thumb_128_128-height] => 128 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-45x67.png [thumb_122_67-width] => 45 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-104x153.png [thumb_153_153-width] => 104 [thumb_153_153-height] => 153 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-110x163.png [thumb_130_163-width] => 110 [thumb_130_163-height] => 163 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-120x177.png [thumb_232_177-width] => 120 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-440x650.png [thumb_272_325-width] => 440 [thumb_272_325-height] => 650 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-234x346.png [thumb_271_346-width] => 234 [thumb_271_346-height] => 346 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-183x271.png [thumb_271_271-width] => 183 [thumb_271_271-height] => 271 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-116x172.png [thumb_272_172-width] => 116 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-312x461.png [thumb_372_461-width] => 312 [thumb_372_461-height] => 461 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-339x500.png [thumb_372_500-width] => 339 [thumb_372_500-height] => 500 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-183x270.png [thumb_385_270-width] => 183 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-163x241.png [thumb_416_241-width] => 163 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-278x410.png [thumb_448_410-width] => 278 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-555x820.png [thumb_448_410_retina-width] => 555 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-259x382.png [thumb_574_382-width] => 259 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Control-46-267x395.png [thumb_776_395-width] => 267 [thumb_776_395-height] => 395 ) ) [1] => Array ( [ID] => 67809 [id] => 67809 [title] => Treatment_#1.png [filename] => Treatment_1-30.png [filesize] => 162811 [url] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [link] => https://www.nextafter.com/experiments/how-changing-the-copy-to-focus-on-a-problem-solution-impacted-email-click-rates-for-a-donor-audience/attachment/treatment_1-png-3177/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-3177 [status] => inherit [uploaded_to] => 67730 [date] => 2024-11-01 21:31:46 [modified] => 2024-11-01 21:31:46 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://nextafter-1a91a.kxcdn.com/wp-includes/images/media/default.png [width] => 805 [height] => 1156 [sizes] => Array ( [thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-209x300.png [medium-width] => 209 [medium-height] => 300 [medium_large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-768x1103.png [medium_large-width] => 640 [medium_large-height] => 919 [large] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-713x1024.png [large-width] => 640 [large-height] => 919 [1536x1536] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [1536x1536-width] => 805 [1536x1536-height] => 1156 [2048x2048] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [2048x2048-width] => 805 [2048x2048-height] => 1156 [ab-block-post-grid-landscape] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-600x400.png [ab-block-post-grid-landscape-width] => 600 [ab-block-post-grid-landscape-height] => 400 [ab-block-post-grid-square] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-600x600.png [ab-block-post-grid-square-width] => 600 [ab-block-post-grid-square-height] => 600 [tp-image-grid] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-35x50.png [admin-width] => 35 [admin-height] => 50 [full] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30.png [full-width] => 805 [full-height] => 1156 [thumb_372] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-372x534.png [thumb_372-width] => 372 [thumb_372-height] => 534 [single_post_thumbnail] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-435x625.png [single_post_thumbnail-width] => 435 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-22x32.png [thumb_32_32-width] => 22 [thumb_32_32-height] => 32 [thumb_48_48] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-33x48.png [thumb_48_48-width] => 33 [thumb_48_48-height] => 48 [thumb_64_64] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-45x64.png [thumb_64_64-width] => 45 [thumb_64_64-height] => 64 [thumb_128_128] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-89x128.png [thumb_128_128-width] => 89 [thumb_128_128-height] => 128 [thumb_122_67] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-47x67.png [thumb_122_67-width] => 47 [thumb_122_67-height] => 67 [thumb_153_153] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-107x153.png [thumb_153_153-width] => 107 [thumb_153_153-height] => 153 [thumb_130_163] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-114x163.png [thumb_130_163-width] => 114 [thumb_130_163-height] => 163 [thumb_232_177] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-123x177.png [thumb_232_177-width] => 123 [thumb_232_177-height] => 177 [thumb_272_325] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-453x650.png [thumb_272_325-width] => 453 [thumb_272_325-height] => 650 [thumb_271_346] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-241x346.png [thumb_271_346-width] => 241 [thumb_271_346-height] => 346 [thumb_271_271] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-189x271.png [thumb_271_271-width] => 189 [thumb_271_271-height] => 271 [thumb_272_172] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-120x172.png [thumb_272_172-width] => 120 [thumb_272_172-height] => 172 [thumb_372_461] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-321x461.png [thumb_372_461-width] => 321 [thumb_372_461-height] => 461 [thumb_372_500] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-348x500.png [thumb_372_500-width] => 348 [thumb_372_500-height] => 500 [thumb_385_270] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-188x270.png [thumb_385_270-width] => 188 [thumb_385_270-height] => 270 [thumb_416_241] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-168x241.png [thumb_416_241-width] => 168 [thumb_416_241-height] => 241 [thumb_448_410] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-286x410.png [thumb_448_410-width] => 286 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-571x820.png [thumb_448_410_retina-width] => 571 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-266x382.png [thumb_574_382-width] => 266 [thumb_574_382-height] => 382 [thumb_776_395] => https://nextafter-1a91a.kxcdn.com/wp-content/uploads/Treatment_1-30-275x395.png [thumb_776_395-width] => 275 [thumb_776_395-height] => 395 ) ) )

Results

 Treatment NameClick RateRelative DifferenceConfidence
C: Control 1.7%
T1: Treatment #1 1.4%-15.5% 100.0%

This experiment has a required sample size of 16,969 in order to be valid. Since the experiment had a total sample size of 227,474, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    15.5% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift

Key Learnings

The key learning from this experiment is that using a relevancy approach for non-donors did not result in an increase in email clicks. In fact, the treatment group experienced a 15.5% decrease in clicks compared to the control group, with a high level of statistical confidence.

Moving forward, it is essential to carefully evaluate the impact of different messaging approaches on email engagement before implementing them in future experiments. It is also important to continue testing to understand what resonates best with the target audience in order to optimize email performance and achieve the desired outcomes.


Experiment Documented by NextAfter

Question about experiment #192541

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.