let us help
Subscribe
The Heritage Foundation

How leading a membership ask email with the problem for existing members affects click-through rate

Experiment ID: #169193

The Heritage Foundation

Experiment Summary

Ended On: 01/31/2024

The Heritage Foundation is running thier annual Kickoff Campaign to acquire members. You can become a member by giving a gift of any size, one-time or recurring. This experiment was ran on the first email of the campaign for existing members in 2023 to Heritage asking them to renew their membership. For this test, we wanted to see how existing members would react to leading the email with a question that poses the problem Heritage is trying to solve rather than leading the email with a direct ask to renew their membership. In doing so, we believe users will be more likely to click through to the donation page and renew their membership because the leading question allows them to relate to the issue and want to be apart of the change.

Research Question

We believe that leading an email with the problem will increase email click-through rate because it will give the reader more context as to why they should renew their membership with Heritage.

Design

C: Control
Array ( [0] => Array ( [ID] => 57115 [id] => 57115 [title] => Control.png [filename] => Control-115.png [filesize] => 133096 [url] => https://www.nextafter.com/wp-content/uploads/Control-115.png [link] => https://www.nextafter.com/experiments/how-leading-a-membership-ask-email-with-the-problem-for-existing-members-affects-click-through-rate/attachment/control-png-6618/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-6618 [status] => inherit [uploaded_to] => 57084 [date] => 2024-01-31 16:28:41 [modified] => 2024-01-31 16:28:41 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 1342 [height] => 795 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-115-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-115-300x178.png [medium-width] => 300 [medium-height] => 178 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-115-768x455.png [medium_large-width] => 640 [medium_large-height] => 379 [large] => https://www.nextafter.com/wp-content/uploads/Control-115-1024x607.png [large-width] => 640 [large-height] => 379 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-115.png [1536x1536-width] => 1342 [1536x1536-height] => 795 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-115.png [2048x2048-width] => 1342 [2048x2048-height] => 795 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-115.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 415 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-115-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-115-50x30.png [admin-width] => 50 [admin-height] => 30 [full] => https://www.nextafter.com/wp-content/uploads/Control-115.png [full-width] => 1342 [full-height] => 795 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-115-372x220.png [thumb_372-width] => 372 [thumb_372-height] => 220 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-115-980x581.png [single_post_thumbnail-width] => 980 [single_post_thumbnail-height] => 581 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-115-32x19.png [thumb_32_32-width] => 32 [thumb_32_32-height] => 19 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-115-48x28.png [thumb_48_48-width] => 48 [thumb_48_48-height] => 28 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-115-64x38.png [thumb_64_64-width] => 64 [thumb_64_64-height] => 38 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-115-128x76.png [thumb_128_128-width] => 128 [thumb_128_128-height] => 76 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-115-113x67.png [thumb_122_67-width] => 113 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-115-153x91.png [thumb_153_153-width] => 153 [thumb_153_153-height] => 91 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-115-130x77.png [thumb_130_163-width] => 130 [thumb_130_163-height] => 77 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-115-232x137.png [thumb_232_177-width] => 232 [thumb_232_177-height] => 137 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-115-544x322.png [thumb_272_325-width] => 544 [thumb_272_325-height] => 322 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-115-271x161.png [thumb_271_346-width] => 271 [thumb_271_346-height] => 161 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-115-271x161.png [thumb_271_271-width] => 271 [thumb_271_271-height] => 161 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-115-272x161.png [thumb_272_172-width] => 272 [thumb_272_172-height] => 161 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-115-372x220.png [thumb_372_461-width] => 372 [thumb_372_461-height] => 220 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-115-372x220.png [thumb_372_500-width] => 372 [thumb_372_500-height] => 220 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-115-385x228.png [thumb_385_270-width] => 385 [thumb_385_270-height] => 228 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-115-407x241.png [thumb_416_241-width] => 407 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-115-448x265.png [thumb_448_410-width] => 448 [thumb_448_410-height] => 265 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-115-896x531.png [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 531 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-115-574x340.png [thumb_574_382-width] => 574 [thumb_574_382-height] => 340 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-115-667x395.png [thumb_776_395-width] => 667 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-115.png [gform-image-choice-sm-width] => 300 [gform-image-choice-sm-height] => 178 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-115.png [gform-image-choice-md-width] => 400 [gform-image-choice-md-height] => 237 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-115.png [gform-image-choice-lg-width] => 600 [gform-image-choice-lg-height] => 355 ) ) [1] => Array ( [ID] => 57116 [id] => 57116 [title] => Treatment_#1.png [filename] => Treatment_1-66.png [filesize] => 150951 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [link] => https://www.nextafter.com/experiments/how-leading-a-membership-ask-email-with-the-problem-for-existing-members-affects-click-through-rate/attachment/treatment_1-png-2115/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-2115 [status] => inherit [uploaded_to] => 57084 [date] => 2024-01-31 16:28:48 [modified] => 2024-01-31 16:28:48 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 1340 [height] => 880 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-300x197.png [medium-width] => 300 [medium-height] => 197 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-768x504.png [medium_large-width] => 640 [medium_large-height] => 420 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-1024x672.png [large-width] => 640 [large-height] => 420 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [1536x1536-width] => 1340 [1536x1536-height] => 880 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [2048x2048-width] => 1340 [2048x2048-height] => 880 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 460 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-50x33.png [admin-width] => 50 [admin-height] => 33 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [full-width] => 1340 [full-height] => 880 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-372x244.png [thumb_372-width] => 372 [thumb_372-height] => 244 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-952x625.png [single_post_thumbnail-width] => 952 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-32x21.png [thumb_32_32-width] => 32 [thumb_32_32-height] => 21 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-48x32.png [thumb_48_48-width] => 48 [thumb_48_48-height] => 32 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-64x42.png [thumb_64_64-width] => 64 [thumb_64_64-height] => 42 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-128x84.png [thumb_128_128-width] => 128 [thumb_128_128-height] => 84 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-102x67.png [thumb_122_67-width] => 102 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-153x100.png [thumb_153_153-width] => 153 [thumb_153_153-height] => 100 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-130x85.png [thumb_130_163-width] => 130 [thumb_130_163-height] => 85 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-232x152.png [thumb_232_177-width] => 232 [thumb_232_177-height] => 152 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-544x357.png [thumb_272_325-width] => 544 [thumb_272_325-height] => 357 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-271x178.png [thumb_271_346-width] => 271 [thumb_271_346-height] => 178 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-271x178.png [thumb_271_271-width] => 271 [thumb_271_271-height] => 178 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-262x172.png [thumb_272_172-width] => 262 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-372x244.png [thumb_372_461-width] => 372 [thumb_372_461-height] => 244 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-372x244.png [thumb_372_500-width] => 372 [thumb_372_500-height] => 244 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-385x253.png [thumb_385_270-width] => 385 [thumb_385_270-height] => 253 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-367x241.png [thumb_416_241-width] => 367 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-448x294.png [thumb_448_410-width] => 448 [thumb_448_410-height] => 294 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-896x588.png [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 588 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-574x377.png [thumb_574_382-width] => 574 [thumb_574_382-height] => 377 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-601x395.png [thumb_776_395-width] => 601 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [gform-image-choice-sm-width] => 300 [gform-image-choice-sm-height] => 197 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [gform-image-choice-md-width] => 400 [gform-image-choice-md-height] => 263 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [gform-image-choice-lg-width] => 600 [gform-image-choice-lg-height] => 394 ) ) )
T1: Treatment #1
Array ( [0] => Array ( [ID] => 57115 [id] => 57115 [title] => Control.png [filename] => Control-115.png [filesize] => 133096 [url] => https://www.nextafter.com/wp-content/uploads/Control-115.png [link] => https://www.nextafter.com/experiments/how-leading-a-membership-ask-email-with-the-problem-for-existing-members-affects-click-through-rate/attachment/control-png-6618/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-6618 [status] => inherit [uploaded_to] => 57084 [date] => 2024-01-31 16:28:41 [modified] => 2024-01-31 16:28:41 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 1342 [height] => 795 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-115-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-115-300x178.png [medium-width] => 300 [medium-height] => 178 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-115-768x455.png [medium_large-width] => 640 [medium_large-height] => 379 [large] => https://www.nextafter.com/wp-content/uploads/Control-115-1024x607.png [large-width] => 640 [large-height] => 379 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-115.png [1536x1536-width] => 1342 [1536x1536-height] => 795 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-115.png [2048x2048-width] => 1342 [2048x2048-height] => 795 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-115.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 415 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-115-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-115-50x30.png [admin-width] => 50 [admin-height] => 30 [full] => https://www.nextafter.com/wp-content/uploads/Control-115.png [full-width] => 1342 [full-height] => 795 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-115-372x220.png [thumb_372-width] => 372 [thumb_372-height] => 220 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-115-980x581.png [single_post_thumbnail-width] => 980 [single_post_thumbnail-height] => 581 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-115-32x19.png [thumb_32_32-width] => 32 [thumb_32_32-height] => 19 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-115-48x28.png [thumb_48_48-width] => 48 [thumb_48_48-height] => 28 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-115-64x38.png [thumb_64_64-width] => 64 [thumb_64_64-height] => 38 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-115-128x76.png [thumb_128_128-width] => 128 [thumb_128_128-height] => 76 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-115-113x67.png [thumb_122_67-width] => 113 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-115-153x91.png [thumb_153_153-width] => 153 [thumb_153_153-height] => 91 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-115-130x77.png [thumb_130_163-width] => 130 [thumb_130_163-height] => 77 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-115-232x137.png [thumb_232_177-width] => 232 [thumb_232_177-height] => 137 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-115-544x322.png [thumb_272_325-width] => 544 [thumb_272_325-height] => 322 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-115-271x161.png [thumb_271_346-width] => 271 [thumb_271_346-height] => 161 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-115-271x161.png [thumb_271_271-width] => 271 [thumb_271_271-height] => 161 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-115-272x161.png [thumb_272_172-width] => 272 [thumb_272_172-height] => 161 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-115-372x220.png [thumb_372_461-width] => 372 [thumb_372_461-height] => 220 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-115-372x220.png [thumb_372_500-width] => 372 [thumb_372_500-height] => 220 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-115-385x228.png [thumb_385_270-width] => 385 [thumb_385_270-height] => 228 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-115-407x241.png [thumb_416_241-width] => 407 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-115-448x265.png [thumb_448_410-width] => 448 [thumb_448_410-height] => 265 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-115-896x531.png [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 531 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-115-574x340.png [thumb_574_382-width] => 574 [thumb_574_382-height] => 340 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-115-667x395.png [thumb_776_395-width] => 667 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-115.png [gform-image-choice-sm-width] => 300 [gform-image-choice-sm-height] => 178 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-115.png [gform-image-choice-md-width] => 400 [gform-image-choice-md-height] => 237 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-115.png [gform-image-choice-lg-width] => 600 [gform-image-choice-lg-height] => 355 ) ) [1] => Array ( [ID] => 57116 [id] => 57116 [title] => Treatment_#1.png [filename] => Treatment_1-66.png [filesize] => 150951 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [link] => https://www.nextafter.com/experiments/how-leading-a-membership-ask-email-with-the-problem-for-existing-members-affects-click-through-rate/attachment/treatment_1-png-2115/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-2115 [status] => inherit [uploaded_to] => 57084 [date] => 2024-01-31 16:28:48 [modified] => 2024-01-31 16:28:48 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 1340 [height] => 880 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-300x197.png [medium-width] => 300 [medium-height] => 197 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-768x504.png [medium_large-width] => 640 [medium_large-height] => 420 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-1024x672.png [large-width] => 640 [large-height] => 420 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [1536x1536-width] => 1340 [1536x1536-height] => 880 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [2048x2048-width] => 1340 [2048x2048-height] => 880 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 460 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-50x33.png [admin-width] => 50 [admin-height] => 33 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [full-width] => 1340 [full-height] => 880 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-372x244.png [thumb_372-width] => 372 [thumb_372-height] => 244 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-952x625.png [single_post_thumbnail-width] => 952 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-32x21.png [thumb_32_32-width] => 32 [thumb_32_32-height] => 21 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-48x32.png [thumb_48_48-width] => 48 [thumb_48_48-height] => 32 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-64x42.png [thumb_64_64-width] => 64 [thumb_64_64-height] => 42 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-128x84.png [thumb_128_128-width] => 128 [thumb_128_128-height] => 84 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-102x67.png [thumb_122_67-width] => 102 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-153x100.png [thumb_153_153-width] => 153 [thumb_153_153-height] => 100 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-130x85.png [thumb_130_163-width] => 130 [thumb_130_163-height] => 85 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-232x152.png [thumb_232_177-width] => 232 [thumb_232_177-height] => 152 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-544x357.png [thumb_272_325-width] => 544 [thumb_272_325-height] => 357 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-271x178.png [thumb_271_346-width] => 271 [thumb_271_346-height] => 178 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-271x178.png [thumb_271_271-width] => 271 [thumb_271_271-height] => 178 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-262x172.png [thumb_272_172-width] => 262 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-372x244.png [thumb_372_461-width] => 372 [thumb_372_461-height] => 244 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-372x244.png [thumb_372_500-width] => 372 [thumb_372_500-height] => 244 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-385x253.png [thumb_385_270-width] => 385 [thumb_385_270-height] => 253 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-367x241.png [thumb_416_241-width] => 367 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-448x294.png [thumb_448_410-width] => 448 [thumb_448_410-height] => 294 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-896x588.png [thumb_448_410_retina-width] => 896 [thumb_448_410_retina-height] => 588 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-574x377.png [thumb_574_382-width] => 574 [thumb_574_382-height] => 377 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66-601x395.png [thumb_776_395-width] => 601 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [gform-image-choice-sm-width] => 300 [gform-image-choice-sm-height] => 197 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [gform-image-choice-md-width] => 400 [gform-image-choice-md-height] => 263 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-66.png [gform-image-choice-lg-width] => 600 [gform-image-choice-lg-height] => 394 ) ) )

Results

  Treatment Name Click Rate Relative Difference Confidence
C: Control 1.5%
T1: Treatment #1 0.66% -55.8% 100.0%

This experiment has a required sample size of 1,184 in order to be valid. Since the experiment had a total sample size of 49,216, and the level of confidence is above 95% the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

    55.8% decrease in traffic
Ă— 0% increase in conversion rate
Ă— 0% increase in average gift

Key Learnings

The key learning from this experiment is that leading an email with a question that poses the problem did not increase email click-through rate for existing members to renew their membership with The Heritage Foundation. In fact, the treatment group showed a 55.8% decrease in clicks compared to the control group. In addition, the treatment group saw a 23.5% decrease in donor conversion with a 97.6% level of confidence. These findings suggest that existing members may not respond well to a problem-focused approach when it comes to renewing their membership. It is possible that they already have a strong understanding of the problem and are more interested in the direct ask to renew their support. In future experimentation, it may be more effective to lead the email with a direct ask to renew membership rather than focusing on the problem. Additionally, it could be beneficial to test other strategies such as emphasizing the impact of their support or offering a specific benefit or incentive for renewing. These approaches may help to engage existing members and encourage them to click through and renew their membership.

Experiment Documented by NextAfter

Question about experiment #169193

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.