let us help
Subscribe
Hoover Institution

How a shorter and more conversational housefile email impacted donations and email acquisitions

Experiment ID: #196142

Hoover Institution

Experiment Summary

Ended On: 03/26/2025

This experiment was conducted for the Hoover Institution, a public policy think tank dedicated to promoting ideas for a free society. Hoover recirculates their content offers by sending out an email to their Donors and Nondonors every Thursday offering a free eBook or course. On this Thursday, Hoover sent out a NEW offer, “Federal Entitlement Programs”. This specific email test was to their donor audience.

These were some key differences between the copy:

  1. Conciseness – The treatment is shorter and more direct, removing the detailed breakdown of course modules.
  2. Engagement – The treatment uses a more conversational tone with rhetorical questions (“Why does this matter?”) to draw the reader in.
  3. Clarity & Readability – The treatment simplifies complex ideas, avoiding dense phrasing like “glacially slow economic growth”.
  4. Urgency & Impact – The treatment highlights the real-world consequences of entitlement spending earlier and more clearly.
  5. Call to Action (CTA) – The treatment places the course link in a more prominent, action-driven sentence.

Research Question

We believe that having a shorter and more conversational emaill for Hoover’s donor audience will achieve an increase in donations because it will improve engagement and enhance clarity.

Design

C: Control
Array ( [0] => Array ( [ID] => 71106 [id] => 71106 [title] => Control.png [filename] => Control-96.png [filesize] => 112879 [url] => https://www.nextafter.com/wp-content/uploads/Control-96.png [link] => https://www.nextafter.com/experiments/how-a-shorter-and-more-conversational-housefile-email-impacted-donations-and-email-acquisitions/attachment/control-png-10106/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-10106 [status] => inherit [uploaded_to] => 70864 [date] => 2025-04-02 18:23:21 [modified] => 2025-04-02 18:23:21 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 407 [height] => 939 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-96-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-96-130x300.png [medium-width] => 130 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-96.png [medium_large-width] => 407 [medium_large-height] => 939 [large] => https://www.nextafter.com/wp-content/uploads/Control-96.png [large-width] => 407 [large-height] => 939 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-96.png [1536x1536-width] => 407 [1536x1536-height] => 939 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-96.png [2048x2048-width] => 407 [2048x2048-height] => 939 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-96-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-96-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-96-22x50.png [admin-width] => 22 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Control-96.png [full-width] => 407 [full-height] => 939 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-96-372x858.png [thumb_372-width] => 372 [thumb_372-height] => 858 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-96-271x625.png [single_post_thumbnail-width] => 271 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-96-14x32.png [thumb_32_32-width] => 14 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-96-21x48.png [thumb_48_48-width] => 21 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-96-28x64.png [thumb_64_64-width] => 28 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-96-55x128.png [thumb_128_128-width] => 55 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-96-29x67.png [thumb_122_67-width] => 29 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-96-66x153.png [thumb_153_153-width] => 66 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-96-71x163.png [thumb_130_163-width] => 71 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-96-77x177.png [thumb_232_177-width] => 77 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-96-282x650.png [thumb_272_325-width] => 282 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-96-150x346.png [thumb_271_346-width] => 150 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-96-117x271.png [thumb_271_271-width] => 117 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-96-75x172.png [thumb_272_172-width] => 75 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-96-200x461.png [thumb_372_461-width] => 200 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-96-217x500.png [thumb_372_500-width] => 217 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-96-117x270.png [thumb_385_270-width] => 117 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-96-104x241.png [thumb_416_241-width] => 104 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-96-178x410.png [thumb_448_410-width] => 178 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-96-355x820.png [thumb_448_410_retina-width] => 355 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-96-166x382.png [thumb_574_382-width] => 166 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-96-171x395.png [thumb_776_395-width] => 171 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-96.png [gform-image-choice-sm-width] => 130 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-96.png [gform-image-choice-md-width] => 173 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-96.png [gform-image-choice-lg-width] => 260 [gform-image-choice-lg-height] => 600 ) ) [1] => Array ( [ID] => 71107 [id] => 71107 [title] => Treatment_#1.png [filename] => Treatment_1-86.png [filesize] => 105664 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [link] => https://www.nextafter.com/experiments/how-a-shorter-and-more-conversational-housefile-email-impacted-donations-and-email-acquisitions/attachment/treatment_1-png-3697/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-3697 [status] => inherit [uploaded_to] => 70864 [date] => 2025-04-02 18:23:27 [modified] => 2025-04-02 18:23:27 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 456 [height] => 881 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-155x300.png [medium-width] => 155 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [medium_large-width] => 456 [medium_large-height] => 881 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [large-width] => 456 [large-height] => 881 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [1536x1536-width] => 456 [1536x1536-height] => 881 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [2048x2048-width] => 456 [2048x2048-height] => 881 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-26x50.png [admin-width] => 26 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [full-width] => 456 [full-height] => 881 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-372x719.png [thumb_372-width] => 372 [thumb_372-height] => 719 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-323x625.png [single_post_thumbnail-width] => 323 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-17x32.png [thumb_32_32-width] => 17 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-25x48.png [thumb_48_48-width] => 25 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-33x64.png [thumb_64_64-width] => 33 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-66x128.png [thumb_128_128-width] => 66 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-35x67.png [thumb_122_67-width] => 35 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-79x153.png [thumb_153_153-width] => 79 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-84x163.png [thumb_130_163-width] => 84 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-92x177.png [thumb_232_177-width] => 92 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-336x650.png [thumb_272_325-width] => 336 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-179x346.png [thumb_271_346-width] => 179 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-140x271.png [thumb_271_271-width] => 140 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-89x172.png [thumb_272_172-width] => 89 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-239x461.png [thumb_372_461-width] => 239 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-259x500.png [thumb_372_500-width] => 259 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-140x270.png [thumb_385_270-width] => 140 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-125x241.png [thumb_416_241-width] => 125 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-212x410.png [thumb_448_410-width] => 212 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-424x820.png [thumb_448_410_retina-width] => 424 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-198x382.png [thumb_574_382-width] => 198 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-204x395.png [thumb_776_395-width] => 204 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [gform-image-choice-sm-width] => 155 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [gform-image-choice-md-width] => 207 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [gform-image-choice-lg-width] => 311 [gform-image-choice-lg-height] => 600 ) ) )
T1: Treatment #1
Array ( [0] => Array ( [ID] => 71106 [id] => 71106 [title] => Control.png [filename] => Control-96.png [filesize] => 112879 [url] => https://www.nextafter.com/wp-content/uploads/Control-96.png [link] => https://www.nextafter.com/experiments/how-a-shorter-and-more-conversational-housefile-email-impacted-donations-and-email-acquisitions/attachment/control-png-10106/ [alt] => [author] => 0 [description] => [caption] => [name] => control-png-10106 [status] => inherit [uploaded_to] => 70864 [date] => 2025-04-02 18:23:21 [modified] => 2025-04-02 18:23:21 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 407 [height] => 939 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-96-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Control-96-130x300.png [medium-width] => 130 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Control-96.png [medium_large-width] => 407 [medium_large-height] => 939 [large] => https://www.nextafter.com/wp-content/uploads/Control-96.png [large-width] => 407 [large-height] => 939 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Control-96.png [1536x1536-width] => 407 [1536x1536-height] => 939 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Control-96.png [2048x2048-width] => 407 [2048x2048-height] => 939 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Control-96-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-96-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Control-96-22x50.png [admin-width] => 22 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Control-96.png [full-width] => 407 [full-height] => 939 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Control-96-372x858.png [thumb_372-width] => 372 [thumb_372-height] => 858 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Control-96-271x625.png [single_post_thumbnail-width] => 271 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Control-96-14x32.png [thumb_32_32-width] => 14 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Control-96-21x48.png [thumb_48_48-width] => 21 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Control-96-28x64.png [thumb_64_64-width] => 28 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Control-96-55x128.png [thumb_128_128-width] => 55 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Control-96-29x67.png [thumb_122_67-width] => 29 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Control-96-66x153.png [thumb_153_153-width] => 66 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Control-96-71x163.png [thumb_130_163-width] => 71 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Control-96-77x177.png [thumb_232_177-width] => 77 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Control-96-282x650.png [thumb_272_325-width] => 282 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Control-96-150x346.png [thumb_271_346-width] => 150 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Control-96-117x271.png [thumb_271_271-width] => 117 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Control-96-75x172.png [thumb_272_172-width] => 75 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Control-96-200x461.png [thumb_372_461-width] => 200 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Control-96-217x500.png [thumb_372_500-width] => 217 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Control-96-117x270.png [thumb_385_270-width] => 117 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Control-96-104x241.png [thumb_416_241-width] => 104 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Control-96-178x410.png [thumb_448_410-width] => 178 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Control-96-355x820.png [thumb_448_410_retina-width] => 355 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Control-96-166x382.png [thumb_574_382-width] => 166 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Control-96-171x395.png [thumb_776_395-width] => 171 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Control-96.png [gform-image-choice-sm-width] => 130 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Control-96.png [gform-image-choice-md-width] => 173 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Control-96.png [gform-image-choice-lg-width] => 260 [gform-image-choice-lg-height] => 600 ) ) [1] => Array ( [ID] => 71107 [id] => 71107 [title] => Treatment_#1.png [filename] => Treatment_1-86.png [filesize] => 105664 [url] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [link] => https://www.nextafter.com/experiments/how-a-shorter-and-more-conversational-housefile-email-impacted-donations-and-email-acquisitions/attachment/treatment_1-png-3697/ [alt] => [author] => 0 [description] => [caption] => [name] => treatment_1-png-3697 [status] => inherit [uploaded_to] => 70864 [date] => 2025-04-02 18:23:27 [modified] => 2025-04-02 18:23:27 [menu_order] => 0 [mime_type] => image/png [type] => image [subtype] => png [icon] => https://www.nextafter.com/wp-includes/images/media/default.png [width] => 456 [height] => 881 [sizes] => Array ( [thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-150x150.png [thumbnail-width] => 150 [thumbnail-height] => 150 [medium] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-155x300.png [medium-width] => 155 [medium-height] => 300 [medium_large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [medium_large-width] => 456 [medium_large-height] => 881 [large] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [large-width] => 456 [large-height] => 881 [1536x1536] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [1536x1536-width] => 456 [1536x1536-height] => 881 [2048x2048] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [2048x2048-width] => 456 [2048x2048-height] => 881 [tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-700x700.png [tp-image-grid-width] => 700 [tp-image-grid-height] => 700 [post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-150x150.png [post-thumbnail-width] => 150 [post-thumbnail-height] => 150 [admin] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-26x50.png [admin-width] => 26 [admin-height] => 50 [full] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [full-width] => 456 [full-height] => 881 [thumb_372] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-372x719.png [thumb_372-width] => 372 [thumb_372-height] => 719 [single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-323x625.png [single_post_thumbnail-width] => 323 [single_post_thumbnail-height] => 625 [thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-17x32.png [thumb_32_32-width] => 17 [thumb_32_32-height] => 32 [thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-25x48.png [thumb_48_48-width] => 25 [thumb_48_48-height] => 48 [thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-33x64.png [thumb_64_64-width] => 33 [thumb_64_64-height] => 64 [thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-66x128.png [thumb_128_128-width] => 66 [thumb_128_128-height] => 128 [thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-35x67.png [thumb_122_67-width] => 35 [thumb_122_67-height] => 67 [thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-79x153.png [thumb_153_153-width] => 79 [thumb_153_153-height] => 153 [thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-84x163.png [thumb_130_163-width] => 84 [thumb_130_163-height] => 163 [thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-92x177.png [thumb_232_177-width] => 92 [thumb_232_177-height] => 177 [thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-336x650.png [thumb_272_325-width] => 336 [thumb_272_325-height] => 650 [thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-179x346.png [thumb_271_346-width] => 179 [thumb_271_346-height] => 346 [thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-140x271.png [thumb_271_271-width] => 140 [thumb_271_271-height] => 271 [thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-89x172.png [thumb_272_172-width] => 89 [thumb_272_172-height] => 172 [thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-239x461.png [thumb_372_461-width] => 239 [thumb_372_461-height] => 461 [thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-259x500.png [thumb_372_500-width] => 259 [thumb_372_500-height] => 500 [thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-140x270.png [thumb_385_270-width] => 140 [thumb_385_270-height] => 270 [thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-125x241.png [thumb_416_241-width] => 125 [thumb_416_241-height] => 241 [thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-212x410.png [thumb_448_410-width] => 212 [thumb_448_410-height] => 410 [thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-424x820.png [thumb_448_410_retina-width] => 424 [thumb_448_410_retina-height] => 820 [thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-198x382.png [thumb_574_382-width] => 198 [thumb_574_382-height] => 382 [thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86-204x395.png [thumb_776_395-width] => 204 [thumb_776_395-height] => 395 [gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [gform-image-choice-sm-width] => 155 [gform-image-choice-sm-height] => 300 [gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [gform-image-choice-md-width] => 207 [gform-image-choice-md-height] => 400 [gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Treatment_1-86.png [gform-image-choice-lg-width] => 311 [gform-image-choice-lg-height] => 600 ) ) )

Results

  Treatment Name Conv. Rate Relative Difference Confidence
C: Control 0.03%
T1: Treatment #1 0.02% -35.6% 92.8%

This experiment has a required sample size of 151,472 in order to be valid. Since the experiment had a total sample size of 255,219, and the level of confidence is not above 95% the experiment results are not valid.

Key Learnings

Key Learnings:

Shorter Emails Didn’t Increase Donations

  • The treatment email, while more concise and conversational, resulted in fewer donations (27 vs. 42) and fewer acquired emails (138 vs. 209) than the control.
  • Next Step: Analyze why donations dropped—did the shorter format remove persuasive elements that resonated with donors?


Audience May Prefer More Detail

  • The control email provided a structured breakdown of the course, which may have been more compelling for Hoover’s donor audience.
  • Next Step: Test a hybrid email that keeps the engaging tone of the treatment while reintroducing key persuasive elements from the control.


CTA Placement and Wording Could Be Optimized

  • The placement and clarity of the donation ask might have impacted conversions.
  • Next Step: Experiment with stronger, more direct CTA wording and placement to drive more action.


Consider Multi-Touch Campaigns

  • A single email might not be enough to drive engagement and donations.
  • Next Step: Explore follow-up emails, retargeting, or landing pages to reinforce the message and increase conversions.


Experiment Documented by NextAfter

Question about experiment #196142

If you have any questions about this experiment or would like additional details not discussed above, please feel free to contact them directly.