How “human” fundraising subject lines affect clickthrough rate
Experiment ID: #3729
Harvest Ministries
Experiment Summary
Timeframe: 04/13/2016 - 04/19/2016
Harvest Ministries was sending their mid-month email, which was a highly transactional, premium-focused offer. The offer featured a book that was relevant to Mother’s Day, which was coming up in just a few weeks. The treatment subject line was appropriately transactional: “Order Today to Receive for Mother’s Day”. We had been running some experiments with “humanized” copy — that is, copy that a normal person would write, rather than copy that is explicitly “marketing”. We hypothesized that a more “human” subject line would increase open rate, but we also wanted to measure the effect on clickthrough rate as well, since the two do not operate independently.
At the same time as the subject line test, we were running an email copy and design test as well. The control (which was matched to the control subject line) incentivized the click, a strategy that has shown to attract more clicks, but often less motivated traffic. The treatment made a more direct ask, which often results in fewer clicks but more gifts.
Research Question
Will “humanized copy” increase open rate and increase motivation to click?
Design
C: Order today
Array
(
[0] => Array
(
[ID] => 12634
[id] => 12634
[title] => Screen-Shot-2016-04-19-at-9.12.16-AM.png
[filename] => Screen-Shot-2016-04-19-at-9.12.16-AM.png
[filesize] => 0
[url] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[link] => https://www.nextafter.com/experiments/how-human-fundraising-subject-lines-affect-clickthrough-rate/attachment/screen-shot-2016-04-19-at-9-12-16-am-png/
[alt] =>
[author] => 6
[description] =>
[caption] =>
[name] => screen-shot-2016-04-19-at-9-12-16-am-png
[status] => inherit
[uploaded_to] => 12633
[date] => 2021-02-25 02:08:16
[modified] => 2021-02-25 02:08:16
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 546
[height] => 608
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-269x300.png
[medium-width] => 269
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[medium_large-width] => 546
[medium_large-height] => 608
[large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[large-width] => 546
[large-height] => 608
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[1536x1536-width] => 546
[1536x1536-height] => 608
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[2048x2048-width] => 546
[2048x2048-height] => 608
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[tp-image-grid-width] => 546
[tp-image-grid-height] => 608
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-45x50.png
[admin-width] => 45
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[full-width] => 546
[full-height] => 608
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-372x414.png
[thumb_372-width] => 372
[thumb_372-height] => 414
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[single_post_thumbnail-width] => 546
[single_post_thumbnail-height] => 608
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-29x32.png
[thumb_32_32-width] => 29
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-43x48.png
[thumb_48_48-width] => 43
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-57x64.png
[thumb_64_64-width] => 57
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-115x128.png
[thumb_128_128-width] => 115
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-60x67.png
[thumb_122_67-width] => 60
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-137x153.png
[thumb_153_153-width] => 137
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-130x145.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 145
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-159x177.png
[thumb_232_177-width] => 159
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-544x606.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 606
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-271x302.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 302
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-243x271.png
[thumb_271_271-width] => 243
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-154x172.png
[thumb_272_172-width] => 154
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-372x414.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 414
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-372x414.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 414
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-242x270.png
[thumb_385_270-width] => 242
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-216x241.png
[thumb_416_241-width] => 216
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-368x410.png
[thumb_448_410-width] => 368
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[thumb_448_410_retina-width] => 546
[thumb_448_410_retina-height] => 608
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-343x382.png
[thumb_574_382-width] => 343
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-355x395.png
[thumb_776_395-width] => 355
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[gform-image-choice-sm-width] => 269
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[gform-image-choice-md-width] => 359
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[gform-image-choice-lg-width] => 539
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 12635
[id] => 12635
[title] => Screen-Shot-2016-04-19-at-9.12.48-AM.png
[filename] => Screen-Shot-2016-04-19-at-9.12.48-AM.png
[filesize] => 0
[url] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[link] => https://www.nextafter.com/experiments/how-human-fundraising-subject-lines-affect-clickthrough-rate/attachment/screen-shot-2016-04-19-at-9-12-48-am-png/
[alt] =>
[author] => 6
[description] =>
[caption] =>
[name] => screen-shot-2016-04-19-at-9-12-48-am-png
[status] => inherit
[uploaded_to] => 12633
[date] => 2021-02-25 02:10:24
[modified] => 2021-02-25 02:10:24
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 544
[height] => 588
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-278x300.png
[medium-width] => 278
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[medium_large-width] => 544
[medium_large-height] => 588
[large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[large-width] => 544
[large-height] => 588
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[1536x1536-width] => 544
[1536x1536-height] => 588
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[2048x2048-width] => 544
[2048x2048-height] => 588
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[tp-image-grid-width] => 544
[tp-image-grid-height] => 588
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-46x50.png
[admin-width] => 46
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[full-width] => 544
[full-height] => 588
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-372x402.png
[thumb_372-width] => 372
[thumb_372-height] => 402
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[single_post_thumbnail-width] => 544
[single_post_thumbnail-height] => 588
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-30x32.png
[thumb_32_32-width] => 30
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-44x48.png
[thumb_48_48-width] => 44
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-59x64.png
[thumb_64_64-width] => 59
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-118x128.png
[thumb_128_128-width] => 118
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-62x67.png
[thumb_122_67-width] => 62
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-142x153.png
[thumb_153_153-width] => 142
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-130x141.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 141
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-164x177.png
[thumb_232_177-width] => 164
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 588
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-271x293.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 293
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-251x271.png
[thumb_271_271-width] => 251
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-159x172.png
[thumb_272_172-width] => 159
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-372x402.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 402
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-372x402.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 402
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-250x270.png
[thumb_385_270-width] => 250
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-223x241.png
[thumb_416_241-width] => 223
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-379x410.png
[thumb_448_410-width] => 379
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[thumb_448_410_retina-width] => 544
[thumb_448_410_retina-height] => 588
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-353x382.png
[thumb_574_382-width] => 353
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-365x395.png
[thumb_776_395-width] => 365
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[gform-image-choice-sm-width] => 278
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[gform-image-choice-md-width] => 370
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[gform-image-choice-lg-width] => 544
[gform-image-choice-lg-height] => 588
)
)
)
T1: Humanized Copy
Array
(
[0] => Array
(
[ID] => 12634
[id] => 12634
[title] => Screen-Shot-2016-04-19-at-9.12.16-AM.png
[filename] => Screen-Shot-2016-04-19-at-9.12.16-AM.png
[filesize] => 0
[url] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[link] => https://www.nextafter.com/experiments/how-human-fundraising-subject-lines-affect-clickthrough-rate/attachment/screen-shot-2016-04-19-at-9-12-16-am-png/
[alt] =>
[author] => 6
[description] =>
[caption] =>
[name] => screen-shot-2016-04-19-at-9-12-16-am-png
[status] => inherit
[uploaded_to] => 12633
[date] => 2021-02-25 02:08:16
[modified] => 2021-02-25 02:08:16
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 546
[height] => 608
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-269x300.png
[medium-width] => 269
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[medium_large-width] => 546
[medium_large-height] => 608
[large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[large-width] => 546
[large-height] => 608
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[1536x1536-width] => 546
[1536x1536-height] => 608
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[2048x2048-width] => 546
[2048x2048-height] => 608
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[tp-image-grid-width] => 546
[tp-image-grid-height] => 608
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-45x50.png
[admin-width] => 45
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[full-width] => 546
[full-height] => 608
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-372x414.png
[thumb_372-width] => 372
[thumb_372-height] => 414
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[single_post_thumbnail-width] => 546
[single_post_thumbnail-height] => 608
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-29x32.png
[thumb_32_32-width] => 29
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-43x48.png
[thumb_48_48-width] => 43
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-57x64.png
[thumb_64_64-width] => 57
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-115x128.png
[thumb_128_128-width] => 115
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-60x67.png
[thumb_122_67-width] => 60
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-137x153.png
[thumb_153_153-width] => 137
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-130x145.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 145
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-159x177.png
[thumb_232_177-width] => 159
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-544x606.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 606
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-271x302.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 302
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-243x271.png
[thumb_271_271-width] => 243
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-154x172.png
[thumb_272_172-width] => 154
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-372x414.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 414
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-372x414.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 414
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-242x270.png
[thumb_385_270-width] => 242
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-216x241.png
[thumb_416_241-width] => 216
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-368x410.png
[thumb_448_410-width] => 368
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[thumb_448_410_retina-width] => 546
[thumb_448_410_retina-height] => 608
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-343x382.png
[thumb_574_382-width] => 343
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM-355x395.png
[thumb_776_395-width] => 355
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[gform-image-choice-sm-width] => 269
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[gform-image-choice-md-width] => 359
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.16-AM.png
[gform-image-choice-lg-width] => 539
[gform-image-choice-lg-height] => 600
)
)
[1] => Array
(
[ID] => 12635
[id] => 12635
[title] => Screen-Shot-2016-04-19-at-9.12.48-AM.png
[filename] => Screen-Shot-2016-04-19-at-9.12.48-AM.png
[filesize] => 0
[url] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[link] => https://www.nextafter.com/experiments/how-human-fundraising-subject-lines-affect-clickthrough-rate/attachment/screen-shot-2016-04-19-at-9-12-48-am-png/
[alt] =>
[author] => 6
[description] =>
[caption] =>
[name] => screen-shot-2016-04-19-at-9-12-48-am-png
[status] => inherit
[uploaded_to] => 12633
[date] => 2021-02-25 02:10:24
[modified] => 2021-02-25 02:10:24
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 544
[height] => 588
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-278x300.png
[medium-width] => 278
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[medium_large-width] => 544
[medium_large-height] => 588
[large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[large-width] => 544
[large-height] => 588
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[1536x1536-width] => 544
[1536x1536-height] => 588
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[2048x2048-width] => 544
[2048x2048-height] => 588
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[tp-image-grid-width] => 544
[tp-image-grid-height] => 588
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-46x50.png
[admin-width] => 46
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[full-width] => 544
[full-height] => 588
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-372x402.png
[thumb_372-width] => 372
[thumb_372-height] => 402
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[single_post_thumbnail-width] => 544
[single_post_thumbnail-height] => 588
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-30x32.png
[thumb_32_32-width] => 30
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-44x48.png
[thumb_48_48-width] => 44
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-59x64.png
[thumb_64_64-width] => 59
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-118x128.png
[thumb_128_128-width] => 118
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-62x67.png
[thumb_122_67-width] => 62
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-142x153.png
[thumb_153_153-width] => 142
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-130x141.png
[thumb_130_163-width] => 130
[thumb_130_163-height] => 141
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-164x177.png
[thumb_232_177-width] => 164
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[thumb_272_325-width] => 544
[thumb_272_325-height] => 588
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-271x293.png
[thumb_271_346-width] => 271
[thumb_271_346-height] => 293
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-251x271.png
[thumb_271_271-width] => 251
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-159x172.png
[thumb_272_172-width] => 159
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-372x402.png
[thumb_372_461-width] => 372
[thumb_372_461-height] => 402
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-372x402.png
[thumb_372_500-width] => 372
[thumb_372_500-height] => 402
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-250x270.png
[thumb_385_270-width] => 250
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-223x241.png
[thumb_416_241-width] => 223
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-379x410.png
[thumb_448_410-width] => 379
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[thumb_448_410_retina-width] => 544
[thumb_448_410_retina-height] => 588
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-353x382.png
[thumb_574_382-width] => 353
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM-365x395.png
[thumb_776_395-width] => 365
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[gform-image-choice-sm-width] => 278
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[gform-image-choice-md-width] => 370
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2016-04-19-at-9.12.48-AM.png
[gform-image-choice-lg-width] => 544
[gform-image-choice-lg-height] => 588
)
)
)
Results
| Treatment Name | Click Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Order today | 2.1% | ||
| T1: | Humanized Copy | 0.87% | -58.1% | 100.0% |
This experiment has a required sample size of 766 in order to be valid. Since the experiment had a total sample size of 81,181, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
58.1% decrease in traffic
× 0% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment subject line actually delivered a statistically valid 7.5% increase in open rate. Which is good…if the experiment ended there. But the ultimate goal was gifts, which relies on a click. So we needed to measure the entire funnel.
Once we looked past the subject line, we saw that the treatment email (which made the direct ask) produced a 59% decrease in clickthrough rate, which resulted in a 43% drop in conversion. So, the increase in open rate didn’t really matter that much. It highlights an important learning: open rate is only the beginning of the relationship. Getting 7.5% more people to open the email is a win, but not if it doesn’t pay off when we look at our ultimate metric.
This shows us that “human” subject lines can work, but they may affect reader motivation differently. The control subject line, though it had fewer opens, resulted in many more people showing interest in the offer — most likely due to a combination of the urgency it produced and the design of the control email.