How increasing the force of the value proposition affects revenue
Experiment ID: #1381
The Heritage Foundation
Experiment Summary
Timeframe: 03/09/2016 - 03/15/2016
The Heritage Foundation was sending lots of traffic to their “Conservative Declaration”, a document that asked conservatives to unite under shared principles. Immediately after signing, the visitors were presented with an offer to become a member of The Heritage Foundation by making a donation. The entire experience took place within a single design.
Heritage had another membership page that had been performing extremely well. However, design was a key part of this page, and it didn’t match the design of the Conservative Declaration. Knowing that congruency is a key part of conversion, they decided to test it anyways to see if increasing the force of the value proposition outweighed design consistency.
Research Question
Will a stronger value proposition increase donations and revenue, in spite of a different design?
Design
C: Control
Array
(
[0] => Array
(
[ID] => 42583
[id] => 42583
[title] => Control.png&unique=1646702954&fullpage=true&width=1440&height=0
[filename] =>
[filesize] => 0
[url] => https://www.nextafter.com/?attachment_id=42583
[link] => https://www.nextafter.com/experiments/how-increasing-the-force-of-the-value-proposition-affects-revenue-2/attachment/control-pngunique1646702954fullpagetruewidth1440height0-11/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-pngunique1646702954fullpagetruewidth1440height0-11
[status] => inherit
[uploaded_to] => 38379
[date] => 2022-10-25 21:57:19
[modified] => 2022-10-25 21:57:19
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[sizes] => Array
(
)
)
[1] => Array
(
[ID] => 47267
[id] => 47267
[title] => Screen Shot 2022-11-04 at 3.02.52 PM
[filename] => Screen-Shot-2022-11-04-at-3.02.52-PM.png
[filesize] => 211499
[url] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[link] => https://www.nextafter.com/experiments/how-increasing-the-force-of-the-value-proposition-affects-revenue-2/attachment/screen-shot-2022-11-04-at-3-02-52-pm/
[alt] =>
[author] => 52
[description] =>
[caption] =>
[name] => screen-shot-2022-11-04-at-3-02-52-pm
[status] => inherit
[uploaded_to] => 38379
[date] => 2022-11-04 20:03:39
[modified] => 2022-11-04 20:03:39
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 350
[height] => 1132
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-93x300.png
[medium-width] => 93
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[medium_large-width] => 350
[medium_large-height] => 1132
[large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-317x1024.png
[large-width] => 317
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[1536x1536-width] => 350
[1536x1536-height] => 1132
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[2048x2048-width] => 350
[2048x2048-height] => 1132
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[tp-image-grid-width] => 216
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-15x50.png
[admin-width] => 15
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[full-width] => 350
[full-height] => 1132
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[thumb_372-width] => 350
[thumb_372-height] => 1132
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-193x625.png
[single_post_thumbnail-width] => 193
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-10x32.png
[thumb_32_32-width] => 10
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-15x48.png
[thumb_48_48-width] => 15
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-20x64.png
[thumb_64_64-width] => 20
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-40x128.png
[thumb_128_128-width] => 40
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-21x67.png
[thumb_122_67-width] => 21
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-47x153.png
[thumb_153_153-width] => 47
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-50x163.png
[thumb_130_163-width] => 50
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-55x177.png
[thumb_232_177-width] => 55
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-201x650.png
[thumb_272_325-width] => 201
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-107x346.png
[thumb_271_346-width] => 107
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-84x271.png
[thumb_271_271-width] => 84
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-53x172.png
[thumb_272_172-width] => 53
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-143x461.png
[thumb_372_461-width] => 143
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-155x500.png
[thumb_372_500-width] => 155
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-83x270.png
[thumb_385_270-width] => 83
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-75x241.png
[thumb_416_241-width] => 75
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-127x410.png
[thumb_448_410-width] => 127
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-254x820.png
[thumb_448_410_retina-width] => 254
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-118x382.png
[thumb_574_382-width] => 118
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-122x395.png
[thumb_776_395-width] => 122
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[gform-image-choice-sm-width] => 93
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[gform-image-choice-md-width] => 124
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[gform-image-choice-lg-width] => 186
[gform-image-choice-lg-height] => 600
)
)
)
T1: Updated Value Prop
Array
(
[0] => Array
(
[ID] => 42583
[id] => 42583
[title] => Control.png&unique=1646702954&fullpage=true&width=1440&height=0
[filename] =>
[filesize] => 0
[url] => https://www.nextafter.com/?attachment_id=42583
[link] => https://www.nextafter.com/experiments/how-increasing-the-force-of-the-value-proposition-affects-revenue-2/attachment/control-pngunique1646702954fullpagetruewidth1440height0-11/
[alt] =>
[author] => 0
[description] =>
[caption] =>
[name] => control-pngunique1646702954fullpagetruewidth1440height0-11
[status] => inherit
[uploaded_to] => 38379
[date] => 2022-10-25 21:57:19
[modified] => 2022-10-25 21:57:19
[menu_order] => 0
[mime_type] => image/jpeg
[type] => image
[subtype] => jpeg
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[sizes] => Array
(
)
)
[1] => Array
(
[ID] => 47267
[id] => 47267
[title] => Screen Shot 2022-11-04 at 3.02.52 PM
[filename] => Screen-Shot-2022-11-04-at-3.02.52-PM.png
[filesize] => 211499
[url] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[link] => https://www.nextafter.com/experiments/how-increasing-the-force-of-the-value-proposition-affects-revenue-2/attachment/screen-shot-2022-11-04-at-3-02-52-pm/
[alt] =>
[author] => 52
[description] =>
[caption] =>
[name] => screen-shot-2022-11-04-at-3-02-52-pm
[status] => inherit
[uploaded_to] => 38379
[date] => 2022-11-04 20:03:39
[modified] => 2022-11-04 20:03:39
[menu_order] => 0
[mime_type] => image/png
[type] => image
[subtype] => png
[icon] => https://www.nextafter.com/wp-includes/images/media/default.png
[width] => 350
[height] => 1132
[sizes] => Array
(
[thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-150x150.png
[thumbnail-width] => 150
[thumbnail-height] => 150
[medium] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-93x300.png
[medium-width] => 93
[medium-height] => 300
[medium_large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[medium_large-width] => 350
[medium_large-height] => 1132
[large] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-317x1024.png
[large-width] => 317
[large-height] => 1024
[1536x1536] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[1536x1536-width] => 350
[1536x1536-height] => 1132
[2048x2048] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[2048x2048-width] => 350
[2048x2048-height] => 1132
[tp-image-grid] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[tp-image-grid-width] => 216
[tp-image-grid-height] => 700
[post-thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-150x150.png
[post-thumbnail-width] => 150
[post-thumbnail-height] => 150
[admin] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-15x50.png
[admin-width] => 15
[admin-height] => 50
[full] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[full-width] => 350
[full-height] => 1132
[thumb_372] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[thumb_372-width] => 350
[thumb_372-height] => 1132
[single_post_thumbnail] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-193x625.png
[single_post_thumbnail-width] => 193
[single_post_thumbnail-height] => 625
[thumb_32_32] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-10x32.png
[thumb_32_32-width] => 10
[thumb_32_32-height] => 32
[thumb_48_48] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-15x48.png
[thumb_48_48-width] => 15
[thumb_48_48-height] => 48
[thumb_64_64] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-20x64.png
[thumb_64_64-width] => 20
[thumb_64_64-height] => 64
[thumb_128_128] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-40x128.png
[thumb_128_128-width] => 40
[thumb_128_128-height] => 128
[thumb_122_67] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-21x67.png
[thumb_122_67-width] => 21
[thumb_122_67-height] => 67
[thumb_153_153] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-47x153.png
[thumb_153_153-width] => 47
[thumb_153_153-height] => 153
[thumb_130_163] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-50x163.png
[thumb_130_163-width] => 50
[thumb_130_163-height] => 163
[thumb_232_177] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-55x177.png
[thumb_232_177-width] => 55
[thumb_232_177-height] => 177
[thumb_272_325] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-201x650.png
[thumb_272_325-width] => 201
[thumb_272_325-height] => 650
[thumb_271_346] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-107x346.png
[thumb_271_346-width] => 107
[thumb_271_346-height] => 346
[thumb_271_271] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-84x271.png
[thumb_271_271-width] => 84
[thumb_271_271-height] => 271
[thumb_272_172] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-53x172.png
[thumb_272_172-width] => 53
[thumb_272_172-height] => 172
[thumb_372_461] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-143x461.png
[thumb_372_461-width] => 143
[thumb_372_461-height] => 461
[thumb_372_500] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-155x500.png
[thumb_372_500-width] => 155
[thumb_372_500-height] => 500
[thumb_385_270] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-83x270.png
[thumb_385_270-width] => 83
[thumb_385_270-height] => 270
[thumb_416_241] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-75x241.png
[thumb_416_241-width] => 75
[thumb_416_241-height] => 241
[thumb_448_410] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-127x410.png
[thumb_448_410-width] => 127
[thumb_448_410-height] => 410
[thumb_448_410_retina] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-254x820.png
[thumb_448_410_retina-width] => 254
[thumb_448_410_retina-height] => 820
[thumb_574_382] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-118x382.png
[thumb_574_382-width] => 118
[thumb_574_382-height] => 382
[thumb_776_395] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM-122x395.png
[thumb_776_395-width] => 122
[thumb_776_395-height] => 395
[gform-image-choice-sm] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[gform-image-choice-sm-width] => 93
[gform-image-choice-sm-height] => 300
[gform-image-choice-md] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[gform-image-choice-md-width] => 124
[gform-image-choice-md-height] => 400
[gform-image-choice-lg] => https://www.nextafter.com/wp-content/uploads/Screen-Shot-2022-11-04-at-3.02.52-PM.png
[gform-image-choice-lg-width] => 186
[gform-image-choice-lg-height] => 600
)
)
)
Results
| Treatment Name | Conv. Rate | Relative Difference | Confidence | |
|---|---|---|---|---|
| C: | Control | 0.65% | ||
| T1: | Updated Value Prop | 3.5% | 443.2% | 99.8% |
This experiment has a required sample size of 192 in order to be valid. Since the experiment had a total sample size of 944, and the level of confidence is above 95% the experiment results are valid.
Flux Metrics Affected
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
0% increase in traffic
× 443.2% increase in conversion rate
× 0% increase in average gift
Key Learnings
The treatment produced a 460% increase in revenue, despite breaking from the design of the original offer. This is a powerful learning that shows that the message is more important than the container. This also points the way to a future test to see if revenue can be further increased by working the new value proposition into the existing design and increasing congruency.