Welcome to the first subscriber-only post of Counting Stuff! And thank you for supporting this extra adventure! If you haven’t signed up, there’s a 60d free trial launch offer available.
I’ve finally opened up paid subscriptions because, after over 130 weekly posts without missing a single week, I’ve accumulated enough madness points to believe I can come up with another 1-3 things to write about in a month. These subscriber posts will be a bit more “in the weeds” about things that are on my mind. They’ll also be a bit less regular, every couple of Thursdays depending on when I think a post is ready to go. At least, that’s the image in my head, things will evolve as I settle in.
So to… celebrate(?) this occasion, I was curious about how the sign-up process for the newsletter might be changing since I’d be inflicting it on humans going forward. Plus, since the hand isn’t feeding me anyways, let’s bite it a bit for fun.
The original free-only process is pretty simple, enter an email and hit the subscribe button, with an option to view the archive. It largely resembles the screenshot below.
The default flow is very simple because all a user has to do is put their email address in and hit subscribe. But when there’s a paid option available, they get dropped into this following page, a stripped down checkout flow:
This subscription choice page has 4 cards on it because (entirely on a whim) I had turned on the extra feature that lets people enter any amount they choose as an extra special tier. Normally, there would only be 3 cards arranged in what is sometimes called a “Good-Better-Best” pricing setup. There some research out there that people are likely to pick the middle priced “Better” option in such setups provided the other prices aren’t too outrageous. Adding a 4th card probably screws things up a bit.
What I find interesting (read: annoying) is that the Free option is shoved furthest to the right. My gut tells me that this page has been A/B tested at least a few times since it’s the primary revenue source for the company. This design decision must have been made because it generated more paid subscribers than not in the test. I just personally hate it.
I also found it interesting that most good-better-best designs show the basic/free tier on the left, list out the features of that tier, and progressively stack even more benefits for the more expensive tiers. Substack has instead chosen to go the other direction and show how the free tier is missing out on features relative to the other ones. They’re really pushing hard on the “PAID IS BETTER!!!!” angle.
I’ve tested this sign-up flow using a spare email account that’s never interacted with Substack before and it seems that unless a user actually selects the free tier and hits the continue button, they don’t actually get subscribed for emails? (Update: this morning the account got an email so I don’t even know what’s true anymore) So it’s entirely possible that a casual user who’s not ready to pay yet might get confused by the design and wind up not subscribing — potentially missing out on getting a precious reader who could be upsold in the future. That seems suboptimal over the long term, especially for newsletters like mine where the majority of energy goes into the free content.
Since they’ve added an extra screen’s worth of friction to the process, there’s always a drop (sometimes small, sometimes big) in the number of people who go through the flow. You’d need to look at the data to figure out if the drop aligns with what your goals are.
It’s entirely possible that on the back end, someone sat down, did the analysis and came to the conclusion that upsells are sufficiently rare that it doesn’t matter? If the marginal benefit of getting a free user is practically zero because they rarely upgrade, then maybe the effect of pushing HARD on the issue of getting a credit card number up front is worth more money to them. I’m a bit skeptical since freemium models often live and die on the upsell and retention but you can’t predict a priori without the data because user behavior on these forms are always very unintuitive.
Super unintuitive “you can edit the price” feature
If you click on the super -support tier (which I had set to be “Super Supporting” in the options), you’re allowed to set whatever price you’d like. Except the only indication is that when you click in, it auto-highlights the price. It’s a hint that you can type in numbers, but there’s no other indication that you can change it that I can see. My guess is that they meant for the user to just “accept the default offer” which the newsletter owner sets.
The more you look at it, the more you realize there’s whole stacks of psychological games going on here. There’s two games of “people just take the defaults” going on. The a default presented to the newsletter owner is $150 (3x the typical $50/year subscription price), as well as the default presented to potential customers who aren’t really presented with the info that they can adjust the price.
With multiple degrees of freedom between what authors and users doing, I’m pretty sure that using A/B tests to discover the optimal pricing strategy takes ridiculous amounts of data, and it’d take too long. Instead they’re likely analyzing the feature as a kind of natural experiment where they’ll just see the adoption rates at various price points over time and adjust things later. In the meantime the initial values were probably set by product using a gut felling of “that feels about correct”.
Either way, it does sound fun to think about what analytical framework you’d need to make sense of the data you’d get, since prices can be set to practically any number and you’d have to plot out some kind of adoption curve.
Also there’s buried features
One last thing I noticed about the already hard-to-understand subscribe page is that under the giant blue call to action, there’s this mysterious, slightly grey “Other subscription options…” choice. Clicking on that brings up a popup that allows for gifting and group subscriptions. If you click on either option you’re sent to a dedicated checkout flow for those options. Wat?
It almost feels like the gift/group-sub features were added due to some big partners requesting the feature and they wanted to centralize all payments to a single page, but didn’t know where to put it on their subscribe page without adding more clutter and hurting conversion even more. So into the penalty box they go.
Since such advanced features aren’t likely to be used for an initial sign-up, it gets reflected in the design by packing them away into an unoffensive link that few people will read. After that, it becomes something of a self-fulfilling prophecy of how such features “don’t get used much”.
At a high level, the product seems to rely on writers themselves to put buttons for these features in posts (like the examples I put below), that puts users into a different, dedicated, checkout flows.
If you click on the buttons, they actually bring you to similar URLs: https://counting.substack.com/subscribe?group=true versus https://counting.substack.com/subscribe?gift=true. These are essentially the same URLs as when you click the “other subscription options” link mentioned before.
All told, it’s somewhat confusing from a design standpoint. They seem to be trying to route all payment/subscription stuff through the /subscribe URL but have completely different checkout pages for them. But they left links to those features “just in case” a user happens to stumble upon them, likely under the logic of “well it can’t hurt”.
At least… there’s room for improvement?
So yeah, that was a look at their subscription flow. It works okay enough to keep the business going, but has already picked up a bit of bolted on bits over time. I imagine that someone will eventually get around to circling back and considering a design refresh. There’s some interesting analytics questions sorta lurking at the edges of things in terms of optimizing revenue and free subscriptions. But since the business appears to be doing fine right now, the product teams are busy implementing other features.
About this newsletter
I’m Randy Au, Quantitative UX researcher, former data analyst, and general-purpose data and tech nerd. Counting Stuff is a weekly newsletter about the less-than-sexy aspects of data science, UX research and tech. With some excursions into other fun topics.
All photos/drawings used are taken/created by Randy unless otherwise credited.
randyau.com — Curated archive of evergreen posts.
Approaching Significance Discord —where data folk hang out and can talk a bit about data, and a bit about everything else. Randy moderates the discord.
Support the newsletter:
You’re already support the newsletter by subscribing! Thanks! But here’s some more options:
Yes, you can share these posts with others
Tweet me with comments and questions
Get merch! If shirts and stickers are more your style — There’s a survivorship bias shirt!