A Quora Designer Breaks Down the Notification Maze

Building a complete notification system is hard. Between cross channel delivery, managing user preferences, and building personalization, there is a lot to think about.

At MagicBell, we are curious to see how businesses are building notification systems for their apps. In my research, I stumbled upon a Medium post by Henry Modisett, a software designer at Quora, that perfectly outlined why creating a great notification system is a lot more complicated than it looks. Having previously worked with companies like Google and Reddit, he’s gotten plenty of exposure to platforms that rely, in large part, on how well their notification systems are designed.

So I sat down with Henry to talk about his experience with the challenges of building notifications and what that looks like for B2B software.

What kind of apps do you like? Which do you find inspiring and think are well-designed?

A principle that I believe in is having a product be self-aware— building features that make sense for it in the context of the product and market it's in. For example, Reddit was losing market share to Discord and added a chat feature, without thinking a lot about how that would work well. When people copy a popular product and plug it into their own product, that can make no sense for the user base.

A product that does seem to understand it's user base well is TikTok. Even though I am not a user of TikTok, I have a scientific interest in when something is successful, and why things are actually working...but that’s more of a product answer than a design answer.

Let’s talk notifications.

You don’t work on notifications anymore, but when you did, what kind of people came together to do the work?

Quora is a very small company, only about 250 employees. Every team is very small, so we have a massive user-to-employee ratio. Any product is going to have a designer, a few engineers and sometimes a product manager if the project requires it. There’s a lot of planning and management. Every product is pretty small, and we are always trying to do a lot with a little.

Because we are small, we have to be thoughtful about the ROI of every project we work on. Every quarter we stack rank across the whole product list and see which will have the most impact. I think at this point there’s enough stuff to do, and notifications are important for engagement, so there’s always some work to do there.

I get so many emails from Quora. The notification scale must be massive, going out each day, each month?

We don’t share numbers…but as you can imagine, we have millions of users and every single person gets more than one notification a day, likely. Whether they read them or not, we send them out. There’s a massive amount of work in scaling that sending system and personalizing it for each person.

How long did you work on this at Quora?

Most of our projects last a couple of months at most, but there have been investments in this, on and off, in the last few years. But my contribution was time was largely spent figuring out what the problem was and designing a notification system at a conceptual level, like building user-facing stuff that prevents a user from too many notifications. If you get 1,000 upvotes we make sure you don’t get 1,000 notifications.

You use machine learning to scale notifications. What does that look like?

Everyone on Quora has a feed, and every feed is personalized based on a million things. Notifications are an extension of that. We want to show people just what they want to see as best as we can. When a user is on the platform, the best way to do that is in a feed. When they’re not on it, it’s through notifications. We want to push notifications for things you might like, and that can be a successful feature when done right.

In your blog post, you said most companies are still measuring the click-through rate to measure success.

This blog was written years ago, and that was my understanding of what companies did at the time. I think to simplify it, there are two types of notifications:

First, the ones a user expects. If I message you, I push a notification when you message me back, whether it's Twitter DMs or text messaging. A lot of notifications fall into that bucket.

Then there are the ones users don’t expect and don’t need to happen, but can happen at any time. We push out notifications for our email digest, or news notifications. They don’t have to happen, but we try to send them to users when we think they will want to click on them.

That's when you measure the efficiency of what, when, and how you’re sending. Why are they clicking on it? There can be a bunch of reasons for that; maybe they don’t like to have an unread batch of messages, or maybe they accidentally clicked on it. Or they might love the notifications.

Ideally, you can collect data on downstream effects, like looking to see if they actually read it after they clicked on it. That’s what you want to train your algorithm on. OS level notification systems don’t have access to that. They can maybe measure how much time you’re in an app, but that’s a different metric. The point of my blog post is that this is all hard.

When you talk about this challenge with other people, what are their thoughts?

I wrote the post, especially for designers, and I have a lot of posts about product design meant for designers that aren’t thinking beyond UI. From a UI perspective, you would think notifications are a simple thing to work on, but I wanted to demonstrate that it’s a complicated thing to work on. Most people don’t get to work on something on such a large scale.

You wrote about how many channels you can get notifications on. You said it is largely unsolved. Why?

I think you could have two different philosophies depending on your resources. It’s easy to get too clever with the whole thing, but there’s an upside to that.

In a perfect world, you can understand things like, ‘this user reads their email on Tuesday night, and mostly from their phone.’ We have built that understanding to some extent. For example, we are a California-based company but have users all over the world. We need to send stuff out to match different time zones, yes, but you can get more specific to each user’s behavior--what channels they prefer, and so on.

I don’t think anyone has built something like that because it's just so super complicated. Especially with machine learning, the more you scale with ML, the more you scale, and the more of a challenge it can be if you do something wrong.

It can be hard to debug, and it's hard to know when it's wrong because it's personalized for many things. So people say let's just go back to measuring click-through rate (CTR).

There are products that are built entirely around notifications, like SmartNews, and I suspect they are thoughtful about their system design in that regard.

Was there something that surprised you about notifications?

How hard it was, particularly working with those operating systems.

At the time iOS didn’t tell us when someone opened something. It felt like all these systems were designed in a vacuum. Apple never planned to have one app to send 1,000 notifications in a day. Maybe it's gotten better since then, I don't know. I showed this post to someone who worked t Apple and he was very sympathetic. (laughs). I sense more people follow the Android cycle.

Are there any companies doing notifications well?

I think everybody that sends a lot of notifications eventually has a different way to do it well--maxing out engagement is one. And making it manageable for users is another. Those things may not be at odds, but they are two things that matter. SmartNew is successful because they max out engagement. You see the news headlines and you know everything that’s going on, that’s why that product is so successful.

I think Twitter has made a lot of progress with its end-to-end user notifications. But you can fall into this trap where you send out notifications that have no meaning. It makes people resent the product and [resentment] is hard to measure.

Generally, I don’t like to see things the same way as most designers. I think of myself as someone just trying to make a product successful, and notifications can do that if done well. It’s only bad If you’re sending bad stuff—or abusing it.

Then they’ll delete your app.

Have you thought about this in a B2B product context?

One time I led the design for a B2B team that let companies run ads on Quora. I think notifications can be put in the same bucket—ones that have to happen (“your ad campaign is finished”) or there are re-engagement things we'd run, like “you haven’t checked in with us recently…,” etc. Those things work; you’re putting yourself in front of their eyes, you just have to be thoughtful about it.

Any parting thoughts?

Apple and Android have to catch up with the fact that we are in a place where the most successful strategy for any product is to max out notifications.

Eventually, it will come to a head. I think it's true that Twitter and TikTok have discovered that...how to compete for people’s attention. This is an attention economy. I don’t think that's Apple’s wheelhouse.

The proper solution is potentially about personalization, and eventually, you have to prioritize what’s good for each person. Maybe Apple should figure out that two news apps are sending notifications on the same thing, but that technical solution is not there. Something Apple does is that they prefer user interactions to solve those problems--like having users manage the settings. It’s just a different philosophy.

I think the next few years will be interesting, especially as more “things,” like Apple watches, can send notifications. I’m thinking the OSs need to catch up.

In a business context, email works...it still has a big place in people's lives and workflows, but I think it's losing market share to push notifications. But we aren’t there just yet.