Defining user engagement
I’m not going to pretend I’ve got a magic formula for defining user engagement because it’s a pretty ambiguous term that means different things to different people. At the same time, I’ve got some strong opinions that I’m happy to share. If they make sense to you, great, otherwise please comment on why not.
I define engagement as the raw number of meaningful actions a user takes in a day, week or month. I don’t consider clicks, bounces, time on site or any of these other superficial actions as meaningful. I think people tend to create engagement algorithms based on these numbers simply because they’re easy to track and measure.
If you’re Youtube then time on site might mean something but if you’re the DMV, probably not. In both cases it’s a lazy metric that if optimized for, can easily lead you astray.
The DMV case is obvious. If someone is taking 20 minutes to schedule a simple appointment, that is probably not a very positive experience. By improving their interface to make scheduling simpler they lower time on site which likely has a positive impact on engagement. I say likely because if people are only completing a single action of scheduling an appointment in 10 minutes as opposed to 20 minutes then I’d say the site is more usable but equally as engaging.
On the other hand, if the DMV further defines engagement as the # of appointments scheduled along with the # of driving tips consumed then by making the scheduling component more efficient, you’re saving people time and frustration. They’ll likely engage with other parts of your application which creates more meaningful actions and in turn results in higher engagement.
For Youtube, the correlation is stronger but still not ideal. I might define engagement as the time of videos watched plus actions taken on videos (comments, likes, shares…) rather than time on site. The reason for this is to place more of a burden on the app to deliver things people react to. If someone spends 10 minutes channel surfing before finding something they want to watch, I don’t consider that to be high engagement and believe this frustration makes the user less likely to return when presented with a similar problem.
It’s also a pet peeve of mine to define engagement actions as things not directly related to the core value of the app. An example of this is social actions. When apps optimize for social engagement, they tend to create an experience where they push sharing to the front and center while neglecting the things that tell you if people are actually getting sustained value from the tool.