In anticipation of further growth, Moovly sought to better understand it user behavior trends in more detail prior to implementing advanced sales and marketing growth strategies and associated systems.
We first took all users from the training sample and computed the number of days to subscription. For all subscribers this was easy - we know when person opened a free account, and know when person first subscribed.
This simple exercise gave us a bonus insight: The Moovly product-market fit is so amazing, that out 49% of all their subscribers opt for paid services in the first two weeks of opening an account!
Then we looked at a group of freemium users who definitely did not subscribe in 6 months from opening an account. Some of these people were using the app a lot, some not so much, but we knew precisely that all of them are 'forever-free'. So, we decided to set the number of days to subscribe to a very large number - 210 days (7 months) - which is virtually never.
It turns out that only three metrics are sufficient to accurately distinguish between subscribers, who have a low number of days till subscription, and 'forever-free' users, who we assume will not subscribe before 210.
It turns out that three metrics are sufficient to robustly and accurately predict the time to subscription:
See example predictions of a sample of 196 users BELOW. The first 97 are paid subscribers, and the rest are non-subscribers.
Note, the two pink spikes for users #28 and #37 where we predict the people will only subscribe on day 194 and 188 (which means not at all in our assumptions), while they actually did subscribe on the first day.
DataStories have been consistently predicting these users to not subscribe, and at the end of 10,000 cross-validations marked them as outliers.
Surprisingly, our models say these people should never have become paying customers based on their interaction with the product.
We shared this fact with Moovly and discovered, that one of the users is the wife of a co-founder, and the other user is the office neighbor, and both these people got an automatic subscription on the first day!
It turns out they are special, and we could determine this automatically. Based on the data!
Predicting the # days to subscription is interesting but because it requires the total number of logins up to today, it may not provide a good business opportunity to plan far ahead.
To improve predictability we divided clients into cohorts by time passed since opening an account, and built predictive models of whether or not any particular person will become a paying customer in the next couple of weeks.
So first we selected three following groups of freemium users:
The choice of time periods to define cohorts should be dictated by business needs. The good thing is that the cohorts are non-intersecting. They will form a funnel of non-subscribers going from 24-hour cohort, into 3-day cohort, into 1-week cohort, and so on, until they subscribe, or churn.
The business may also have different budgets to convert clients in each cohort (e.g. people who haven't subscribed in a month are less warm for conversion than people who opted to try the product yesterday).
Out of 277 metrics considered only 3 to 5 were selected as drivers in each cohort. Of course, they may change in time as the business changes, but the process of identifying them with new data only takes 30 minutes with DataStories.
For example, the metric of whether of not the user is subscribed to the newsletter was never selected as the subscription driver. However, if Moovly further customizes the newsletters, or starts a new breed of tutorial letters - it may become a converting factor in the future.
Then we went ahead and built several types of predictive models using the handful of drivers, discovered by DataStories.
More complex algorithms did give us better accuracy, but even the simplest scoring models did output acceptable results! This is what we love the most in our job - going to hell (heavy-duty machine learning and computational intelligence) and back to simplify the problem, all for the pleasure of finding a solution and everyone can understand and use!
A great thing about the heat score model is that it also helps identify unique score thresholds for your business.
One way to do this is to compute a statistically optimal score, which optimally splits non-subscribers from subscribers. In the simple example above this threshold is 321.
A better way to find the threshold is to use the available marketing budget as the guidance.
|Can compute conversion rate, but have no control on who will convert.||Score each freemium user early & get reliable estimation of how likely each user will subscribe.|
|Enjoy 4-16% conversion rate from targeting all users.||Enjoy 79-84% conversion rate from targeting the top 20% of heat-score customers.|
|Lack of insights into user intention makes it hard to change user behavior.||Precise heat-scores help identify 'nudge-eable' users who are a tad short of subscribing and target them to change their behavior.|
|"Spray & Pray" is one of the few options to spend marketing budget.||Each new customer gets assigned a heat score. Low and High value customers don’t need to be contacted. Now you know to spend 90%+ of your marketing budget on Medium Heat Score customers. A simple call or bonus gift to these "on the fence" customers can convince them to purchase.|
Or call us: +32 47 638 84 97