At the spring TSIA conference, I had 22 one-on-one meetings with members and partners, and two-thirds of those meetings focused on measuring customer health and how to automate the creation of health scores. I’m seeing technology firms coming at this challenge from multiple perspectives.
In this blog, I will touch on three approaches to creating an automated health score, with pros and cons for each approach. Then I will go into additional details on the most common approach, give guidance to companies just embarking on this initiative, and increase the accuracy of your current health score calculations.
Why Do Health Scores Matter?
Why is there so much interest in measuring customer health? In a subscription technology world, account health indicates customers are adopting your technology, receiving business value from them, and ideally are achieving the desired outcomes from their investment.
For customer success organizations whose charters include product adoption, subscription renewals, and expanded selling, having accurate data on account health allows them to:
- Identify accounts encountering issues that may impact renewals or expand selling efforts. Early warning of potential problems will allow a customer success manager (CSM) to be more engaged and proactive with the account, such as starting renewal conversations earlier than normal.
- Create a get-well plan for accounts not adopting or receiving value as quickly as anticipated. Customers slow to adopt may need additional onboarding or training to boost adoption. A TSIA survey earlier this year found that 85% of technology buyers said product usage or utilization reports were the most valuable information in determining whether to renew a technology subscription. The account users consuming the technology is critical.
- Prioritize CSM time, and focus efforts on accounts that will have the biggest impact. According to the TSIA Customer Success Benchmark Survey, the median account to CSM ratio for high touch accounts is 8.0, for low touch accounts the median ratio is 55.0. For CSMs juggling many accounts, it is critical to know which accounts are in the most need of additional engagement. Health scores inform them where to prioritize their time for the most impact.
Three Primary Approaches to Automating Health Scores
At a high level, I am seeing three primary approaches to automating customer health scores, listed here from the easiest to the most sophisticated.
Sentiment Analysis
This approach leverages technology to analyze every conversation with the customer, including support case notes, chat conversations, emails, survey ratings and verbatims, etc., to determine a red/yellow/green status for each account.
On the plus side, this approach is easy to do. It can be accomplished using data in a CRM system, and the rating will be reevaluated with each conversation automatically. On the downside, the sentiment analysis approach does not include all data sources and does not factor in the CSM disposition of each account. So although it is relatively easy (companies report results within 30 days), the score or rating is not as accurate as more sophisticated approaches. Sentiment analysis is a “back of the envelope” sort of rating that may be a good place to start, but ultimately will not provide enough insight for larger accounts and complex contracts.
Multiple Data Points with Weighting
The most common approach used by TSIA members is building a calculation pulling in both qualitative and quantitative data from various sources and assigning weights to each to arrive at a health score.
This approach can be much more accurate than just sentiment analysis, as it includes more data points, and weighting can be adjusted as the model improves over time. However, to do this effectively, you need integration to multiple systems or data sources, which increases the cost of the implementation, and ownership costs to maintain each integration over time. Also, the accuracy of the health score is questionable if weighting emphasizes the wrong information. If weighting is based on guesses, and not analysis, your score will not be actionable.
Customer Journey Analysis
The most accurate approach is of course the most complex and currently only done by pacesetters with a core competency in data analysis, artificial intelligence (AI), and machine learning. This approach analyzes years of historical data about every step of the customer journey, i.e., the sales experience, the implementation experience, product adoption, the service experience, financial data, etc., and identifies patterns in accounts that contribute to high or low renewals, growing or shrinking wallet share, NPS scores, reference-ability, or other desired account outcomes or activity. Current accounts are analyzed by these patterns to identify likelihood of renewal and propensity for other desired account behavior.
Though few companies have achieved full implementations of this approach, pacesetters have found the results to be highly accurate and predictive, and can provide insight to constantly improve success plans. Unfortunately, this approach requires a lot of data, very strong data hygiene, a team of data scientists to do correlation analysis, and machine learning to continually “learn” as more data and activity is collected.
For now, this level of sophistication is beyond the reach of many tech companies, though stay tuned, as more “out of box” capabilities are quickly emerging from innovative customer success vendors to enable this.
Considerations for Accurate Weighting
With the majority of companies using the multiple data points with weighting approach, I wanted to provide some guidance on creating a calculation with the highest accuracy.
The first hurdle is deciding what data to include in the score calculation, and to do this you need to inventory all available data sources. The data should include information on how well the customer is adopting the technology, including both adoption data and successful onboarding information. CSM disposition is important, as they have 1:1 interactions with accounts and will have cues from these conversations about the perception of the products and value received.
TSIA has found that the support experience is a major contributor to customer health, so be sure to include information about the support case history of accounts. In particular, the number of cases, severity or priority, resolution time, escalations, and post-interaction CSAT scores should all be considered, as we tend to see correlations between these metrics and renewals.
Once you have identified the available data, you need to assign weighting to each. Understanding how to weight each data source in the calculation is key to an actionable health score. For companies that have invested in a “best of breed” customer success work management platform, some tools will help. I recently saw an impressive demo from one vendor that allows you to select any piece of data and see how it correlates to renewals and expand selling success. This information makes it easy to identify which content has the biggest impact and then prioritize it in the weighting.
As a reference, here is some data from the TSIA Customer Success Benchmark Survey showing common data elements and weighting.
Common Challenges to Accurate Health Scores
Based on conversations with TSIA members, companies face some common challenges when creating automated account health scores.
- Access to data and data accuracy. Lots of data sources means lots of integration work, and if you have data in proprietary systems, your implementation team will have to do custom development. Data accuracy is a challenge for every company, and building a score with questionable data will never deliver actionable results. Often key information may be in emails or spreadsheets and difficult to incorporate. As critical as accurate health scores are, this project may help drive new processes to capture data in a more structured and useable way.
- Accuracy of weighting. We need to be honest with ourselves about the reason we are creating health scores, as there have been unfortunate examples of companies tweaking weighting and scoring to give a false impression of health to look good to management or achieve personal MBOs. Ultimately CS management pays the price when their rosy portrait of account health doesn’t materialize into revenue. So be sure to base your weighting on historical analysis, not assumptions.
- Effectively leveraging the score. Even the most accurate score isn’t helpful if it is only on a dashboard no one accesses. Ideally, scores should be written to CRM account records, so they are visible to everyone who touches a customer. And, health scores should be used to trigger events and proactively notify CSMs or even executives of issues. For example, a poor score or a score trending down needs additional interaction and a “get well” plan. The CSM may need to become more involved in issues impacting the account outside of their control, such as support escalations. Renewal activities should begin sooner for accounts with potential problems.
Final Recommendations
Whether you are embarking on your first project to create an automated health score or trying to improve the accuracy of your existing scoring, here are some final recommendations.
- Start with what’s easy to access. Even if you only begin with a few data points, such as CSM disposition, adoption, and support CSAT scores, it is better to have a starting point than to languish with no insight for months (or years) trying to create the perfect score.
- Create a health score roadmap. For data that may be hard to access or has questionable accuracy, create a roadmap for how to better capture information in accessible systems and drive better data hygiene, so additional data points can be incorporated to improve health score accuracy ongoing. Have a CRM platform that’s conducive to customer success. I have heard horror stories from CS platform vendors about customers who attempted to launch success functionality as an add-on to the renewal motion and fell completely flat because their core CRM system didn’t have the capabilities to deal with data input/qualitative input or triangulating scores that were usable in production. Be upfront about your current technology environment when communicating your roadmap with your CS platform vendor, and ask them for advice on potential CRM conflicts that may influence your plans.
- Leverage AI. The single most important piece of an accurate health score is doing the correlation analysis on how each piece of data relates to health. While I’ve given some examples in this blog, the impact of various data sources will be different for every company—there is no “one size fits all” customer health score formula. The more intelligence you can incorporate into the project to continually refine and improve both data sources and weighing accuracy, the better.
- Define the rules of engagement across the enterprise. For many companies, the customer success organization is new, and as the point person for the account, the rest of the enterprise needs to understand that the rules of engagement have changed. A seamless experience for the customer is critical. When the customer receives conflicting information from sales account managers, support techs, professional services consultants, etc., it confuses and impacts the relationship’s health. CSMs must have insight into customer interactions, so roles, communications, and expectation setting must be well defined.
Still not sure where to begin or optimize an accurate customer health score? Schedule a call today to learn how our experts can help.