About continuous listening, employee experience, people analytics and visualization
If you are interested in learning more on (1) how to listen to your employees, (2) how to have a deeper understanding of what is on their minds and (3) how to execute on the insights, we are pretty sure you will enjoy reading this post.
Last year I promised I would share more details about our approach on what we call continuous employee listening. We achieved and learned a lot in the past year. We now have a better understanding of the employee experience. What do employees like about our organization and what do they want to see improved? Needless to say, this is extremely valuable information. We can use this information to take down barriers our employees are facing during their career. Taking down barriers, so employees can use their talents and stay engaged, healthy and productive in the same time.
Previously I posted two articles on this topic. The first was two years ago, together with Frank van den Brink (CHRO), where we shared our framework on continuous listening and some of our first steps (see “HR is hitting a second wall“) . Last year I showed, together with Luuk Sombezki (Employee Experience Lead), how we were implementing this framework in order to better understand what is on the minds of our employees (see “The 10 golden rules of continuous employee listening“) . Today we want to provide a more detailed insight on what we do to understand our employees from a people analytics perspective. Many thanks to my team and specifically to Jaap Veldkamp, Sabrina Korteman and Anouk Breugelmans. They deserve the credits for most of the work described in this post.
Listen, understand and act
First of all, we think it is great to see that our organization, like more and more companies, explicitly mention Employee Experience in their companywide strategy. To be able to steer in the right direction, a continuous employee listening framework is needed.
Let’s start to recap this framework before we move on. It is actually quite simple. (1) Start listening to your employees by using survey data and transactional data, (2) love the problem, by putting effort in understanding what your employees are telling you via service design and extra data analytics and finally (3) try a solution and take action. The trick is, you have to repeat these steps. In that way you can evaluate if your actions are effective and if the employee experience increases. We repeat all steps periodically and share them within the organization with employees, experts and business management teams.
If you want to read a bit more on our continuous employee listening framework, we refer you to our previous posts. From here on we will elaborate in more detail on how we capture and transform the data, how we do our topic detection (text analytics) and how we decided to visualize the insights to drive actions.
The data and the analytics
On top of everything we already do to listen to our employees, we implemented a monthly employee experience survey. The main question in this survey is as follows: “How likely are you to recommend our organization to a friend or relative as an organization to work for?”
For most of you, this question will look familiar, because it is very similar to the standard and proven net promoter score (NPS) question asked to (also our) customers. Employees can give a numerical score, which we use to calculate the overall Employee Experience score. But since we can’t define actions based on this question, we added two open questions: (1) what is our organization doing well as an employer (top) and (2) what could our organization do better as an employer (tip)? The open questions provide us with more contextual information.
We want to avoid survey fatigue, so we use samples so that our employees will receive this questionnaire only once a year. Due to stratified sampling techniques, we know that for the quantitative results the monthly samples are representative for the whole organization. And on quarterly basis, it is also representative for most business lines. For us, especially the open comments are of great interest. At this moment we receive over a thousand of comments a month. So, we needed a little help in processing all the information. This is why we invested in our topic detection capabilities.
First, we needed to transform all the comments from our employees into a standard format that our topic detection algorithm could understand and process. We did this by translating all comments to English, splitting the answers of our employees into multiple comments when they contained multiple subjects, and by converting all words to lower case and removing punctuations. On top of that we are using a technique called lemmatization. All this was necessary to reduce different forms of a word with similar meanings to a common root (lemma). For instance, ‘Opportunities’ will be reduced to ‘opportunity’. Also depending on the context whether, for example, ‘meeting’ is a noun or a verb, the correct lemma will be determined (‘meeting’ or ‘meet’). Once the comments were split and cleaned, we used techniques like Word2Vec and TF-IDF to translate the comments into vectors/ numbers. This provided us with a more sophisticated way to let a classification algorithm learn and predict which topic belongs to each comment.
In our first approach, we used an unsupervised model to classify all comments. But we significantly improved the classification by initially reading samples of the comments ourselves. From there we started building a supervised model. We strongly advise you to do this as well because it boosts the precision of your classification. Based on this dataset, we were able to test many classification algorithms using cross-validation. We found that in our case the best performing algorithm is the Support Vector Machine (SVM Linear).
In our experience open questions provide us with more contextual information.
Remember, we ask two open questions (tip and top). For each open question we classify the comments. So, for instance a new comment related to the question ‘What is our organization doing well as an employer (top)?’ could be: ‘I can do the training I need’. This could be classified to the topic ‘Training and development opportunities’. At this moment, we have a roughly 150 topics.
Periodically we maintain and improve our model, because the model is never finished due to monthly collected new comments. We do this in two ways. First, we actively ask our users of the data to provide us with feedback on comments that are labelled in a wrong way. Second, we check which comments are not labelled at all, and we see if we can label those comments into new topics by using statistics. Again, we make sure that we adjust the training set and retrain the model.
The visualization of the employee voice
Like already mentioned, we collect new data every month. We share the insights within the organization in three different ways.
First, it is important to share the insights with all employees. We inform them on what they are telling us and what we have done with their input. This increases support for the survey and the response rate. We do this by placing an infographic on the intranet with the main results and actions that are being taken.
Second, all experts in HR, IT, Facility Management and Communication should be able to make independent use of the insights. So we build a Power BI dashboard that is updated every month. To make sure they don’t have an information overload, we cluster these 150 topics into expert domains like recruitment, learning or leadership. If desired they can find in the dashboard more detailed information on demographics.
Last but not least, per quarter the insights of our work are discussed in all top business management teams to evaluate and prioritize interventions. All this together proves for us that our organization takes the voice of the employee seriously.
Now that we found a way to ‘listen’ to our employees and all their comments, we needed a strong visualization that helped us in telling the story of our employees and share it in the organization.
Data analytics is just a tool that helps us to understand what people are saying, but we still need to communicate it back to the organization, so we can make sense out of the data and determine what actions to take. We tried multiple graphs and charts but finally we chose to go for the bubble chart as shown below.
We hope you understand we don’t share all the insights we found so we left out the labels of the topics in this screen shot of our Power BI dashboard. In our case this visual turned out to be successful, but by no means we imply it is the only or the best one. It can also be beneficial to visualize the data in multiple ways when having a conversation with colleagues on what is being said.
So how to read this bubble chart:
- The size of each bubble represents the total number of comments per topic (tip and top).
- The topics positioned above the horizontal line are topics with more top then tip comments. Below the horizontal line it is the other way around.
- Topics positioned left of the vertical line are topics with a relative low score on the numerical recommendation question mentioned earlier in this post. Topics to the right of the line have a relative high score.
Now the fun thing in our EX dashboard, build in Power BI, is that you can follow the topics in time, per business line and per role. This is very valuable because in this way you actually see topics disappear or pop up in areas of your organization and grow or shrink over time. For a specific role, for instance data scientist, it is helpful to understand what employees in that role are telling us about what they like and don’t like.
Prioritizing on the voice of the employee
In our HR organization the results from our analysis, or in other word ‘the voice of the employee’, is used in prioritization. Per quarter we evaluate our new and existing initiatives. When initiatives are aimed at improving the employee experience, we check if the solution actually solves a problem or creates an opportunity for our employees, based on what employees tell us on a monthly basis.
The applications of these insights are numerous. We can use them in branding, recruitment, development, retention and in taking away barriers for employees in general to boost employee experience. The real excitement, as an analytical nerd, is of course using all this topic data not only to understand employee experience but also to predict customer satisfaction, customer turnover, sales, fraud etcetera. But we leave that for our next post.
We hope you appreciate us sharing our experiences with you. And we hope it inspires you to start your initiative. Why wouldn’t your organization do the utmost to understand what your employees are thinking and talking about? If you have any suggestions or feedback, please feel free to leave a comment below.
We want to thank and highlight some thought leaders and companies that helped us shape our thinking and/or helped us on specific topics in our continuous employee listening journey.
David Green, Jonathan Ferrar, Al Adamsen (Insight222), Laura Stevens (iNostix by Deloitte), Volker Jacobs (TI-People), Andrew Marritt (Organization View), Jacob Morgan (The Future of Work University), Elliott Nelson (Kennedy Fitch).