When and how to use a skill survey
Steven Forth is a Co-Founder of TeamFit. See his Skill Profile.
Surveys of various types have been a popular tool for people in talent and performance management. So popular that many of us feel surveyed out. I confess that Ibbaka contributes to this as every year we conduct a number of surveys, a general one on skill management and additional surveys focused on specific professional skills. Currently, we are conducting a survey on the skills needed for pricing expertise for a presentation we are making at the Professional Pricing Society Conference in Chicago in May.
We are also designing surveys on the skills required for product management and the skills required for talent management (including learning and performance).In our Skill Insight engagements, our consulting offering, we also use surveys as a way to gather initial skill and benchmarking data.
Surveys are imperfect for the following reasons
Response rates are often low, so self-selection and bias becomes an issue
Responses are limited to pre-determined choices, limiting the range of conversations
Survey data is treated as the reality, when in fact it offers only one perspective and needs to be integrated with other approaches
Why do people use surveys?
Given the known issues, why do people use surveys? There are some good reasons and some bad reasons.
Bad Reasons
It is a way to avoid deeper and often difficult conversations
It gives the appearance of engagement and listening (but engagement and listening only take place in conversations)
It makes the data simpler and easier to process
Good Reasons
It is a way to reach more people
The provide openings for new conversations
They provide a rich and diverse source of data
Carefully used, the good reasons outweigh the bad. Surveys are going to remain part of how we gather data and generate information for many years to come. HR and talent management will various short and long form surveys for everything from pulse checks on engagement, to 360 reviews, to getting input on benefit plans and working conditions.
The best companies will not rely on surveys alone but will triangulate with data gathered from their enterprise skill, collaboration, project management and other systems, then combine the survey data and collected data with interviews to leaven quantitative analysis with qualitative understanding.
They will also make sure that surveys are two way so that as employees are asked questions they will have a chance to ask questions in return and to see the conclusions drawn from the information they have contributed.
Why does Ibbaka use skill surveys and how do we use the data?
At Ibbaka, we use surveys for several reasons. In some cases we want to gather data about a new field that we can seed into the Ibbaka Talent skill graph, deepening it and developing new connections. When we onboard a new company, especially one in a new industry for us, we often do a skill survey of the company and its industry, in order to seed our system with relevant skills.
Each year, we undertake four to six general pieces of skill insight research. Currently, we are conducting the Pricing Skills survey mentioned above and we are planning additional skill research on Product Manager skills, Talent and Performance Management skills and are investigating a project on Fintech skills and Digital Transformation in the financial sector.
We don't rely on surveys alone. At Ibbaka a skill research project will generally include the survey, interviews, data scraping from a variety of sources, the injection of all of this data into the Skill Graph and a variety on analysis. Our goal is to get a holistic view of current and future skill requirements.
What does a good survey look like? (Survey Design Guidelines)
Over on the LinkedIn Design Thinking Group, there has been a discussion of good survey design. We came up with a few rules for good survey design that we think are generally applicable.
Predictability - it should be clear how to prepare for a survey and how much time will be required. (Some people think all surveys should be short, to paraphrase Einstein, I think a survey should be a short as possible and no shorter.
Transparency - the purpose of the survey should be clear and everyone asked to take the survey should be told how the data will be used and who will have access to it.
Reciprocity - people and organizations that ask questions should expect to be asked questions in return, and to answer them.
Insight - the survey should be designed so that the person taking it gets insights as they are taking the survey.It is this last design rule that interests me. The first three are just common sense and fairness. Design a survey that gives insights as you are taking it is quite a challenge. The survey can give information of course, and that can be valuable in itself, it can suggest the current hot issues in an organization, profession or industry, but none of that satisfies me. I want to design surveys that will help the person taking it pause to reflect, and to think more deeply about the topic. That is what I want from surveys and that is the kind of survey I want to design.
Skill surveys are a good tool to use to frame thinking around building skill profiles. Surveys need to be designed to engage the user and provide value for the respondents and the wider community. When they are applied with other methods, they enable the creation of more holistic skill profiles.
