If you're anything like me, statistics and their validity is a mess of gobbledy-gook and hard to apply concepts.
Surveys in the HR Market
I was complaining to a colleague about the heaps of misinformation in the marketplace. She was quick to correct me. “Marketing surveys delivered by vendors and consultants provide valuable information. You just have to remember the source.”
“Sure,” I said, “but by the time it’s quoted in the fifth infographic on the subject it’s been transformed from marketing material to something resembling a fact.”
“It’s not like the people who consume and regurgitate marketing info aren’t culpable,” she said. “Nature abhors a vacuum. People want certainty. It’s not possible to actually do a representative survey. That’s a recipe for confusion and abuse. The market behaves predictably.”
This is the International Year of Statistics.
If you’re anything like me, statistics and their validity is a mess of gobbledy-gook and hard to apply concepts. It makes me nervous to consider writing about the subject. (I have found the recent best seller Naked Statistics: Stripping the Dread from the Data
to be a useful crutch).
If you want to do a survey that has meaningful results, you have to acquire a random sample of the universe you want to understand. The whole idea is that by randomly picking the survey subjects, you will get a result that is a meaningful representation of the whole.
Shortly, the CareerXRoads team will release their annual Source of Hire report (here’s the slideshare) The data comes from between 25 and fifty very large employers (between 1,500 and 10,000+ employees). As interesting as the data is, it doesn’t really tell you anything about the market as a whole. It just averages out the practices of a few employers.
The actual report always includes a disclaimer like this on its first page
“If the reader assumes that the data sliced and diced in this whitepaper is truly representative of where firms find their hires in the US, then you will have missed our point entirely. Indeed, this whitepaper, which we have published now for nearly a decade, is constructed as a lab report to examine the problems and the promise of how well corporations measure one part of the staffing process.”
In other words, CareerXRoads is completely up front about the fact that their data is not representative.
Still, the stats are cited and recited as gospel truths. (You can probably find some of that in the pages of the HRExaminer).
The same thing happens with surveys generated by SilkRoad, Jobvite and Bullhorn. Jam packed with interesting anectdotes, the surveys (and their owners) lose sight of the fact that the work isn’t particularly scientific or representative.
For the most part, these entertaining documents are surveys of customers who agreed to take surveys.
Would it surprise you to discover that 90% of the customers of the local car dealer drive cars? Probably not. This is more or less the same thing these vendors are telling you with their pronouncements of social recruiting success and effectiveness.
Those surveys are among the most effectivepieces of marketing material I’ve ever seen. And, they are useful for understanding what’s possible. They make great reading for anyone trying to figure out hthe technology.
They just don’t prvide an interesting picture of what ‘everyone’ is doing. Real market adoption of social recruiting is not significant yet. In fact, claiming that it is is one of the things currently holding it back.
