Big Data, Predictive Analytics and Hiring

During the Winter of 2012-2013, the Boston Red Sox faced a monumental challenge. The team was coming off a last-place, 69-93 train wreck of a season that had sent its fervent fan base into paroxysms of anguish and left many empty seats at Fenway Park after a decade of sold-out games. The Red Sox ownership was well aware that its vaunted brand was fraying at the edges and that Red Sox Nation would have little patience with multiple rebuilding seasons ahead. Entrusted with the task of building a competitive team for the 2013 season was the Red Sox’s young General Manager Ben Cherington.

The temptation to sign high-priced superstars was great. Big names bring excitement and hope. They sell tickets. For Cherington, however, that strategy was anathema. In August 2012, in fact, he had shed more than $270 million worth of payroll when he traded superstars Adrian Gonzalez, Carl Crawford and Josh Beckett to the Los Angeles Dodgers. Rather than creating a dynasty in Boston, those monster contracts had become expensive boat anchors helping to sink the team and its future fortunes. Instead, Cherington employed his own version of predictive hiring analytics to find and sign a group of what the press labeled “complementary” players, productive but generally unheralded players who might perform just well enough to make the Red Sox contenders again. Among those he signed were Shane Victorino, a speedy, talented outfielder; Mike Napoli, a slugging first baseman; Jonny Gomes, an average outfielder; David Ross, a backup catcher; and Koji Uehara, a relief pitcher who had done little to distinguish himself in his years in the major leagues.

Cherington wanted players with specific talents, solid but unspectacular statistics, and that crucial but difficult to quantify locker-room presence that might produce the needed chemistry to field a winner. It was a strategy reminiscent of those described in Moneyball, the famed Michael Lewis book that profiled the “sabermetrics” created by Bill James and employed successfully by Oakland A’s General Manager Billy Beane for more than a decade. Unlike Oakland, a small-market team with limited financial resources, the Red Sox could spend as much as they wanted. But by focusing on these types of players—backed by reams of analytical data compiled by his front office team—Cherington believed the whole would be much better than the sum of its parts. And as even casual sports fans know, the strategy paid off. The 2013 Red Sox shocked all of baseball by rising from worst to first and winning the World Series, its third championship in a decade. The outcome surprised the baseball pundits and Las Vegas gamblers, and proved, among other things, that hiring analytics was a potent method for measuring talent.

Hiring By Algorithm

Big data, the hottest addition to the corporate lexicon, has moved well beyond the baseball diamond, across the business spectrum into the world of hiring and talent management. “For more and more companies,” a recent Wall Street Journal article reported, “the hiring boss is an algorithm.”

The use of predictive hiring analytics is surging. In a time of corporate belt-tightening, the cost of a failed hire is too great to ignore the potential benefits of these new methods. Corporate giants, including Google and Sears, are embracing the power of data and analytics to vastly improve the success rate in talent acquisition and retention, and the days of the traditional job interview are rapidly disappearing in the rearview mirror.

Consider the realities of finding and retaining top talent. Survey data from the Corporate Executive Board, an advisory firm, revealed that nearly a quarter of all new hires leave their company within a year of the start date, according to a 2013 article in Atlantic Monthly. Younger workers are no longer viewing a job as a lifetime commitment, and most are constantly on the lookout for a better opportunity. Thus any tool that helps identify a better hiring decision is going to be invaluable. Gartner, the research firm, predicts that data will grow by 800 percent over the next five years. Search firms such as Monster Worldwide and Korn Ferry are incorporating predictive analytics into their recruiting activities.

According to Dr. John Sullivan, a Silicon Valley-based human resources expert, analytics are superior because traditional HR metrics “are overly simplistic in that they merely report what happened last year. Analytics are superior because they analyze past and current data and reveal patterns and trends” for the future.

In truth, measured talent assessment is not new. “This sounds really different, but in many ways it is something that the field of industrial psychology has been doing for the past 50 years,” said Stu Crandell, senior vice president at Korn Ferry, the global talent management firm. Corporations since the 1950s have used personality and aptitude tests to enhance the interviewing process. But the advent of more sophisticated digital technology has vastly increased the effectiveness of these assessments. According to Crandell, the underpinning of any successful search effort is the assessments that are built upon the research and data, and the current wave of predictive analytics is only as good as those assessments.

“We have developed and improved assessments—which range from online self-assessments to simulation—to interview techniques and integrated all those into algorithms,” Crandell said. “More and more companies are now trying to get on the bandwagon of analytics by saying they have online cloud-based analytics for predictive hiring, but often it means they’ve just developed another online personality test. You have to be careful about sifting through a lot of bad data that is out there.”

That said, Crandell believes that the simulations made possible by technology are a key talent indicator, especially at the most senior levels. “We get to see how someone does in action,” Crandell said. “How well do they influence that authority? How well do they think strategically? Can they coach and develop? This goes beyond what they tell us in an interview. Can they actually do it?”

For example, in one Korn Ferry search for a new CEO for a Midwestern industrial equipment manufacturer, an internal candidate was pitted against an external prospect. The external candidate interviewed impressively with the board and became the front-runner. But when he was put through the simulation exercises, “he turned out to be an empty suit,” said Crandell. “He could talk a good game, but in action, he didn’t know how to execute.” In the past, he likely would have been hired, with dire results. The internal candidate performed far better on that simulation and ended up with the job, which he performed superbly, guiding the company through difficult patches and turning around the business.

Perhaps more important in this move toward Big Data is the ability to analyze and understand the information that is being gathered. Many organizations are doing all types of assessments, but what comes out of the graphs, bar charts and numbers? “It’s very difficult to make sense of it,” Crandell said. “It is critical that the results are made clear immediately. What is all this telling you about the kinds of decisions you need to make?”

Shopping at Sears

At Sears holdings Corporation, the $36 billion retailer with more than 230,000 employees, the move to predictive hiring analytics over the past two years is paying dividends. According to Dean Carter, chief HR officer, Sears has a rich history of pre-employment selection that dates to the 1960s. Its current hiring analytics, however, go well past its traditional techniques. Sears has multiple efforts on this front, but its two primary targets are retail sales staff and executive-level hiring, opposite ends of the corporate ladder.

For Sears, which is in the middle of a major transformation in an attempt to return to profitability (Sears lost a staggering $1.4 billion in 2013), the company is trying to move from a traditional retailer to one that focuses on “members” who join its Shop Your Way rewards program. Part of this overhaul is the corporatewide introduction of predictive hiring analytics.

Sears hires between 140,000 and 160,000 retail sales representatives each year, a large number of those brought in on a seasonal basis during holiday crushes. With 6 million applicants a year, Sears had to find a way to improve its methods. Sherry Nolan, head of talent acquisition, and Don Moretti, head of analytics for talent acquisition, were tasked with incorporating sophisticated analytics into the hiring process. Under the new system, prospective employees fill out an online application, which includes a retail tech simulation test. In this video game-like simulation, an applicant encounters a real, interactive sales scenario and must sell to a variety of customer types, from the angry, impatient customer to the maddeningly indecisive shopper. Navigating through the 35-minute process is actually fun, according to Nolan, and improves the candidate’s job-seeking experience. More important, it introduces potential employees to the reinvented Sears. Candidates are given an opportunity to join Shop Your Way, the customer loyalty program, and even if they are not hired, they may become Sears customers. Sears has established a “talent hive” of more than 2.5 million prospective employees and found a way to generate revenue from the process.

“The first thing we did was to redefine the way our test questions were weighted and defined,” Nolan said. “Most retailers focus on work orientation and reliability. ‘I’ll show up. I won’t steal,’ that kind of thing. But that is just a basic price of entry, and you won’t maintain employment if you are stealing and showing up late every day, anyway. So we shifted our weighting to be about customer orientation and digital savviness. Can you use tools like iPads and tablets to communicate with your customer? And ultimately, we focused on their selling ability.”

This simple reweighting of the assessment cut the number of candidates that eventually would be passed on from 90 percent to less than 60 percent, thus reducing time and cost in the hiring process. The company hires more than 30,000 temporary workers during holiday seasons, and a process that once took three months is now streamlined down to 35 days. More importantly, using analytics on temporary hiring for the first time seemed like common sense. “We said, ‘Why would we want our most precious selling days to be entrusted to associates who we didn’t feel were our best?’ ” said Nolan. The result was a significant uptick in the quality of service and customer satisfaction during the most recent holiday season.

Equally important for Sears is enhancing its talent management at the division vice president level and above. According to Moretti, external candidates for executive positions are measured against the Sears leadership model, which identifies 14 to 17 desired competencies. Nolan and Moretti tested some of the company’s best current executives to understand the competencies they brought to a particular position. Was someone especially resilient or good at strategic thinking? Are they adept at leading a turnaround? By creating a baseline measure for successful current executives, “we might end up with a higher predictability of someone being successful in our organization,” Nolan said.

The result is a “really good road map for us” as a candidate is going through the interviewing process. Nolan calls it the “art and science” of interviewing, which is based on the idea that it is possible to predict whether a prospective executive will enjoy working at Sears, especially during a stressful transformation.

Carter says Sears Holdings is a “moving target, and as we move and change and adapt, the things that make someone successful move and change as well.” Part of the mandate for HR is to find the profiles that “are going to help us be successful tomorrow. So the elements we used three years ago aren’t nearly as relevant today as they were then,” Carter said. “We continually have to look at the elements and test and retest because what meant success three years ago certainly isn’t necessarily going to mean success three years from now.”

Google Leads the Way

At Google, the HR function is called “People Operations,” and under Laszlo Bock, the leader of that organization, Google has become the gold standard for hiring analytics. Indeed, all hiring at Google is based on data and analytics and is guided by a “people analytics team.” Given its meteoric growth—from its founding just 15 years ago, the company has 45,000 employees and is now the world’s second-most-valuable company with a market capitalization of nearly $400 billion—Google is clearly focused on finding and hiring the best and the brightest.

According to John Sullivan, the human resources expert referenced earlier, Google’s workforce productivity is off the charts. Reportedly, on average, each employee generates nearly $1 million in revenue and $200,000 in profits each year. It would appear that Google’s commitment to its unique talent management methods has paid off.

Bock told The New York Times that Google determined that “G.P.A.’s and test scores are worthless as a criteria for hiring. We found they don’t predict anything.” Instead, Google has focused on ways to use data to measure leadership skills, cognitive ability, humility and ownership.

Working under analytics guru Prasad Setty, Tina Malm is a people analytics manager who has been with Google for nearly seven years. With a Ph.D. in industrial organizational psychology, Malm heads a team that focuses on staffing analytics with the aim of introducing more data, analytics and science into the hiring process. Beyond removing much of the gut-feel and guesswork from the hiring process,

Google has four key goals:

• Using analytics to expand the candidate pipeline and bring more talented people into that pipeline.
• Using analytics to improve decision making and identify the best candidates.
• Making the candidate experience remarkable. Every candidate should have a “magical” inter-view experience and a “magical” hiring process.
• Making the hiring process fast and efficient.

According to Malm, Google receives 2 million resumes every year. “And even though we have a very thorough and rigorous hiring process, sometimes strong candidates who would be a great fit don’t get hired,” Malm said. A candidate might have a single bad interview, or the position might have already been filled, or the candidate might be a better fit in a different role. “We don’t want to lose out on these great candidates, especially if it happens for these wrong reasons,” she said.

The people analytics team created a systemic approach to reviewing rejected candidates and established metrics for scoring the resumes of prospects who were turned away. From this, a new list of candidates is created and shared with internal recruiters, staffing teams and hiring managers to decide whether to call back these candidates for an open role. In other words, at Google, “don’t call us, we’ll call you” is not just lip service. According to Malm, the system has generated more than 20,000 return visits for rejected candidates since 2012.

Google has also incorporated best interviewing practices through research. First, it cut down its onerous 12-interview process to no more than four for non-technical jobs and five for technical jobs. Not only was this process incredibly stressful for candidates, but having studied every interview ever done at Google, the group found patterns of predictability that changed the company’s outlook.

“We found something surprising,” Malm said. “Our most predictive interviewer was actually the wisdom of the crowd. In other words, we didn’t find that any particular interviewer group, like senior Googlers or longtime employees, were better at identifying who we end up hiring. It is really the average score of four interviews.” The wisdom of the crowd was correct 86 percent of the time in picking the best candidate, and any additional interview beyond that added only one percent more accuracy.

Google also used analytics to revamp its interviewing process, removing brainteasers and similar challenges because they didn’t find any connection between an ability to solve those puzzles and a successful candidate. Instead, Google identified its best interviewers and introduced consistent rating scales and a common set of questions that each interviewer must ask. Every Googler who does interviews must attend regular training so that they all know what constitutes a bad answer and what makes a really great answer.

The results are stark: Since 2005, Google has halved the hiring-process time, and candidates have a far better hiring experience. And Google has never wavered from its core value: embracing smart people who are excited to do cool things, who love solving problems and love to learn and collaborate with others. “One thing that has never changed is how seriously everyone takes survey results here,” Malm said.

The Human Touch

Predictive hiring analytics is not a panacea; it’s more a potent tool than a magic bullet.

Korn Ferry’s Crandell warned that talent managers who seek to embrace analytics must be aware of the type of data they are employing in their searches. Many companies, he pointed out, are attempting to use predictive analytics without having the depth and breadth of data that comes with effective assessments.

“Arguably something like LinkedIn has a lot of data. But it is broad but not deep.” Crandell said. “We have data about people’s capabilities on a whole range of competencies—strategic thinking, coaching, leading others, building a team, as well as personality attributes and leadership style—that is much deeper and thus more effective.”

And what about the human touch? Tom Davenport, a professor of management at Babson College who has written several books on Big Data and analytics, said, “Analytics are a transformative force of our age. It turns out they improve decision making in all walks of life—not by a huge amount, but there is a little edge to be gained everywhere. In some areas, humans are still pretty good at hiring, and there are aspects of the recruiting process that still need the human touch. It’s folly to hire someone without meeting them and talking to them.”

Dean Carter of Sears Holdings agreed. “One of the things we know about this selection process is that if we went on a purely analytic basis, we’d make very bad decisions,” Carter said. “Conversely, if we went purely on gut instinct, we’d also make very bad decisions. So somewhere in this, the analytics help us raise the bar for the pool that we’re looking at, and then we have to use really great interviewing techniques and analytics to help us make a better decision.”

Even data-centric Google understands the dangers of relying too heavily on the numbers. “We take data and analytics very seriously,” said Malm. “But data is not everything. It can’t be viewed in a vacuum. We all bring unique flavors and personalities into the mix, and we have to ensure that our programs, processes and interviews stay human.”

How talent management will evolve in the age of Big Data remains to be seen. But reliance on a resume and a round of interviews, the popular-but-flawed conventional approach, is going the way of the dinosaur. “With these new approaches, the numbers are all there, and you can eliminate the weirdness and discrimination that goes into personal judgment,” Brynjolfsson said. “If you can get lots of quantitative data, you can really improve the allocation of talent and make people more fulfilled in their careers and companies more profitable by having the right people in the right job.”