Hiring for Smarts
So much has been written about leadership personality and style that hiring managers are in danger of neglecting the most critical factor in executives’ success: intelligence. More specifically, those responsible for hiring and promoting haven’t been given the tools necessary to evaluate the cognitive abilities that allow a person to consistently reach the “right” answer. How could they recognize such smarts? Historically, the only reliable measure of such brainpower has been the standard IQ test, which, for good reasons, is rarely used in business settings. But in rejecting IQ testing altogether, hiring managers have turned their backs on the single most effective assessment of cognitive abilities, simply because there isn’t a version that applies to the corporate world. They have dismissed the one method that could help them identify business stars.
Yes, it’s nice when a leader is charismatic and confident, and a great résumé can tell you a lot about a person’s knowledge and experience. But such assets are no substitute for sheer business intelligence, and they reveal very little about the leader’s ability to get to the truth of the matter. Thinking critically is the primary responsibility of any manager, in any organization, and a leader’s capacity to engage in this process is largely determined by his or her intelligence. Of course, there are many academically brilliant people who might score in the genius range on an IQ test but who could never make it as the CEO of a Fortune 500 company. That’s not surprising, since IQ tests focus on the cognitive skills central to success in school, not success in business.
Nevertheless, there’s a lesson to be learned from the predictive power of IQ tests. That is, to accurately forecast how successful someone will be in a particular activity, you must examine the cognitive skills he or she possesses that directly affect that activity—in this case, in the workplace rather than in the classroom. In this article, I’ll define the specific cognitive abilities that make up what I call “executive intelligence” and describe what to look for when interviewing job candidates or considering a manager for promotion.
The Main Ingredient: Critical Thinking
For many years, management scholars and practitioners have acknowledged that business leaders must be able to think critically. Lucent Technologies CEO Patricia Russo, who has led the company’s turnaround, described this ability to me as “clarity of thought.” The people who have it are rare, she said, but if you get a team of clear thinkers, “the possibilities are endless.” Avon CEO Andrea Jung made a similar observation: “Clear thinking in senior leadership is a primary attribute we look for. I’ve seen little correlation between those who have a formal business education and those who possess clear thinking.…Some people have a knack for this, some don’t.” What Russo and Jung are referring to is a very specific ability—critical business thinking, which is the foundation of executive intelligence.
To better understand this concept, it’s useful to consider a business decision that could have benefited from some solid critical thinking. Let’s look at the introduction of the Segway Human Transporter. The upright powered vehicle was heralded by its inventors as the catalyst for a revolution in human mobility. But despite the hype, the Segway got a lukewarm reception from consumers and has not transformed urban transportation. Could the inventors have anticipated this outcome? A critical thinker might have analyzed the Segway’s market potential like this:
The motorized scooter has been around for a while, is functionally similar to the Segway, and sells for a fraction of the cost. Yet motor scooters have not been widely adopted, and cities have not altered their infrastructures to accommodate this mode of transportation. So why would the Segway succeed where the scooter has failed? The Segway has two advantages over the scooter: Users can stand and balance completely upright while the transporter is moving or stopped, and they can go backward. The basic question remains, though: Did the lack of these two features keep the scooter from being more widely adopted? If not, there is little reason to anticipate any greater demand for the Segway than there has been for the scooter. In fact, because of the Segway’s dramatically higher cost, there may be less demand.
The Segway is fun. But is it worth the cost? Not according to the market. The minds behind the Segway may be sophisticated and technologically brilliant, but they appear to be somewhat naive when it comes to business. Perhaps their excitement over the technology clouded their ability to challenge their market assumptions to any significant degree.
The Segway example is notable because it reveals a fundamental flaw in a company’s business plan. But it isn’t enough to look generally at what went well or poorly with a particular business. To hire potential business stars, we need to understand the basic attributes that lead individuals to make good or bad decisions. We need to understand what constitutes executive intelligence.
Getting Smart About Smarts
In school, students focus on “subjects”—history, math, language, and so on. Similarly, we can identify the subjects of executive work and the distinct set of aptitudes that a manager must be able to demonstrate in each. All managerial work falls into one of three subjects: accomplishing tasks, working with and through others, and judging oneself and adapting one’s behavior accordingly. (For a description of the research supporting these classifications, see the sidebar “Creating a Measure of Executive Intelligence.”) Here’s how executive intelligence is manifest within these three subjects.
Accomplishing Tasks.
In this subject, intelligent executives make decisions using a set of six core cognitive skills. Among them are critically examining underlying assumptions and identifying probable unintended consequences. (For the full list of cognitive skills in each subject, see the exhibit “The Skills That Make Up Executive Intelligence.”) With these aptitudes in mind, consider the way two CEOs accomplished tasks in response to a business crisis.
In the 1980s, General Motors was losing market share to its more efficient Japanese competitors, and, at the same time, it was struggling with terrible labor relations. Then-CEO Roger Smith developed a bold plan to solve both problems by replacing nearly all of GM’s manufacturing force with robotics. By the end of the 1980s, GM had spent more than $45 billion on plant automation—a sum that at the time would have been enough to purchase both Toyota and Nissan. Yet its market share and plant productivity continued to decline every year following automation. To Smith, automation had seemed like such a logical move and, obviously, such high-risk initiatives are very difficult to undertake. But Smith demonstrated a severe lack of executive intelligence in his analysis. First, he failed to question his underlying assumption that more robots equals cheaper cars. A review of readily available data would have revealed that machines entail huge capital expenses and call for highly skilled support technicians. Second, he failed to anticipate the unintended consequences of his initiative: Automation can severely limit a plant’s flexibility and, hence, its ability to change product lines. Had Smith more skillfully analyzed the situation, he might still have chosen to invest in automation, but he could have done so in a way that maximized his chances for success. As Robert Lutz, a senior GM executive, later explained, the best answer to GM’s productivity problems was, in fact, a combination of people and machines that capitalized on the strengths of each.
By contrast, when D. Keith Grossman was hired as Thoratec Corporation’s CEO in 1996, the medical devices company was struggling to survive. Grossman was charged with helping the company profitably produce and market its flagship product, a ventricular assist device for recovery from open-heart surgery. But Grossman was quick to question the industry’s fundamental assumption: that getting a successful product to market would result in a viable business. Thoratec was competing with large, global companies that were focused not just on a single device but on whole diseases, combining drugs and devices. What’s more, those firms had deep pockets to market to doctors and consumers.
Thoratec’s product-focused assumption had created an expensive unintended consequence: the need to build an infrastructure to produce and sell a single product, leading to a severe cost handicap since the same infrastructure would be required regardless of how many products the company was selling. Thoratec could never hope to compete with companies offering vast and integrated product lines. To succeed, Grossman concluded, the company would have to gain scale, either by acquiring another company or by being acquired. Grossman had effectively anticipated key obstacles to achieving the company’s objectives and identified sensible means to circumvent them, another of the cognitive skills required in accomplishing tasks.
Grossman’s articulation of his facts and conclusions was so sound in its logic that he was eventually able to convince the board of rival Thermo Cardiosystems—a company three times Thoratec’s size—to be acquired and even to accept as terms of the deal Thoratec’s stock and management team and a minority position on Thoratec’s board. Today, Thoratec is a thriving, highly profitable company with a virtual monopoly in its medical niche.
Working with and Through Others.
Any discussion about a leader’s “people skills” generally focuses on that individual’s personality, likability, and manners. By contrast, any discussion about a leader’s executive intelligence with respect to people should focus on six core cognitive skills that allow the leader to appreciate and navigate the complexities of interpersonal situations in an intelligent way. Among these are recognizing the underlying agendas of others and considering the probable effects of one’s actions.
Former Boeing CEO Phil Condit, who was blindsided by a series of public scandals, seemed to be lacking these aptitudes. While no evidence links Condit to the managerial mistakes and ethical lapses that ultimately prompted him to resign from his post several years ago, he failed to recognize how far some of his salespeople might go to meet their numbers. Specifically, he never understood that their underlying agendas could compromise their ethics. He didn’t recognize the “ends justify the means” sales culture that was developing around him. Senior members of his team became involved in highly questionable dealings—practices that a more socially aware CEO would have discovered and stopped early on. Further, Condit never appropriately considered the probable effects of his actions as he allowed his salespeople to police themselves. Ultimately, the short-term financial gains that the sales team generated were dramatically eclipsed by the costs to Boeing—in both money and reputation.
Now consider the response of Van Johnson, the recently retired CEO of Sutter Health, when Blue Cross’s meager and untimely payments threatened to bankrupt Sutter’s hospitals and physician groups in 2001. Sutter’s employees were screaming for a fight, but Johnson questioned the prevailing opinion within Sutter (widely shared by other health care providers) that Blue Cross was an evil and greedy corporation bent on squeezing out profits, even if it meant forcing community medical care groups into insolvency. He attempted to understand the underlying agendas at Blue Cross by studying the ways Blue Cross’s business (health care payment and reimbursement) differed from Sutter’s (delivery of care). And Johnson considered the probable effects of a battle with the insurance giant. If Sutter adopted an aggressive stance, the fight with Blue Cross could be costly and protracted and could result in Sutter being removed from Blue Cross’s list of providers.
Johnson realized that while the issue of reimbursement needed to be addressed, the biggest problems between Blue Cross and Sutter had to do with how the companies’ processes worked together; there were lapses on both sides. By accepting Sutter’s role in the problem and approaching negotiations with the other party’s perspective in mind, Johnson emphasized fairness rather than power. The two companies were at odds for approximately three months but ultimately resolved their differences and were able to create the foundation for a solid long-term relationship. In fact, Johnson has described Blue Cross as one of Sutter’s fairest payers.
Judging Oneself.
Effective executives need to be able to cast a critical eye on their thinking and behavior. This involves applying five core cognitive skills, including recognizing personal biases or limitations in one’s perspective and pursuing feedback that may reveal an error in judgment and making appropriate adjustments. It’s natural for people to get defensive when challenged, but an effective leader must be able to assess his or her ideas, test them against those of others, and adjust them if need be. Failure to do this can leave a company highly vulnerable to changes in the marketplace.
Consider the situation at Rubbermaid: It was one of Fortune’s most admired companies in 1993. Just six years later, it was in such bad shape that it was acquired by turnaround specialist Newell Manufacturing Company. What happened? During the 1990s, Rubbermaid’s market changed dramatically, as retailers began to shift their focus from selling innovative products to selling lower-cost products. Rubbermaid’s then-CEO, Wolfgang Schmitt, refused to bend to pressure from the retailers because the company had always done well with price increases. And he steadfastly ignored his team, which tried to explain the changes in the market. Schmitt maintained that customers just needed to be educated about why price increases were necessary. His inability to look critically at his own biases and limitations led him to devalue the essential information that others were trying to provide.
Now look at how another CEO more skillfully addressed his own significant oversights. Cedars-Sinai president and CEO Tom Priselac had made a special effort in his 26-year tenure at the health care organization to establish and maintain personal relationships with managers and staff. He prided himself on these connections. But in late 2002, a union effort to organize the hospital’s nurses opened Priselac’s eyes to the troubling reality that he was out of touch with far too many of his employees. Union recruiters had taken advantage of a growing animosity and a lack of trust between the nursing staff and the administration.
Priselac began an aggressive organizationwide initiative to figure out how the executive team, managers, and staff had grown so far apart—and he insisted that the effort focus on how his own actions had contributed to this rift, despite what he thought were his best efforts to the contrary. Over time, Priselac came to recognize that he’d gotten into a comfort zone; he hadn’t realized that he was no longer having meaningful contacts with people he already knew and that he hadn’t gotten to know many of the new employees. Priselac began meeting with directors, managers, and employees (one-on-one and in small groups) much more frequently. His efforts paid off: About 85% of the participants in a recent employee satisfaction survey (a total of 8,000 Cedars-Sinai employees were polled) rated senior management as fair, honest, and trustworthy. About 92% of respondents said they would recommend working at the organization—a 30% increase in favorable responses from 2002. By late 2004, the union that had been attempting to represent the nurses had withdrawn its petition. How did Priselac engineer such an impressive turnaround? He actively sought and used information that revealed errors in his judgment. His response to the union effort explicitly focused on his own role in creating a breach with employees. And Priselac looked critically at the limitations in his own perspective—namely, how he had unintentionally become isolated.
What do Grossman, Johnson, and Priselac have in common? They faced very different business problems, but each arrived at a solution through the application of certain cognitive skills. These are not the skills you pick up in business schools or executive training programs. Those institutions provide useful techniques for decision making, but most executives have access to all the same tools, and even the best problem-solving models require sharp thinking if they are to be applied effectively. Making sound business decisions requires a form of intelligence—an organic, adaptive, ever-evolving set of cognitive skills.
The Limits of IQ Tests
Until now, the only cognitive skills measured were those initially identified to predict schoolchildren’s academic performance—and traditionally such skills have been measured using IQ tests. Although IQ tests were not originally intended for use in business, studies have shown that these instruments predict work performance at least as well as competency interviews do (the most common assessment tool used today for hiring and promotion) and about ten times better than personality tests do. That’s because some of the thinking skills that support academic success are also crucial to executive performance.
Yet IQ testing is not widely used as a way to identify top talent (though it plays an indirect role, as companies may choose to hire people with degrees from elite schools). The skills that IQ tests assess represent a fraction of a person’s existing cognitive abilities. Some of the skills measured—such as vocabulary, arithmetic, and spatial reasoning—have almost no relevance to managerial work. Moreover, the topics tested would seem academic and elementary—indeed, almost insulting—to people with extensive professional experience.
The format is also ill suited to business. Executives rarely if ever confront problems that have just one right answer; nor do they have the option of picking one answer from several choices listed. IQ test questions don’t assess the practical, on-your-feet thinking skills needed in business. What’s more, these tests have been repeatedly accused of racial and gender bias.
Yet, despite these very real shortcomings, IQ tests are still a better predictor of managerial success than any other assessment tool. The business world’s reluctance to use intelligence testing of any kind (other than assessments of emotional intelligence, which is really about personality and style) has robbed companies of a powerful tool for evaluating candidates for employment or promotion. It is, however, possible to create a comparable measure of intelligence for executives, one that tests for the skills managers need—such as evaluating the quality of data or accurately identifying the core issues in a conflict—and in a format that more accurately emulates the real business environment.
Interviewing for Intelligence
The most common interviewing methodology is the “past behavioral interview” (PBI). A PBI includes questions about a person’s experiences performing certain activities—such as managing deadlines or resolving conflicts—but does not include personal questions. This form of interview has become the accepted best practice over the past 30 years, and, in fact, the PBI is a good predictor of performance. It can explain about 25% of the variances in performance among employees.
Still, PBIs miss a lot of what determines executives’ success. That’s because they don’t measure what they claim to. Take two sample PBI questions. “What is the strategic direction of your company or division, and how did you go about developing it?” is designed to assess someone’s competence at devising strategy. And “Describe a situation in which you had to interact with a difficult colleague and resolve a conflict” is supposed to test a person’s capacity to handle conflicts. Surprisingly, you can just as accurately predict an executive’s ability to devise strategy based on her answer to the second question as you can based on her answer to the first. This is not just a single example unique to these two competency questions; the same circumstance holds true for any competency question.
Research by professors Jesús F. Salgado and Silvia Moscoso of the University of Santiago de Compostela in Spain explains why. A person’s performance on any behavioral interview question is dominated by the same three qualities: experience, job knowledge, and social skills. A candidate with a long work history has lots of compelling examples to draw from when asked to recount events that might illustrate a particular competency. A candidate’s job knowledge—specifically, his awareness of industrial and managerial best practices—can make it easier for him to punctuate his answers with stories that will earn him high marks from interviewers. And a candidate who can relate his stories in a positive, likable manner has a distinct advantage over someone with inferior social skills. Because each question in the behavioral interview essentially assesses the same qualities, there’s no need for the grueling three-to-four-hour sessions favored by hiring managers today. They need only ask enough questions to get a reliable appraisal of the candidate’s work experience, job knowledge, and social skills.
Despite their advantages, behavioral interviews really only establish a candidate’s minimum qualifications; they don’t identify star talent. A candidate’s experience, for example, is obviously an important hiring factor, but we all know seasoned executives who aren’t stars. Similarly, being likable doesn’t mean you have the intellectual horsepower to be a stellar leader. In short, behavioral interviews measure knowledge, not intelligence. Knowledge is information acquired through experience or formal training. Intelligence is the skill with which someone uses knowledge to solve a problem. Knowledge questions require people to recite what they have learned or experienced, while intelligence questions call for individuals to demonstrate their abilities.
So how do you measure executive intelligence? The best way is to use questions that require candidates to demonstrate their skills in an interview format. For such a measure to assess intelligence, it must raise questions and situations that the candidate has never confronted. The more novel the situation, the less rote knowledge can be applied and the more cognitive ability is required to render an answer.
The interview format is a departure for intelligence tests, which have traditionally been presented as written, multiple-choice exams because their developers believed that human judges could not make objective assessments. But that’s not true. The Educational Testing Service recently changed the format of the Scholastic Assessment Test (SAT) to include essay writing. The move was controversial, because it introduced a human element into the judging, presumably making scores less objective. But, in fact, research had shown that other essay-based standardized tests, such as some Advanced Placement exams, were in many cases better predictors of academic success than the SAT. That’s because most university students are graded on essay exams; almost none of their grades are derived from multiple-choice testing. It turns out that the best way to predict how well people will write essays in the future is to test them using an essay format today. In other words, to most accurately predict someone’s performance, you must closely mimic the context in which the individual will have to perform.
The same holds true in the office. Executives exchange information through conversations, questions are posed, and decisions are made on the fly. The most accurate predictor of business performance would have to imitate these dynamics, and human evaluators are far and away the best judges of such interactions.
Rather than concentrating on academic subjects, executive intelligence tests should focus on the particular cognitive subjects associated with executive work: accomplishing tasks, working with and through others, and judging oneself. The questions shouldn’t require specific industry expertise or experience. Any knowledge they call for must be rudimentary and common to all executives. Only then can a hiring manager be assured that the disparities among job candidates are because of differences in their processing power, not in their knowledge. And the questions should not be designed to ask whether the candidate has a particular skill; they should be configured so that the candidate will have to demonstrate the skill in the course of answering the question.
Imagine you want to determine whether someone can critically examine underlying assumptions and can anticipate likely unintended consequences. Rather than ask the candidate to recount an occasion in which she did either of these things, you must present a fact-based situation in which she would need to apply such skills. An executive intelligence evaluation designed to test these abilities might look like this:
You are the CEO of a large software company. Your prices are being severely undercut by both domestic and foreign competitors. Your executive team recognizes a desperate need to cut costs. Your COO concludes that the answer is to outsource most of the company’s programming to foreign subcontractors, thereby reducing labor costs. In fact, your COO has already received a number of bids from service firms in both India and South Korea. What questions do you have about his proposal?
A candidate displaying a high level of executive intelligence while answering this question would explain that the core assumption underlying the COO’s conclusion needs to be confirmed—that is, outsourcing automatically equals cheaper production. She might point out that there may be indirect costs (up-front investment, ongoing customer service, and software development issues) involved with such a move that must be considered. Further, she would cite the probable unintended consequences of the COO’s proposal, such as how using a distant workforce might affect productivity or labor relations.
A candidate displaying a high level of executive intelligence while answering this question would explain that the core assumption underlying the COO’s conclusion needs to be confirmed—that is, outsourcing automatically equals cheaper production. She might point out that there may be indirect costs (up-front investment, ongoing customer service, and software development issues) involved with such a move that must be considered. Further, she would cite the probable unintended consequences of the COO’s proposal, such as how using a distant workforce might affect productivity or labor relations.
You are the general manager of sales and marketing at Rinaldi Manufacturing. You have been with the company only a year when Amanda, one of Rinaldi’s senior account managers, asks to talk to you about concerns she has about the way her immediate superior, Rick, has been handling one of Rinaldi’s best customers. Rick is a senior vice president and reports directly to you. Rinaldi employees are expected to respect the chain of command; they are to raise concerns with their immediate supervisors before bringing such issues to others. You know Amanda has not discussed this issue with Rick. What do you do?
A candidate who answers this question well will recognize that he actually knows very little about what is happening with Rinaldi’s customer and why Amanda has chosen to disregard the chain of command. He will acknowledge and balance the needs of different stakeholders—his employees, on the one hand, and one of Rinaldi’s best customers, on the other. He will recognize the probable effects of his chosen course of action. Whether or not he chooses to meet with Amanda, Rick, or both, he will need sound justification for his choice.
There is no single right answer to either of these questions—which highlights the essential role the interviewer plays in guiding the assessment process. The executive intelligence test doesn’t measure the “absolute” quality of a candidate’s solutions but rather the likelihood that his reasoning skills and lines of inquiry will lead him to a desirable outcome. Different candidates may address the same situation in different ways, and their solutions may be equally successful. It takes a trained interviewer to explore the logic underlying each solution and determine the candidate’s mastery of the cognitive skills involved.
Assessing the Assessment Tool
The executive intelligence measure is a new tool, but considerable evidence supports its validity. In a study conducted in 2002, I interviewed and scored 35 executives from different industries using an executive intelligence instrument. I then asked their peers, subordinates, and superiors to rate the executives’ performance (feedback was anonymous). The correlation between the executives’ interview scores and their 360-degree performance ratings suggested that the executive intelligence scores were strong predictors of managerial success.
In 2004, Western Michigan University’s Evaluation Center, which specializes in the evaluation of assessment instruments, reviewed the executive intelligence tool and all related empirical findings. Led by assistant management professor Jennifer Palthe, the group there concluded that executive intelligence tests measure what they purport to and capture aspects of real-time managerial performance that other methodologies cannot. To date, executive intelligence interviews have been administered to more than 500 senior executives in 18 countries and in seven languages. Language, country of origin, gender, and race have not demonstrated any influence on testing performance.
Executive intelligence assessment is not a magic bullet for leadership evaluation. Like behavioral interviews, executive intelligence interviews can predict about 25% to 30% of the variance in performance between candidates. But these tests measure completely different skill sets from those assessed by behavioral interviews, with almost zero overlap. So together, the two approaches to interviewing offer a much bigger picture of a candidate’s potential—around 50% to 60%. Of course, hiring managers can’t know all of a candidate’s characteristics, even after factoring in personality assessments and references. But given the strong correlation between intelligence and success—and the fact that there’s a way to gear intelligence tests to executives—managers can now interview and assess candidates with much greater accuracy.
Source:
https://hbr.org/2005/11/hiring-for-smarts/
