Can AI Replace Consultants? Signs Point to No.

Artificial intelligence (AI) is the buzzword of the moment, sparking excitement and curiosity. But how effective is it at replicating human work when put to the test?

Generative AI (GenAI) tools like ChatGPT and Microsoft Copilot are reshaping productivity across industries, with some experts predicting that nearly half of current jobs could be automated or transformed. As AI’s impact on work expands, companies will need to explore its potential to disrupt and shape the future of work—but also consider areas where its capabilities might be overstated.

Can AI Do it Alone?

With the advent of large language models (LLMs), new questions arise about their potential to replace or augment the role of human consultants. To dive deeper into this issue, Korn Ferry, in collaboration with students in NYU’s Human Capital Analytics and Technology program, investigated the use of ChatGPT for interpreting and summarizing group-level assessment results. The goal: to see if ChatGPT could enhance or replace human efforts in generating Korn Ferry assessment reports.

The study, which was conducted with the GPT 4.o model, consisted of three stages, each focusing on a different aspect of the report production process. Our research team tested the effectiveness of ChatGPT at each stage and found that its performance varied in generating graphics, company insights, and interpretation materials.

Our key findings include:

Stage 1: Generating Graphics

Can ChatGPT turn data into accurate, eye-catching visuals? We used the GenAI tool to replicate Korn Ferry’s report graphics, such as bar charts and heatmaps. While ChatGPT performed well with simpler visuals, it struggled with more complex ones like scatter plots and spider charts. The varying time required for production highlighted the need for careful data preparation. This stage showed that AI can assist with basic charting but falls short with complex graphics requiring precision.

Stage 2: Creating Interpretation Guides

When turning data into talent insights, can ChatGPT capture the nuances? Our team tasked ChatGPT with identifying strengths, development areas, and actionable insights from data. While the tool quickly highlighted leadership competencies and gaps, the output lacked nuanced language and the structure of human assessments. Supervision and refinement were needed to ensure accuracy and readability, highlighting the challenge of using GenAI for complex insights that require contextual understanding.

Stage 3: Producing Company-Specific Insights

ChatGPT’s final challenge: distill data into clear, concise recommendations like a seasoned consultant. We asked the LLM to generate insights on specific companies and summarize key challenges, leadership changes, strategic priorities, and financials. This was ChatGPT’s most successful application, efficiently synthesizing data from online sources. Although it provided valuable starting points, human expertise was still needed to refine these insights for accuracy and relevance.

Where Intention Meets Action

Although generative AI excelled at gathering and summarizing contextual company information, it struggled with complex visualizations and delivering nuanced insights. The variability in the LLM’s performance underscored the need for companies to plan carefully and evaluate the use of GenAI tools to make sure they align with existing workflows. In doing so, here are three recommendations for leaders to consider:

• Be strategic. Leverage generative AI where it excels (e.g., idea generation, basic graph creation, and initial content drafting) while acknowledging—and planning for—its limitations.
• Develop a structured approach to GenAI. Establish clear prompt guides and iterative processes to refine outputs, ensuring thorough review for accuracy and completeness.
• Consider the trade-offs. Balance potential efficiency gains with the risk of reduced output quality, recognizing that LLMs may not always be the most effective tool for every task.

Hold On to Your Consultants 

The question has shifted from "What can AI do?" to "What can we do with AI?"

AI can’t yet stand on its own—and it doesn’t have to. The role of a human consultant remains critical as ChatGPT struggled considerably with more complex tasks and producing work in an engaging, human-like tone. By combining AI’s capabilities with the deep expertise and nuanced understanding that only human consultants can offer, we can promote innovation and shape the future of work.

AI-powered tools can get the job done, but it’s the human-powered insights that turn good into great.

Wonder if AI will ever be capable of replacing humans? We have the answer. Read our article, Human or AI?: The conscious agent.