When someone who has served on the boards of 21 firms over the last 62 years starts questioning the judgment of independent directors, the corporate world takes notice.

Warren Buffett, the investing icon and CEO of the conglomerate Berkshire Hathaway, says too many directors who, on paper, are independent because they don’t have any ties to the organization, are too cozy to company management. CEOs, Buffett complains, can dangle a hefty annual paycheck to a potential independent director in exchange for his or her cooperation on major issues such as acquisitions and restructurings. “When seeking directors, CEOs don’t look for pit bulls. It’s the cocker spaniel that gets taken home,” Buffett writes in his recent letter to shareholders.

But board experts wonder how widespread the problems Buffett describes are. “In today’s corporate governance environment, that may be overly harsh and not completely accurate,” says Joe Griesedieck, a Korn Ferry vice chairman and member of the firm’s Board and CEO Services practice.

Buffett’s complaints against directors come against a backdrop where directors are facing increasingly complex problems on which to question or advise to a company’s C-suite. Organizational culture, talent strategy, and cybersecurity are just three of an ever-expanding list of topics that directors are expected to absorb and give feedback on to management. It’s their willingness to challenge management that Buffett questions.

The “Oracle of Omaha” believes that director pay, between $250,000 and $300,000 a year—plus stock—is a major incentive for non-wealthy directors to agree with management. If a director is seen as being too aggressive, he or she may not be renominated, or in some cases won’t receive the CEO’s recommendation to serve on another company board. “Director compensation has now soared to a level that inevitably makes pay a subconscious factor affecting the behavior of many non-wealthy members,” Buffett writes.

Experts, however, say Buffett may be giving CEOs too much power. For one thing, while CEOs do get a say about new directors, it’s the board’s nominating and governance committee that actually does the selecting. The directors on that committee need to be willing to ask director candidates tough and constructive questions, Griesedieck says. Organizations can supplement that by, every three to four years, bringing in an outside independent group to conduct a board effectiveness review. “That can also help identify directors who are not pulling their weight in this regard,” Griesedieck says.

As for director pay, it certainly is a contentious issue, but maybe not for the same reason Buffett mentions. Several shareholders have recently sued firms for providing “excessive” compensation to non-employee directors. And some directors could be influenced by pay, Griesedieck concedes.

But CEOs don’t actually have control over pay, either. Director pay has been growing around 3% a year for a decade. Most organizations base changes to their pay programs on peer group medians. A company’s total revenue plays a big factor in director pay, as does the industry. But even then, most directors make about the same amount. For example, a Korn Ferry study found a difference of only $52,000 in total pay for directors in the 75th percentile versus those in the 25th percentile. By contrast, the difference in total pay for CEOs in the 75th percentile versus those in the 25th percentile is around $8 million.

The complaints notwithstanding, experts say Buffett does point out that directors have a singular, consistent challenge: to find and retain a talented, ethical CEO who, in Buffett’s words, “will be devoted to the company for his or her business lifetime.” That doesn’t always happen, of course; business schools are littered with case studies of boards that made bad CEO hires or were complicit in the CEO’s questionable habits. “Few can legitimately take Warren Buffett’s board governance experience to task,” says Alan Guarino, a vice chairman in Korn Ferry’s Board and CEO Services practice. “He is spot-on that selecting and guiding the CEO is job number one for board members.” Part of that role includes having the courage and confidence to challenge CEOs on a regular basis. That