Yesterday's Boston Globe published an excellent story by Drake Bennett called "Luck Inc," about the questionable value of books about how to build great companies (the graphic is to the left). Drake provides an excellent summary of the arguments in The Halo Effect that rip apart the methods used in many such books, as well as arguments from Hard Facts, the book Jeff Pfeffer and I wrote on evidence-based management. The story focuses most heavily on new research by Michael Raynor and his colleagues, which apparently shows that luck (i.e., randomness) provides the best explanation for which companies enjoy exceptional performance and are then celebrated as superstars in books like Good to Great and In Search of Excellence. This point makes sense to me, and in fact, follow-up studies of Peters and Waterman's excellent companies and Collins' good to great companies are consistent with that view -- and also consistent with an argument that -- when it comes to picking which stocks will perform best -- a "random walk" is mighty tough to beat -- that most stock pickers don't top a randomly selected stock portfolio.
I had a pretty detailed conversation with Drake, and although most of it focused on the drawbacks of these kinds of books, he ended-up quoting me as defending these books. I think he was completely fair, and in any case, I am on record many places raising concerns about the suspect methods used in both In Search of Excellence and Good to Great. But I have an especially ambivalent reaction to Good to Great -- let me explain why.
There are a lot of things that bother me about the book:
1. It is a very small and flawed sample. Most notably, we have 11 companies that used the practices that Collins celebrates, but the sampling strategy made it impossible to discover how many companies used these practices yet failed to make the leap to greatness.
2. The main method was retrospective -- they would label a company as "great" and then look for articles and do interviews to determine why it happened. This is a quite biased method -- if you ask someone to explain the secrets of their success, you get a certain kind of story that differs from if you ask them to explain why the failed (regardless of actual performance). Winners will report having better leaders, being more focused, and persistent -- and trying to untangle what is part of the sensemaking process versus what really happened is tough.
3. Good to Great cites almost no prior research, even though there are literally thousands of more rigorous studies that are pertinent to claims in the book, especially studies of leadership. Indeed, as knowledge accumulates one study at a time, and there are few if any definitive studies. So any author who claims or implies that he or she has done THE definitive study is immediately suspect -- indeed, it is something that well-trained researchers never do, even Nobel Prize winners. I think that Collins needs to say -- "this is just one study, we learned a lot from it, but it isn't definitive... and it has flaws."
4. Perhaps the biggest problem of all is that Collins makes bold and excessive claims based on the research; ironically, this book about the virtues of modest leaders reveals considerable hubris in its claims. Perhaps that is necessary to get a bestseller -- but, as an example, Malcolm Gladwell would never make such claims.
BUT despite all these concerns, what if Collins had actually reviewed and integrated rigorous research and had built a book based on that body of evidence? If he had done so, he could have found considerable support for his ideas in published peer-reviewed research. Although there is a good deal of randomness in the process, and Collins probably overstates the wallop packed by leaders, the fact remains that leaders do need help (it is a damn hard job), and the simple and compelling ideas in Collins' book are probably mostly right and have probably helped a lot of leaders and managers. Spreading the message to leaders that they need to face facts and to be persistent and humble strikes me as a good thing, and also consistent with studies from diverse places.
So, although it is mediocre research, I think the message has done a lot of good. I just wish that Collins had shown more modesty. The upshot is that I end-up being the one who defends Good to Great at the ending of the article. So I did say, and agree, that ""There's value in mastering the obvious," he says. "If Jim Collins's impact is to get people to do stuff that they know they should do already - facing the hard truths or being selfless or whatever - I certainly don't think that's a bad thing."
As I think about this now, perhaps the most important standard for business books is that, whatever basis is used to support the authors advice should be stated clearly and not be overblown. I don't expect a book by Jack Welch to be based on anything but his experience, and my favorite business book of all time, Orbiting The Giant Hairball, is based only on Gordon MacKenzie's personal experience and opinions -- but you know where the claims have come from.
Indeed, as I think of the books I have written, and things I am writing now, the lesson I take away is that my values and biases do affect what I write, but I also draw as heavily as I can on peer reviewed research, as that is so much of a part of my history and identity. So I am going to start making more clear that what I write is best seen as "evidence-based opinion." I also think that is the most honest way to describe what Gladwell does so well and Dan and Chip Heath do too... management is a craft, requiring a complex mix of experience and evidence. Think of what great doctors do, it is much the same thing. If they ignore the evidence too much, they are making a big mistake, but they also need to take into consideration the particular case, what they and the patient want and value, and their clinical experience. To that end, as Pfeffer and I wrote in Hard Facts, we believe that management will always be a craft, but that evidence needs to play a bigger role in how the craft is practiced -- so "evidence-based opinion"" fits that perspective well.