Generating 600 Ideas to Get 18: Failing Forward at The Onion

One of the hallmarks of creative people, teams, and organizations is that they accept failure and view it as an essential part of their life.  That is why, as Diego Rodriguez and I like to say, failure sucks but instructs.  There is growing evidence that people learn more from failure than from success -- although an important caveat is that people seem to learn the most when they review  both successes and failures that occurred during an experience.  Regardless, whether it is venture capitalists, pharmaceutical researchers, or product designers that you are talking about, there is always a high failure rate in creative work. I used the the example of toy design at IDEO in Weird Ideas That Work.  Note that Brendan Boyle, the star of this story, is still at IDEO and they still design toys and other products (and experiences) for kids, but Skyline has now been integrated into the rest of the company.

Here is what I wrote back in 2001:

Brendan Boyle is founder and head of Skyline, a group of toy designers at IDEO in Palo Alto, California Boyle provides compelling evidence that innovative companies need a wide range of ideas and that success requires a high failure rate.  Boyle and his fellow designers keep careful track of the ideas they generate in brainstorming sessions and informal conversations, and that just pop into their heads.   Skyline keeps close tabs on its ideas because it sells and licenses ideas for toys that are made, distributed, and marketed by big companies like Mattel and Fisher-Price.   Boyle showed me a spreadsheet indicating that, in 1998, Skyline (which had fewer than 10 employees) generated about 4000 ideas for new toys.  Of these 4,000 ideas, 230 were thought to be promising enough to develop into a nice drawing or working prototype.  Of these 230, 12 were ultimately sold.  This “yield” rate is only about 1/3 of 1% of total ideas and 5% of ideas that were thought to have potential.   Boyle pointed out that the success rate is probably even worse than it looks because some toys that are bought never make it to market, and of those that do, only a small percentage reap large sales and profits.  As Boyle says, “You can’t get any good new ideas without having a lot of dumb, lousy, and crazy ones.  Nobody in my business is very good at guessing which are a waste of time and which will be the next Furby.”

I use this example a lot in talks on innovation because Brendan was kind enough and brave enough to give me information about his team's failure rate, but unfortunately -- although I am always asking for this kind of information -- most people decline to give it to me (sometimes they say it is propriety information, other times it is clear they just don't want to have information out there about their failures).  But I just accidentally ran into another example on the radio show, "This American Life," in an episode called "Tough Room," which aired in February.  Check out this podcast, notably "Act One: Make 'em Laff," which is about the creative process at The Onion, the famous fake news organization. It describes and has audio of the sessions where the writers pitch headlines for Onion stories to their fellow writers.  As host Ira Glass says, this is a tough room where most ideas being shot down immediately and, even those that strike people as funny at first usually don't make it into print.  According to the story, to get the 18 headlines they need for each week's edition, the writers usually propose about 600.  This is actually a higher success rate than IDEO's toy group (about 3% survive), but printing a bad story is a lot cheaper than launching a bad product.  I found the other nuances to be fascinating too -- especially the constructive conflict and criticism in the group and the tensions between veterans and newcomers. 

Also, I found the above picture of the white board with lists of possible future Onion stories on the This American Life website -- look at all those ideas, that is what creativity looks like.  

P.S. As you may recall, The Onion ran this story mocking researchers who discovered that assholes are bad for employee morale, which sure sounds like a parody of The No Asshole Rule to me (at least I hope so).

Comments

[Ben]

Hi Bob thanks for the interesting post.

I'm with the first commenter - Paul To - I think "fail" isn't the best word to use here. Off the top of my head I can't think of better word or phrase.

A few months ago at work I was having inconsistent results in completing one of my regular tasks. I spent four days on refining the steps that I followed and the software I use. I finally found the best order of process steps and software.

I never considered the "failures" as failures - instead they taught me the best way to complete this regular task.

So in the case of "The Onion" if 600 proposed headlines yield 18 usable headlines, the other 582 help the writing team reach the usable headlines.

I'll finish off with one of all time favourite quotes from Thomas Edison - "I have not failed. I've just found 10,000 ways that won't work."

Cheers,

Ben

[Wally Bock]

Wonderful post, Bob. Innovation is one of those things that companies seem to believe they can engineer the messiness and humanity out of.

One of the Holy Grails in business is an innovation process that produces only successes. As you note here, innovation follows a power law. You need a thousand or so ideas to get maybe a hundred, maybe fifty worth considering further. Of those you may come up with five or ten that are worth an experiment with real money. And, if you're lucky one of those will work and 10 percent might be breakthrough.

The success/failure dichotomy makes it hard to generate trials, especially of operational ideas. A better framework is the one at Koch Industries where new ideas/practices are seen as experiments with learning as the outcome.

[Melissa Miranda]

Hi Bob,

I love this post- I do it everyday as a product manager at The Auteurs.

There are two kinds of failure: the early small ones that get the conversation/iterative prototyping going. And the big disastrous ones that often happen from locking yourself in a closet and trying to create something perfect in a vacuum.

As a product manager, I embrace the early failure. It's freeing to present a quick, ugly mockup with terrible text that my team tears apart. It gets the conversation and iterative process going automatically (they can't help but fix the blatantly bad). The more perfect I strive to be initially, the less viewpoints I'm able to bake in up front when the design is still flexible. This saves me a lot of time.

Hence deliberately failing early and often is utterly strategic to avoiding big disastrous failures down the road.

[Paul To]

Hi Bob:

Long time no talk. Great post!

Just wondering though, is there an alternative to the word "Fail"?

After taking your course @ AEA/Stanford, I often use "Fail Early, Fail Often" as the tag line to motivate the research and prototyping teams that I manage. A portion of people though, reacted and said we should not encourage or reward failures.

"Failing Forward" is a possible alternative. I started using the phrase "Engage-Learn-Morph"...

Thoughts?