C&B Notes

Data Overload

Data volume suffers from diminishing marginal returns because of the burden it places on users to separate the relevancy of signal from the distraction of noise.  Psychological biases further complicate the ability to interpret mountains of data.

We like to think that more information drives smarter decisions; that the more details we absorb, the better off we’ll be. It’s why we subscribe to Google Alerts, cling to our iPhone, and fire up our TweetDeck.  Knowledge, we’re told, is power.  But what if our thirst for data is actually holding us back?  What if obsessing over information actually reduces the quality of our decisions?  That’s the question raised by Princeton and Stanford University psychologists in a fascinating study titled On the Pursuit and Misuse of Useless Information.

Their experiment was simple.  Participants were divided into two groups.  Group 1 read the following: Imagine that you are a loan officer at a bank reviewing the mortgage application of a recent college graduate with a stable, well-paying job and a solid credit history.  The applicant seems qualified, but during the routine credit check you discover that for the last three months the applicant has not paid a $5,000 debt to his charge card account.

Do you approve or reject the mortgage application?  Group 2 saw the same paragraph with one crucial difference. Instead of learning the exact amount of the student’s debt, they were told there were conflicting reports and that the size of the debt was unclear.  It was either $5,000 or $25,000.  Participants could decide to approve or reject the applicant immediately, or they could delay their decision until more information was available, clarifying how much the student really owed.  Not surprisingly, most Group 2 participants chose to wait until they knew the size of the debt…  The experimenters then revealed that the student’s debt was only $5,000.  In other words, both groups ended up with the same exact information.  Group 2 just had to go out of its way and seek it out.  The result?  71% of Group 1 participants rejected the applicant.  But among Group 2 participants who asked for additional information?  Only 21% rejected the applicant…  So why would the rate of rejection be three times higher in Group 1?  The answer underscores a troubling blind spot in the way we make decisions.  One that highlights the downside of having a sea of information available at our fingertips, and just might convince you to ditch your iPhone the next time you’re faced with an important choice.

* * * * *

The research underscores a sobering message: We’re fascinated with filling information gaps and that obsession can lead us astray.  Especially today, when reducing uncertainty has become all too easy…  And it’s not just trivial information that’s easily accessible. It’s data that drives major business decisions. There’s always one more report, one more analysis, and one more perspective that’s a click or two away.  Neurologically, information is addicting.  Learning is associated with the release of dopamine, the same as powerful drugs like cocaine.  It’s why we are so vulnerable to an Internet rife with attention parasites that leave us worse for the wear.  In a world where every click brings the promise of a discovery, we are all at risk of becoming addicts.  The challenge lies in differentiating between questions worth exploring and questions best left unasked.

More on the big data trend at Techcrunch.com