We live in a world where everyone needs to learn how to make informed decisions with incomplete and sometimes conflicting data. And that’s a big challenge because humans are inherently bad at making decisions.  And if you want proof, just visit Las Vegas.  Yes, Las Vegas has figured out how to monetize our stupid human decision-making (SHDM) flaws (Figure 1). 

Figure 1: Data-driven by Dilbert (Scott Adams)

Figure 2: Human Decision-making Flaws

Unfortunately, every day we are asked to make important, sometimes life-or-death decisions with imperfect and/or conflicting data.  There is a whole class of decisions, preventive decisions, which are proactive actions taken to minimize the probability of serious injury or death.  Examples of preventive decisions include:

• Wearing a seat belt
• Wearing a motorcycle or bicycle helmet
• Wearing eye, face, and head protection
• Wearing hearing protection
• Wearing gloves
• Not smoking
• Not drinking and operating machinery
• Safety lock for guns
• Wearing safety coats and aprons
• And, of course, getting vaccinated.

Preventive decisions can be particularly tricky, especially in highly emotional (politicalized) situations where the costs associated with being wrong are distorted.  We are seeing this situation play out in the real world with the COVID vaccination resistance (Figure 3).

Figure 3: COVID-19 Decision-making Extremes

“Nobody’s right if everybody’s wrong” – “For What It’s Worth” by Buffalo Springfield

For my class at Menlo College, I am creating an exercise where the students will explore how to make informed decisions in situations of imperfect or even conflicting data (using the decision about wearing seatbelts).  The exercise starts by reviewing a process for making informed decisions in imperfect situations.  I call this “Decision Literacy.”

Decision Literacy

Whether we are overtly aware of it or not, everyone creates a “model” to guide their decision and gathers data as appropriate for the scale or importance of that decision.

• Some models can be quite extensive in gathering a broad range of data in making the decision, such as buying a house, buying a car, or deciding where to go on vacation.
• Other models are just straight-forward “rules of thumbs” or heuristics such as changing the oil in your car every 3,000 miles, seeing a dentist every 6 months, or changing your underwear at least once a week.
• And, unfortunately, some decisions are based upon the loudest, most repetitive voice in the room. To quote Nazi Propaganda Minister Joseph Goebbels “If you tell a lie big enough and keep repeating it, people will eventually come to believe it.”  Sad, but true.

Framing the Decision to be Made

To aid in making informed decisions in imperfect situations of incomplete, conflicting, and/or distorted data, we can embrace the Decision-making Framework of Figure 4.

Figure 4: Informed Decision-making Framework

Some steps in the Informed Decision-making Framework:

• Identify & Triage Decision to Be Made. The starting point for decision making process is to create a hypothesis[1] around which we are seeking to make a decision. Clearly state Decision Hypothesis that you wish to assessment, and KPIs against which Decision progress and success will be measured
• Create MECE Decision Matrix. Construct the mutually exclusive and collectively exhaustive (MECE) Decision Matrix that maps actions to potential outcomes.
• Research and Gather Data and Statistics. This step is tricky in that the sources and credibility of the data and statistics must be objectively  Research and gather data and statistics for each cell of the MECE Decision Matrix with an assessment of the data reliability.
• Create Cost-Benefit Assessment. This is where a significant amount of time should be spent, especially if the decision has high consequence. Identify the direct and indirect costs and benefits of each cell of the MECE Decision Matrix.
• Explore Worst Case Scenarios. This step doesn’t get nearly enough attention in the decision-making-process.  Identify ramifications and costs of decision model being wrong; that is, what are the costs associated with making a wrong decision (sickness, death, embarrassment, a better world).
• Create Clear Presentation. Build analytic visuals and graphics that present the assessment in a format that enables the decision maker to make an informed decision.

The Importance of Properly Framing the Decision to be Made

How one frames the decision becomes important in the data gathering part of making an informed decision.  Are you trying to gather data to determine the right decisions or are you gathering data to support the decision that you have already made? 

• Many folks succumb to confirmation bias – the tendency to interpret new evidence as confirmation of one’s existing beliefs or theories – and only seek data that supports the decision that they have already made.
• Finally, careful consideration must be given in presenting the resulting analytics in an unbiased way that helps us to make that informed decision.

To reiterate this critical point in trying to make informed decisions:

A true data scientist frames the hypothesis so that we can learn what decision is best versus framing the hypothesis to support the decision that we have already made. 

There is a big difference!

Befriending the Decision Matrix

To evaluate our decision options (to wear or not wear a seatbelt, to get or not get vaccinated) requires the following steps:

1. Identifying the different action-outcomes combinations
2. Brainstorming, identifying, and validating the associated benefits and costs of making a decision for each action-outcomes combination
3. Gather relevant data (even though the data may be incomplete) carefully considering the reliability, trust, and transparency of the data source
4. Analyze, assemble, and present the data in a manner that enables making an informed decision considering the multiple action-outcome combinations

The Decision Matrix can guide us in the data gathering and analysis process (Figure 5).

Figure 5: Decision Matrix

Objectivity is everything.  If you come into this process with your mind already made up, then you will only find data that support your position and fabricate reasons to ignore the data that run counter to what you already believe.  If you have a vested interest in a certain decision outcome, then your objectivity is threatened, and the results of your analysis are likely to be biased.

Informed Decisions Summary

The COVID pandemic has frustrated many of us who believe in data and facts to make informed decisions.  Being able to ingest new facts, toss out outdated facts, and ignore the droning of so-called “expert’s” self-serving and skewed opinions is the key to survival, not only as professionals, but as a species.

Finally, the decision process needs data that users can trust in order to make informed decisions. And having data that one trusts is part of our problem today. Even if the data comes from trustworthy sources, it’s very easy to take to social media and scream #FakeNews and create unjustified fear in the data. That’s why Critical Thinking is critical to making informed decisions.

Figure 6: The Key Aspects of Critical Thinking

Critical thinking is more important than ever in a world where the media is constantly screaming their point of view (many times just to drive engagement, views, and profits).

The key to your bottom-line (and not someone else’s), is to think for yourself.  Period.

 

[1] A hypothesis is an assumption or an idea that is proposed for the sake of argument so that it can be tested to see if it might be true.