Data-Driven Decision Making
Making decisions with data isn’t just about having dashboards and reports—it’s about understanding the cognitive biases that affect our judgment, even when we have perfect information in front of us.
This chapter explores why decision-making matters, the common traps we fall into, and frameworks to make better choices.
Why Decision-Making Matters
As data professionals, we often focus on building models and pipelines.
But the ultimate goal is to inform decisions.
And decisions are made by humans—who are susceptible to biases.
The Cost of Cognitive Load
- Decision fatigue: As the number of options increases, decision quality decreases
- Analysis paralysis: Too much data can be as harmful as too little
- Satisficing: We tend to pick “good enough” rather than optimal when overwhelmed
The best dashboards don’t just show data—they reduce cognitive load and guide action.
The Two Systems of Thinking
Based on Daniel Kahneman’s work:
| System | Type | Characteristics |
|---|---|---|
| System 1 | Fast, intuitive | Automatic, effortless, prone to biases |
| System 2 | Slow, analytical | Deliberate, effortful, logical |
Data-driven decision-making aims to engage System 2—but our biases often hijack the process.
Common Decision-Making Traps
These biases affect everyone—including analysts and data scientists who should “know better.”
1. Sunk Cost Fallacy
“Why stop now when we’ve invested so much?”
We continue investing in failing projects because of past investments, not future value.
The Fix: Ask yourself: “If we weren’t already invested, how much would we invest NOW?”
| Biased Thinking | Data-Driven Thinking |
|---|---|
| ”We’ve spent 6 months on this model" | "What’s the expected ROI from here?" |
| "We can’t abandon the project now" | "What’s the opportunity cost of continuing?“ |
2. The Endowment Effect
“What I have is worth more than what I could have.”
We overvalue things we already own—including processes, tools, and legacy systems.
Examples in data work:
- Keeping a slow ETL pipeline because “it works”
- Refusing to migrate from Excel despite clear limitations
- Overvaluing a model you built vs. an off-the-shelf solution
The Fix: Regularly ask: “If I were starting fresh today, would I choose this?“
3. Status Quo Bias
“It has always been done this way.”
The preference for the current state, even when change would bring better outcomes.
Why we default to status quo:
- Cost of change: Real and perceived switching costs
- Stability preference: Change feels risky
- Selection difficulty: More options = more cognitive effort
- Anticipated regret: Fear of making the “wrong” choice
The Reversal Test:
A powerful technique to detect status quo bias:
- Someone says: “Increasing parameter P will have bad outcomes”
- Ask: “Would decreasing P also have bad outcomes?”
- If yes to both → Ask: “Why is the current value of P optimal?”
- If they can’t explain → Likely status quo bias
4. Confirmation Bias
“The data confirms what I already believed.”
We seek out, interpret, and remember information that confirms our existing beliefs.
In data work:
- Cherry-picking metrics that support our hypothesis
- Ignoring outliers that contradict our model
- Stopping analysis when we find supporting evidence
The Fix:
- Actively seek disconfirming evidence
- Ask: “What would change my mind?”
- Have someone play devil’s advocate
5. Anchoring Bias
“The first number I saw shapes all my estimates.”
Initial information disproportionately influences subsequent judgments.
Examples:
- A stakeholder mentions “I expect 20% improvement” → All your analysis gravitates toward 20%
- Last year’s forecast becomes this year’s baseline, regardless of changed conditions
The Fix: Generate your own estimate before hearing others’ opinions.
Frameworks for Better Decisions
The Pre-Mortem
Before launching a project, imagine it has failed. Ask: “What went wrong?”
This surfaces risks and blindspots while there’s still time to address them.
Decision Journals
Track your decisions and their outcomes:
| Date | Decision | Reasoning | Outcome | Lessons |
|---|---|---|---|---|
| 2024-01-15 | Chose Model A over B | Better accuracy on test set | Model A underperformed in production | Test set wasn’t representative |
The 10/10/10 Rule
Ask yourself:
- How will I feel about this decision in 10 minutes?
- How about in 10 months?
- How about in 10 years?
This helps escape short-term thinking and emotional reactions.
Start from a Blank Page
When evaluating existing processes or systems:
“If we were building this from scratch today, with everything we know now, what would we build?”
This bypasses status quo bias and endowment effect.
The Role of Data in Decisions
Data doesn’t make decisions—people do. But data can:
| Data Can… | Data Cannot… |
|---|---|
| Reduce uncertainty | Eliminate uncertainty |
| Highlight patterns | Guarantee future outcomes |
| Challenge assumptions | Override human judgment |
| Quantify trade-offs | Make the decision for you |
The Data-Informed Mindset
- Hypothesis-driven: Start with a question, not a dashboard
- Probabilistic thinking: Embrace uncertainty, don’t hide it
- Counterfactual reasoning: “What would have happened if…?”
- Feedback loops: Track decisions and learn from outcomes
Key Takeaways
- Biases affect everyone—awareness is the first step
- Sunk costs are sunk—focus on future value, not past investment
- Question the status quo—use the Reversal Test
- Seek disconfirming evidence—fight confirmation bias actively
- Use decision frameworks—pre-mortems, journals, blank-page thinking
- Data informs, humans decide—build systems that reduce cognitive load
The goal isn’t to eliminate human judgment—it’s to make sure our judgment isn’t hijacked by cognitive shortcuts that served us in the savannah but fail us in the boardroom.