A really good talk about debugging.
Talk: Debugging with the Scientific Method, S. Halloway
- Github:
https://github.com/stuarthalloway/presentations/wiki/Debugging-with-the-Scientific-Method
- Talk:
https://www.youtube.com/watch?v=FihU5JxmnBg
People see debugging as an occult art, when often it is the easiest part of software development. Debugging is a perfect closed world in which to apply the scientific method.
The scientific method integrates:
- problem statements
- testable hypotheses
- observation
- refinement
Read Andreas Zeller’s excellent Why Programs Fail, particularly chapters 5-7 and 11-14.
Zeller is also the instructor for the Udacity course on debugging.
David Agans’ Debugging book is a good introduction to many of the same ideas.
Blithe Rocher gave a nice Scientific Method of Troubleshooting talk at Distill 2014.
TL;DR
+------------+ something is wrong
| FAILURE | <-----------------------+
+------------+ |
| |
| why |
v |
+------------+ |
+----------> | HYPOTHESIS | <+ |
| +------------+ | |
| | | |
| | suggests | |
| v | |
| +------------+ | needs refinement |
| falsified? | EXPERIMENT | | |
| +------------+ | |
| | | |
| | produces | |
| v | |
| +----------------------------------------+
+----------- | OBSERVATION |
+----------------------------------------+
|
| ?
v
+------------+
| THEORY |
+------------+
Definitions
FAILURE:
- lack of success
- omission of expected action
HYPOTHESIS:
- a proposed explanation made on the basis of limited evidence as a starting point for further investigation
EXPERIMENT:
- test, trial of tentative procedure
OBSERVATION:
- active acquisition of information from a primary source
falsified?:
- deductive process using modus tollens:
needs refinement:
- The process of removing impurities or unwanted elements from a substance
THEORY:
- A hypothesis offering valid predictions that can be observed
CAUSE:
- an event preceding an effect without which the effect would not have occurred
ACTUAL CAUSE:
- difference between the actual world and the closest possible world in which the effect does not occur
FIX:
- an experiment that establishes an actual cause
WHAT TO DO
-
clear problem statement
-
efficient hypotheses
-
good experiments
-
useful observations
-
writing it all down
Problem Statements
- steps you took
- what you expected
- what actually happened
Good Experiments
- reproducible
- driven by hypotheses
- small
- change only one thing
- try to keep the experiment environment clean (no gulp, pluggins, ect)
Making Observations
- understand all the outputs
- suspect correlations
- use good tools
Write It Down
- problem statement
- hypotheses
- what experiment should show
- why experiment even make sense
- observations
Software Specific
- work at a high level of abstraction
- the failure is not the defect
- RTEFM, a bug start as an “unknown unknown” so that means read the entire manual
- don’t trust, reproduce
- automate
Cheers and happy coding