Do you understand how you know when you’ve found a problem?
Last updated by Lee Hawkins [SSW] over 1 year ago.See historyWhen asked how they spotted a problem, testers often say things like "based on my experience of the product" or "by comparing the product against the specification".
It's worth noting that users often find problems in your software without ever having seen a specification or user story!
So, how do you recognize when you've found a problem? The key concept here is the oracle.
Oracles
An oracle is a heuristic principle or mechanism by which someone recognizes a problem.
Oracles are, by their nature, heuristic. That is, oracles are fallible and context-dependent.
Oracles do not tell us conclusively that there is a problem; rather, they suggest that there may be a problem. There can be no absolute oracle, so we use many. Consistency is an important theme in oracles.
As an example, we can detect a bug when the product behaves in a way that is inconsistent with its specification or user story.
An example oracle
The oracle here is the specification or user story and the problem is that the state of the product is not consistent with that oracle.
The problem might be caused by an out-of-date specification, a genuine bug in the product or something else – or there might actually be no problem at all!
Tip: Take the conscious step to understand which oracle you are using to spot a potential problem.
By doing this, you move away from personal opinion towards proof. This is a key step to enhancing your credibility in the eyes of your stakeholders.
Your bug reports are more credible when stakeholders clearly understand why you are claiming your observation to be a potential problem.
Further reading
- Cultivate Your Credibility With Oracles And Heuristics by Lee Hawkins
