Null Points: Absence of evidence and evidence of absence.

Some were saying the candidate Scotland had entered for the Olympic sailing team was giving them an unfair advantage.

Some were saying the candidate Scotland had entered for the Olympic sailing team was giving them an unfair advantage.

Recently I attended a family wedding in Inverness. That is to say one of the people getting married was (and is) a member of my family rather than both. We’re not that unusual as a clan. While there I took it upon myself to sample some whiskies (Ardbeg was my favourite. I wouldn’t be sad if you sent me some for free.) and of course to visit Loch Ness. It’s a beautiful part of the country which I wholly recommend. Additionally I visited the Loch Ness Centre and Exhibition in, the pleasingly-named, Drumnadrochit. The centre is dedicated to the Loch Ness monster. Probably one of the world’s most famous cryptids, the Loch Ness Monster is reputedly a large, unknown animal that is said to inhabit the loch. The centre is really excellent and takes you through the history of the “monster” and the evidence for its existence which is anecdotal and consists of minimal, much-disputed photographic material and sonar readings. If you get a chance you should go. The gist is that the scientific community mostly regards the Loch Ness Monster as mythical; explaining sightings as misidentifications of other objects or animals, outright hoaxes, and other psychological phenomenon. I don’t think any of that counts as a spoiler.

My favourite stories were as follows: in the 1930s, a big-game hunter named Marmaduke Wetherell went looking for the monster. Mainly because with a name like Marmaduke Wetherall you either have to become a big-game hunter or a raincoat for a large dog. He claimed to have found footprints, but when casts of them were sent for analysis, they turned out to be from a hippopotamus. While Loch Ness itself is not naturally riddled with hippopotami, a hoaxer (it is not known if it were Wetherell himself) had used a hippopotamus-foot umbrella stand or ashtray to make the footprints. Secondly, also in the 1930s, a lady described a huge animal resembling a crocodile with tusks swimming up the river towards Loch Ness. This has been interpreted as being a description of a sturgeon. Sturgeons also have long barbels under their mouths which could be seen as tusks. For completion it’s also been interpreted as being a monster. While there are no records of sturgeon in the river or loch, I find the idea that at least some monster sightings may be sturgeons quite pleasing. Not least because it could mean a lucrative business selling Loch Ness Monster caviar.

A sturgeon being released. I don't know what crime it had committed to warrant imprisonment.

A sturgeon being released. I don’t know what crime it had committed to warrant imprisonment.

All of this got me thinking about the well known phrase, “absence of evidence is not evidence of absence”. In the case of the monster, while the evidence that it does exist is not convincing, there are people that would argue that just because the monster hasn’t been found, it doesn’t mean that it isn’t there. This has the benefit of being technically correct without adding much more to any debate.  Like an instruction manual for a Betamax video player written in Klingon.

Evidence of absence is evidence of any kind that suggests something is missing or that it does not exist. In propositional logic this can be formulated as P implies Q, but Q is false, therefore P is false. This is known as modus tollens. Minding your Ps and Qs is nothing to do with this and is known as good manners. The statement “absence of evidence is not evidence of absence” was made popular by the great Carl Sagan. Sagan referred to such arguments from ignorance as follows:

“Appeal to ignorance: the claim that whatever has not been proved false must be true, and vice versa. (e.g., There is no compelling evidence that UFOs are not visiting the Earth; therefore, UFOs exist, and there is intelligent life elsewhere in the Universe. Or: There may be seventy kazillion other worlds, but not one is known to have the moral advancement of the Earth, so we’re still central to the Universe.) This impatience with ambiguity can be criticized in the phrase: absence of evidence is not evidence of absence.”

In coining this sentence, Sagan isn’t saying that if there is no evidence for something then it must be by default, believed.  He is criticising it as an appeal to ignorance. If you don’t know something and there’s no evidence to tell you about it then you should reserve judgement until there is evidence. Don’t guess and just believe the guess you like best no matter what. Homeopaths. There are lots of things I can’t prove don’t exist; the invisible blue tit that lives in my ear and eats earwax, the purple flee at the centre of the great storm on Jupiter, Paddy McGuinness’ sense of humour, but that doesn’t mean I have to believe in them by default.

The difference between evidence that something is absent and a simple absence of evidence can be difficult to determine.  For example, in the case of a new drug and if it poses a health risk. If scientists were to argue it was safe based on no research it would be considered an informal fallacy. Whereas if sufficient research had been performed, but found no dangers, then to some extent this can be used to argue few or no risks exist. It’s unlikely though that no risk exists. No matter what board game denialists tell you.

A similar idea is expressed in scientific experimentation in the null hypothesis.  This term was first used by geneticist, statistician and not, like his name might suggest, a Beatrix Potter character, Ronald Fisher. According to Fisher,

“The null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation.”

In a scientific experiment, the null hypothesis refers to a general or default position that there is no relationship between two measured phenomena. For example a well thought out point in an article by James Delingpole. Rejecting or disproving the null hypothesis is the primary task in any scientific research. If an experiment rejects the null hypothesis, it concludes that there are grounds greater than chance for believing that there is a relationship between the two (or more) phenomena being observed.  Again the null hypothesis itself can never be proven. If participants treated with a medication are compared with untreated participants and there is found no statistically significant difference between the two groups, it does not prove that there really is no difference. Or if we say there is a monster in a Loch but cannot find it. The experiment could only be said to show that the results were not sufficient to reject the null hypothesis.

Ultimately, in carefully designed scientific experiments, even null results might be seen as evidence of absence. A hypothesis might be rejected if a vital predicted observation is not found empirically. The monster in the Loch, the effects of homeopathy or a film where Sean Bean doesn’t die. Whether the scientific community will accept this null result as evidence of absence depends on a number of factors too numerous to get into here. Overall while “absence of evidence is not evidence of absence” is not a good way to run the beliefs in your life, it is a useful way of thinking about statistics in research. Luckily there is no absence of evidence for that.


3 thoughts on “Null Points: Absence of evidence and evidence of absence.

  1. Great demonstration (intentional?) of the limits of logic, empiricism, “science” to explain anything other than provisionally. A+ IF that was your point. Er… what was your thesis, exactly? You are too funny! OX

  2. Pingback: Evidence, Absence, and the Type II Monster | In the Dark

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s