Civilizations, until Renaissance, suffered from dogmatic & authoritarian beliefs. Scientists & Philosophers of that age, like Galileo (founder of modern experimental science) had to work hard to pull the society out of dogmatic clutches. The narrative was gradually shifted from “trust what you’re told” to “trust what you see” – evidence! This movement gradually built up across the works of art, science, philosophy and took the shape of the Enlightenment called, Renaissance. People started questioning dogma. The quest for science catapulted mankind to a ‘elevated’ Industrial Society from an agrarian society.
The over-extension of the good idea
As with most good ideas, the idea of “trust what you see” got over extended to an illogical overuse and misuse. Statistics entered the scene. Concepts of statistics – sampling & correlation joined hands with the necessiity of show evidence in the name of science, lead to an outburst of empirical approach to science. The rough-cut approach that got adopted is this:
- Form a sample – of humans if possible, else rats
- Set an experiment testing correlation between two variables – say salt & blood pressure
- Apply statistical tools and derive relationships
- publish in science journals
There’s an ocean of such “lab-test” reports. PubMed, a platform accessing references and abstracts on life sciences and biomedical topics, boasts of more that 35 million citations. A simple query of “salt and blood pressure” yields over 21 thousand citations.
Now, on the face of it, there’s nothing wrong in this and shows the rigour with which the hypothesis that excess salt caused high blood pressure is tested. That’s what science is all about, right.
So, What’s The Problem?
We have gone too far in asking for and giving evidence in the name of science….to the extent that science is construed as nothing but lab tests. This approach has infested all streams of “health” science – medical, nutrition, ageing etc. The society, at large accepted dogma as a norm in the pre-Renaissance days; now it refuses to accept any theory as science, unless it is ‘backed’ by evidence. It sounds very scientific to ask for evidence before accepting a nutritional/health advice, no? No!
“The present trend in the so-called empirical investigations into the sociology of the natural sciences is likely to contribute to the decay of science. Superimposed upon this danger is another danger created by Big Science: its urgent need for scientific technicians. More and more Ph.D. candidates receive merely technical training in certain techniques of measurement. They are not initiated into the scientific tradition, the critical tradition of questioning, of being tempted and guided by great and apparently insoluble riddles rather than by the solubility of little puzzles.”
– Karl Popper, The Myth of Framework
Before I list down some of the problems with the prevailing approach of science, I want to put forward, my views on what a good approach to science is.
What is Science?
Science is a set of our best explanations of natural phenomena. These are our best explanations among various theories all the other explanations being not good enough, or loose or vague or outrightly wrong. For example, Darwin’s Theory of Evolution is science as against, “God created all species”. It is to date, our best explanation of how life emerged on Earth and expanded through blind variation & selective retention based on fitment to the prevailing environment.
In the context of this article, Science of good health is a set of our best explanations of how human body works, how it stays healthy and what leads it to deterioration. It explains neatly, with “hard to vary explanations” how the current design evolved and under what conditions it works well.
These best explanations for the accepted theories which are then tested for errors. Theory comes first and only then it is critiqued and tested for its validity; never the other way around! On 11th Feb, 2023, Naval Ravikant interviewd David Deutsch, where Naval says, “So testability can’t be arbitrary testability. It has to be within the context of the explanation and has to arise from the explanation.”
This is a very important point. Theory & testing might appear as chicken & egg situation to the uninitiated. But, it is always theory or a hypothesis that comes first. Testing without a theory is like putting a cart before the horse!
“All observations are theory-impregnated, and that their main function is to check and refute, rather than to prove, our theories.”
– Karl Popper, The Myth of Framework
The testing is for invalidity of a theory, not for its validity. Validation of a theory through a test, no matter how many times, does not prove the theory correct. However, even a single invalidation is good enough to find fault in the theory.
A Typical Scientific Process
- Conjecture of explanations or hypothesis or theories
- Refutation of weak/wrong explanations either through critical deductive analysis or through empirical evidence or experimentation
- Proposing the best prevailing theory as Science
Please note, it is not the other way around….setting up experiments and using their results to propose theories. That is an Inductive Error of the highest order!
Prevailing Malaise in Empirical Science
So much of published medical journalism these days, is this upended & erroneous inductive generalisation. Here are some errors that I can spot:
1) Testing to prove rather than disprove
Here, I’m saying it again (this is my biggest grudge!), testing excess salt for high blood pressure is upending the scientific process on its head. Because, for all you know, there be a third hidden factor affecting the result. eg: excess salt may not cause high BP directly, it may trigger high BP in the presence of some externalities, or it may simply be an effect along with high BP of some other causes (therefore just a correlation, not causation).
“A theory can very well be found to be incorrect if there is a logical error in its deductions or found to be off the mark if a fact is not in consonance with one of its conclusions. But the truth of a theory can never be proven. For one never knows if future experience will contradict its conclusions; and furthermore there are always other conceptual systems imaginable which might coordinate the very same facts.”
– Albert Einstein, in his essay, “Induction and Deduction in Physics”, 1919
2) Sampling Error
“Don’t accept your dog’s admiration as conclusive evidence that you are wonderful.”
– Ann Landers, American Columnist
Testing a hypothesis on a sample and generalising a result for the entire population is valid only when the test is carried out to negate the hypothesis, not for validating it. No matter how big your sample is, it is erroneous to generalise a test result for validating a theory. The only relevant sample size for such a test is the entire population!
“What are the three largest, most relevant sample sizes for identifying universal principles? Bucket number one is inorganic systems, which are 13.7 billion years in size. It’s all the laws of math and physics, the entire physical universe. Bucket number two is organic systems, 3.5 billion years of biology on Earth. And bucket number three is human history.”
– Peter Kaufman on sample sizes
3) Taking correlation for causation
When I was 15 years old, I used to go for morning jog. A girl from my neighbourhood, would also soon arrive in the public garden for a walk. There were days, when I would not be able to go…on many such days, even she wouldn’t step out for her walk. My juvenile teenage heart would draw the causal links. Today, the experienced me, would read the situation as a correleation, rather than causation. The coincidence of non-jogging can also be explained for reasons like rain or school exams!
“One of the first things taught in introductory statistics textbooks is that correlation is not causation. It is also one of the first things forgotten.”
– Thomas Sowell
4) Having a reductionist approach for a Complex Entity
Our body is a complex mechanism. Uncountable parts interact with uncountable factors and create emerging behaviours. Only linear, simple systems can be understood and explained by breaking them down into parts. Complex systems thrive on multiple inter-linkages and their 2nd order effects. It is a futile exercise to try and understand them by reducing them to smaller parts, isolating two of them and studying their correlation on a sample.
A lot of medical publications are nothing but correlation analyses – statistical tools applied on a sample data set, focusing on a two factors at a time (ignoring the underlying complexity), stumbling on a observation set and concluding theory from there theories like,
- X is beneficial for Y
- X causes Y
- X prevents Y
I addressed this error of applying reductionism in Complex matters in this Twitter thread:
Sir Austin Bradford Hill came up with nine criteria for evaluating the strength of empirical findings and determining the likely direction of causality.
I randomly scanned some of the research reports quoted on ageing and found them miserably failing The Bradford Hill Test. One of the latest, I stumbled upon was titled, “The association between body height and longevity”. All the death records in Poland between 2004-2008 was set up for an ANOVA (Analysis of variance) and concluded that longevity is inversely correlated with height. Meaning taller people live shorter lives! Also, “the longest lifespan was found for individuals born in December, and the shortest for those born in May.” “…these results strongly suggest that shorter people can outlive their taller counterparts.”
In my humble opinion, this analysis is nothing but a statistical play between gender, height and age – the three most prominent data available in the death records probably; nothing more. It does not take human knowledge any further.
- If similar study is conducted for a sample over larger time frame, or over more countries, can the results vary?
- In other words, with what level of confidence can the conclusions be extended over time & geography.
- How different is this result over 848,860 sample size different from a result of 848,860 coin tosses?
To begin with, there has to be a theory explaining how & why height of an individual impacts ageing or longevity. That theory needs to be logically refuted. A careful experiment or test has to be set up, knowing a-priori, what is put to test and what result will expose faults in the theory.
I saw my sentiment and frustration reverberating in this statement from Dr. Malcom Kendrick in his book, The Great Cholestrol Con.
“These facts (medical research) are really only partially true. They are rather like the false-fronted buildings used in Westerns. If you look at them from the dead ahead, you see what looks like an entire town laid out in front of you. But if you move sideways, just a little bit, you can see that the supposedly solid buildings are just four-inch-thick plywood with nothing behind them at all.”
– Dr. Malcolm Kendrick, The Great Cholesterol Con
Here’s The Thing
In matters of health & fitness, it is important to look at science. A large part of this science is having an evolutionary perspective. On top of it, the modern medical knowledge about our anatomy and diseases is very useful. However, watch out for the shallow statistical noise, as empiricism side-stepped ahead of scientific explanations and deluge you with correlations posing as spurious causations.