07. Why Validated Therapies Can Lack Validity

Once upon a time, psychotherapists allegedly practiced in wild and random ways. In an effort to improve the standard of care to patients, authoritative lists of empirically validated (i.e., scientifically proven) treatments were compiled to guide and regulate treatment selection.

The use of research evidence to guide psychotherapy sounds perfectly logical on paper, but is poorly executed in practice. Critical flaws lied in the type of research conducted and the sort of evidence accepted. Consequently, even overtly dubious treatments managed to receive the scientific seal of approval and gained formal recognition.

Reinventing The Wheel

A clear pattern emerged amongst the new treatments. First, senior practitioners noted many were simply reinventing the wheel and rebranding more or less the same old ideas in slightly new ways, at times still managing to add something useful (even if it is just applying the same therapy principles in different ways, or using a different set of jargons, so it works for or speaks to a different population), but often “nothing new is useful, and nothing useful is new”. Specifically, these “innovations” tend to just add some technical looking procedures or rituals to existing therapies (eg, sequential tapping while processing painful emotions). Perhaps unsurprisingly, Randomized Controlled Trials (RCTs) often showed the new therapies to be better than placebo and as effective as established treatment, but importantly, they work just as well with or without their defining and innovative components.

In scholarly texts, these “innovative” therapies have been satirically compared to the hypothetical Purple Hat Therapy. Imagine if someone claims to have a purple hat that emits curative magnetic waves and test it by asking patients to wear the hat during (obscured) classical therapy. Patients improve and the hat gets credited – even though Purple Hat Therapy suspiciously works just as well with or without the trademarked hat.

Superficial Methodologies

Under the prevailing scientific paradigm, a brand of therapy can be wholly validated as “evidence-based” if something within its package of procedurally-defined interventions can somehow lead to symptom reductions in some patients. As long as the improvements are methodically “evidenced” in RCTs then that is sufficient. Thanks to the lax rules, many have created novel evidence-based treatments via packaging “innovative” components with tried-and-true ones, or use new procedures plus new rationalizations (that are often shrouded in neurological babble and other fashionable lingo) to deliver what is otherwise familiar to seasoned clinicians.

Non-scientific Influences on Science

Why did Psychiatry and Clinical Psychology embrace such a shoddy research paradigm, which led eminent researchers to question whether our “gold standard” methodologies are gold plated or fool’s gold? Likewise, why did we subscribe to a shallow diagnostic system that (according to its creators) is not designed to meaningfully distinguish between problems, and thus cannot guide treatments (see Newsletter #4)?

While nothing resembling the full answer will fit into this short read, I hope it helps to highlight that decision-making in scientific disciplines (just like those in politics) are not always based on what is most logical or societally beneficial. Sometimes, they are the product of unwanted compromises, selfish interests, power plays, trends and dogmas etc.

I wish evidence-based practice is as straightforward as what many make it out to be, but it is not. The issues outlined here are just the very tip of the iceberg, so it is important to have practitioners who can make informed and judicious use of the available evidence.