Does Therapy Really Work?

A lot has been written about what therapies work for what disorders in what patients.  A more central question is whether talk therapy in general has any evidence to support its efficacy in relieving human suffering.  It's a fair question but first let me say a little about the nature of the debate surrounding therapy effectiveness.

Many of the articles out there fall into a category that I refer to as "horse race studies."  These studies basically try to establish the superiority of therapy x over therapy y.  While interesting, they have more academic than practical relevance in most cases.  A broad literature establishes that non-specific factors, things like the alliance between the therapist and patient that exist across all therapies, are actually more related to outcome than specific techniques associated with say Cognitive Behavioral Therapy or Psychodynamic therapy. (For a review of research on non specific factors look here).

About a year ago I had the pleasure of attending a talk given by Jonathan Shedler in which he presented a summary of the research on psychotherapy effectiveness with an emphasis on longer term psychodynamic therapies.  Here is a brief summary of that excellent talk.

First, the current methodology for evaluating therapy effectiveness is the statistical technique called Meta Analysis.  Using this well-established technique, researchers can pool data from a number of studies and compare studies using a metric called Effect Size.  An Effect Size of 1 means that there was a change of 1 standard deviation on whatever is being measured.  Put another way it would mean that an individual was one standard deviation better on a measure of depression.  In the general population a 1SD change in height would mean that the average person would be six inches taller, or in terms of weight 70lbs lighter.  In other words this would be a big change.

In meta analysis an effect size of .20 is considered small, .50 medium and .80 and up large.

If you watch any TV at all you might think that medication treatments for depression were massively effective but are they?  According to meta analytic comparisons from FDA data Prozac and Zoloft have an effect size of .26, Lexapro .31 and Celexa .24.  These are all SMALL effect sizes.  What is even more disturbing is that there is evidence of bias in the reporting of this data in that about a third of studies never get published because they are not favorable.

OK, so medication is not the cure all we have been promised by advertising.  How does therapy stack up?

Smith, Glass and Miller (1980) looked at 475 studies and concluded that Psychotherapy produced an effect size of .85.  That would be a LARGE effect size and this finding has been replicated by others. In fact, one study including almost 1500 subjects found that psychodynamic therapy had an effect size of .97 immediately after treatment.  More importantly the positive effects INCREASED over time to an effect size of 1.51 nine months after treatment ended.  This may be because the therapy gave clients skills that they could apply to many aspects of their lives.

So psychotherapy is effective as established by rigorous science.

Evaluating Scientific Sounding Claims

Two stories on the radio this week got me thinking about the ways that we use science and how scientific discoveries are communicated to us. The first had to do with research on the toxoplasma parasite carried by cats (toxoplasma gondii). This parasite can cause toxoplasmosis in humans, an illness that can cause death in children and adults and birth defects in the fetus.  (If you have a cat litter box, wash your hands after cleaning it!) The argument went something like, toxoplasm is carried by cats. Cats began to be domesticated by humans and were very common at about the same time that rates of schizophrenia increased rapidly around the turn of the century, therefore there might be a relationship between keeping cats as pets and schizophrenia. The full episode of Radiolab, which aired the segment, can be found here.

Another story examined the role of microbes in human life.  Everything from the beneficial bacteria in our stomach to more esoteric microbes that can affect our behavior.  The point seemed to be that microbes could be the master controller of human behavior. The full NPR story is here.

Both stories are admittedly intriguing and the scientists interviewed are not crackpots.  So how do we really evaluate the science we hear and apply it sensibly in our lives?

One thing to keep foremost in our minds is that a relationship between two events does not tell us anything about the causality that may or may not link them.  Because schizophrenia increased when we started keeping cats as pets does not establish a causal relationship between cats and schizophrenia.  In fairness, the relationship between certain mental illnesses and toxoplasma has been reported elsewhere, I'm exaggerating somewhat to make a point.

The relationships between these issues is complex.  There is more interdependence, or what statisticians call covariance, than direct causal relationships among the variables.  It is more interesting to talk about the possibility of grand relationships and science fiction than to tackle the nuances of the relationships.  Who wants to read a dry science article when we can talk about protozoa taking over our minds.

So the point is to take science reporting at face value.  Dig deeper if a topic interests you and treat grandiose claims as what they are, bad science.