Psychotherapy aims to help patients make progress and experience improvement throughout their treatment. Premature termination of therapy is undesirable as it hinders patients from achieving their objectives. It is crucial’t leave patients worse off than they were before starting treatment. Also, patients should at least receive an adequate dose of psychotherapy before leaving treatment. Ideally, they should do so only after achieving some positive change.
It is common for patients to assume that their psychotherapists will detect any negative changes in their condition during therapy. Derek Hatfield and his team analyzed case notes from psychotherapies in which patients got worse. They wanted to test if therapists noticed this and write it down. However, this was rarely the case in the notes they looked at. It was much more likely that therapists did not even mention the deterioration and continued the treatment as if nothing happened. We took a slightly different angle in a study that I did during my time at the University of Greifswald's clinical psychology lab. We looked at a database of more than 20,000 patients that included patients’ self-reported therapy outcomes and the therapists’ perceptions of these outcomes. We noticed that, if the patients reported a positive outcome, therapists almost always agreed with them. However, if patients reported no meaningful change, or if they deteriorated, therapists have a completely different view. Only 7.8% of patients reporting no meaningful change were rated as such by therapists. Of more than 1000 patients in this sample who deteriorated, only 2% were correctly recognized.
Apparently, we cannot assume that deteriorations during therapy are always recognized. This is where feedback systems come in. In a way, feedback systems work by correcting therapists’ perceptions. While they are usually highly trained and skilled professionals, they are still humans with biases. Like many other professions, psychotherapists think highly of their competence: a survey found that 25% of therapists think that they are among the 10% most competent therapists, they tend to overestimate their improvement rates and underestimate the proportion of their parents who deteriorate.
How do feedback systems work?
Apparently, some events in psychotherapy get overlooked easily. This does not mean that psychotherapists are doing a bad job - my take would be that it is only human to miss some patterns and even the most thoroughly trained therapist will have subjective biases that distort his view. Not detecting patients who do not make progress or even deteriorate, however, can prevent bad outcomes. This is where feedback tools come in. They come in different flavors in terms of technical sophistication, but they all work with weekly surveys of patients on their current stress levels. Some are as short as four items (like the Outcome Rating Scale) and some are more comprehensive, like the Outcome Questionnaire. With these frequent surveys alone, therapists can determine how stressed patients feel. Spontaneous deteriorations or improvements become visible and can be addressed promptly. Most systems (such as the Greifswald Psychotherapy Navigation System, GPNS) offer clear, quickly interpretable graphs that show therapy progress like in this figure:
Typical feedback graph showing the course of a brief symptom scale measured before therapy sessions. 1: observed symptom score, 2: cut-off value for clinically meaningful improvement, 3: cut-off value for deterioration, 4: cut-off value for remission, 5: individual failure boundary, 6: expected course of symptoms.
Typical feedback graph showing the course of a brief symptom scale measured before therapy sessions. 1: observed symptom score, 2: cut-off value for clinically meaningful improvement, 3: cut-off value for deterioration, 4: cut-off value for remission, 5: individual failure boundary, 6: expected course of symptoms.
There are very simple variants of feedback that work by just collecting raw symptom scores and comparing them to certain cut-offs. Some of these systems, like the one shown in the figure, also offer individualized predictions for the expected symptom course. These so-called “expected treatment response” (ETR) curves are useful because not every patient can be expected to show the same course. Meaningful deviations from this predicted course can be used to warn therapists. Finally, some systems also offer insights into potential reasons for deterioration, dubbed “clinical support tools”. This includes the assessment of the therapeutic alliance, therapy motivation, life events, or other potential causes of sudden deterioration or a lack of progress. Some systems even offer practical advice and suggest specific therapeutic techniques. For example, a patient who does not make the expected progress and reports a low therapeutic alliance could be addressed with alliance rupture techniques.
Is there empirical support for using feedback?
Studies have been conducted on feedback systems for over 20 years now, to determine whether they actually improve therapy outcomes and prevent undesired effects. A recent meta-analysis led by Kim de Jong, which included 58 studies, concluded that the use of feedback systems leads to better outcomes in psychotherapy on average, and also reduces the likelihood of premature termination of therapy. The researchers also looked at whether it makes a difference if feedback systems include additional features like individual prediction curves or clinical support tools. While the average effect was small, it was substantially higher in a subsample of patients who did not realize adequate change in their therapies. Feedback was much more effective if it included clinical support tools. Larger effects for patients who do not show improvement are expected. After all, they have nothing to detect in patients who improve anyway.
Do we really need them?
Although there is much to be said for the use of feedback systems, there are some vocal critics of the approach. What the critics have in common is that, instead of using technological tools, they put the person of the psychotherapist in the foreground: he or she is, above all, very well trained and, with sufficient clinical experience, has all that is needed to be maximally effective. This is why I call it the "therapist-centered perspective." I will briefly review the main propositions this perspective makes.
„Well-trained therapists do not need feedback“
According to the therapist-centered perspective, therapists will not profit from feedback systems if they receive adequate training. They reflect on their own therapeutic activity so well that they are aware of their own shortcomings and actively work on their weak points. Their will to improve and overcome their resistance in this respect is so great that the information from patient feedback would be nothing new for them.
There are, at least, two empirically founded arguments against this.
First, empirical evidence suggests that feedback systems improve outcomes for both highly effective and below-average therapists. Let’s assume that not all psychotherapists are equally competent. This is a pretty robust finding in psychotherapy research. If differences in effectiveness are at least in part the result of differences in competence, there should be no additional effects on outcomes of individual therapists. A team of researchers analyzed data from six clinical trials to investigate whether the use of a feedback system by therapists influences therapist effects. What they found was rather surprising: in the group of therapists who used such a system, the differences in outcomes between therapists became smaller while, on average, the therapies became more effective. Among the therapists using feedback, 40% showed below-average treatment outcomes while four therapists scored significantly above average. In the group not using feedback, 70% of therapists had below-average treatment outcomes and one therapist in the group did not - even though this group had, on average, worse outcomes. This indicates that using feedback leads to more highly effective therapists and less below-average therapists.
Second, even experienced therapists seem to profit from feedback. One might also argue that therapist competence grows with clinical experience. In fact, some researchers argue that “it takes 10,000 hours to achieve mastery” in psychotherapy. I could not find any empirical evidence for this statement and some evidence against it. But let’s still assume that therapists become better the more experienced they get. A randomized controlled trial from Norway gave one group of therapists access to the “Partners of Change Outcome Monitoring System” (PCOMS) while the other group continued to offer therapy as they normally do. The effect was similar to those found in the de Jong meta-analysis. Interestingly, however, the therapists in this study were mainly highly experienced: 15 out of 20 therapists have worked for over ten years. They were also reported to be most influenced by psychodynamic therapy models. Apparently, using a feedback system still has some benefits for experienced therapists.
By the way: later, the authors of the Norwegian trial looked into potential working mechanisms of feedback. They found that using the PCOMS led to an improved therapeutic alliance which, as I wrote previously, leads to slightly better outcomes.
Conclusion
Psychotherapy aims to provide positive change and improvement for patients, but often therapists may overlook or not recognize deterioration in a patient's condition. Research indicates that therapists frequently overestimate their competence and may not detect when patients do not progress or even regress. Feedback systems, which employ periodic patient surveys, offer an empirical method to gauge patient stress levels and therapy progress, thus aiding therapists in adjusting their approaches. While critics argue that experienced therapists might not need such systems, empirical evidence suggests that these feedback tools can enhance even experienced therapists' effectiveness and promote better patient outcomes.