Could the Psychological Studies We Base Mental Health Decisions On Be Faulty?

Pin

Apparently, yes they could.

The past several years have been bruising ones for the credibility of the social sciences. A star social psychologist was caught fabricating data, leading to more than 50 retracted papers. A top journal published a study supporting the existence of ESP that was widely criticized. The journal Science pulled a political science paper on the effect of gay canvassers on voters’ behavior because of concerns about faked data.

Now, a painstaking yearslong effort to reproduce 100 studies published in three leading psychology journals has found that more than half of the findings did not hold up when retested. The analysis was done by research psychologists, many of whom volunteered their time to double-check what they considered important work. Their conclusions, reported Thursday in the journal Science, have confirmed the worst fears of scientists who have long worried that the field needed a strong correction.

The vetted studies were considered part of the core knowledge by which scientists understand the dynamics of personality, relationships, learning and memory. Therapists and educators rely on such findings to help guide decisions, and the fact that so many of the studies were called into question could sow doubt in the scientific underpinnings of their work.

This isn’t great news for anyone who is dealing with a mental health issue, or anyone studying the effects of childhood abuse. We tend to look at study results we find online and assume them to be true, but maybe we shouldn’t do that. On the other hand, this would help explain why we see studies that seem to contradict one another because in truth, one study with one group in one circumstance might actually only show those results that one time and another study with a different group in different circumstances would produce different results. That seems to be the gist of this work, in trying to reproduce the results of studies, half the time they weren’t reproduced, which means that maybe, just maybe, we should not jump on the next internet bandwagon about treatment, but wait until we have more evidence to corroborate the information.

But why let further science get in the way of a good internet meme? 😉

On the other hand, once we do have further studies that back up information about childhood abuse, depression, and so on, we should absolutely push that information out as widely as possible! But, as always, and even with stuff that I share here, let’s not jump to conclusions without further evidence.