A warning to psychology students, present and future.

I recognise that this forum is, by and large, a space for innocent questions about universities, courses, admissions processes, accreditation, and A-Level choices. Here, I have something more serious to say.

There is now a large and credible literature on the consequences of how psychological science is, and has been, conducted.

Long story short, the discipline is suffering from decades of scientific and statistical nonsense, such as:

•

Inappropriate sampling, which limits the scope of valid inferences, and the ability to detect true effects in one’s data.

•

A wide-spread misuse of the null hypothesis significance testing paradigm, which ultimately does not lend itself to the answering of scientific questions.

•

Poor operationalization of hypotheses, which leads to unjustified generalisations.

•

A lack of overarching theoretical frameworks or cumulative theory-building, which means research programs are conceived and pursued unconstrained.

These issues are exacerbated by the perverse incentive structure of academia, under which researchers are pressured to publish attractive (read: new, positive, exciting) findings fast. This is known to tempt bad practices, namely data dredging and hypothesising after one’s results are known.

And for what it’s worth, it was my own experience as a psychology undergraduate that alerted me to these issues.

•

Knowledge gained in one class was rarely, if ever, useful in another class.

•

Lectures were a seemingly arbitrary hodgepodge of empirical findings.

•

Assignments could be bluffed because in many cases markers cannot really evaluate the validity of your claims, and instead award points based on your ability to reproduce patterns of criticism (e.g., “this study had a small sample size”, “this finding is not cross-cultural”, “this design did not account for factor f”). Perhaps it was no coincidence, then, that assignment feedback was almost always generic.

•

The methodological issues in the field were not given their due diligence, not even at a graduate-level statistics seminar I audited.

Things are getting better. But more than ever, psychology needs rigorous, courageous, and forward-thinking students. I sympathise that research methods and statistics topics are not generally enjoyed, but the price of neglecting this aspect of your education is simply too great. A little psychology is a dangerous thing. Consider the possibility that there is an intellectual cause beyond your grades.

(edited 10 months ago)

Reply 1

Lord Asriel

All really good points, and shows how much psychology has to go. That said, it is still a relatively young science, and you aren't going to get many actual psychologists disagreeing with your criticisms. In the field, most of us will be aware of the limitations of the research base and will be out their capturing data to see what works, what doesn't refining what we do, testing that and then capturing further data.

If you see undergrad psychology as a starting point for enquiry rather than a concrete and irrefutable body of knowledge that helps. Most undergrad research isn't publishable, and is limited in scope, but it is there to introduce students to the broader scientific method of setting a hypothesis and at least trying to objectively test it. The research methods taught at undergrad are basic, but are a decent platform to build on for postgraduate studies, where you really start to see the limits of your methodology. For instance, I started off doing MRI work in my clinical reserach during my doctorate (and thought as an undergrad that proved things), but I now realise that actually most of the useable material in that field will probably be coming into usable form after I die.

I also see the issues around academia (which is why I left research to focus on a clinical career) but think that will get better as data and research moves beyond universities and paywalls, and is increasingly open access and available. I also think AI will have a huge part to play in meta analysis, incorporating 'unpublished' material etc. As a young science, we do have far to go but I am optimistic on the whole.