I think one of the biggest things is that Psychology is seen as such a 'general' degree. There are of course people who go into it with realistic career plans and knowledge, but I think it's often seen as a subject that people choose if they don't know what they want to do. It's a generalisation, but it can be that generalisation for a reason.
I think it's also got a bit of a reputation for people going into it thinking 'Clinical Psychology/Forensic Psychology' - all the 'popular' options. Obviously there are people who again go into it, do what they need to do to make that a reality, but I think it's one where it all sounds quite 'exciting' and then people sort of realise the reality and are put off.
Having said that, I am biased but I absolutely love it as a subject. I went into Psychology thinking I wanted to do my MSc in Forensic Psychology - I have wanted to be a police officer since I was very very little, so had the attitude that Forensic Psychology would be the next best thing since policing isn't a possibility for me. As it turned out, I then turned down my offer for MSc Forensic Psych to do my MSc in Sport & Exercise Psychology instead, which is again a course which is often seen as quite 'mickey mouse', not a 'real' subject. And honestly, I absolutely loved it. There is nothing more amazing than going into uni absolutely loving your course, studying the thing that makes you happiest, and I just knew I wanted to do research forever.
And although it's not specifically 'psychology', I was employed for my current (and first) job (Social Researcher in a Civil Service department) specifically because of my background in psychology, because they wanted somebody who could lead on the behavioural analysis side of things in my department, so it was a definite positive for me!