I don't understand it. If a student is seriously considering withdrawing, it's not often because it's 'too hard' or 'boring' - I've been in this game over ten years and can count on one hand the amount of student withdrawals that've been along those lines, it's a myth. Similarly, if they realise it's 'not what they want to do' then that's 100% fine. It's a rational, adult decision. And again, it's pretty rare that students drop out to do nothing at all. Anecdotally I have like two mates that did this. But we follow up with our drop out students and the vast, vast majority go do something better/different.
So there is a stigma, I think, but it is very seldom justified and is largely based on misunderstanding.