Are the notes above £20 rare?Watch
It's a bit of a circular problem. People are reluctant to accept them, because they're not familiar with them because they don't show up in general circulation. But it's precisely because they're not in general circulation that people aren't familiar with them.
Consider the £20 note. You won't get it in change because you generally aren't paying with higher value notes. But you get them all the time from cash machines. So the average person who uses cash is going to be reasonably familiar with the size and design of a £20 note. They're more likely to notice if there's something a bit odd about it. But most people won't ever see a £50 note. Cash machines don't dispense them, and you won't get them in change. You'd need to go some effort to get hold of one (e.g. visit a bank). If someone offered one in payment, you would be reluctant to accept it as you're not familiar with the size/design/security features. Plus £50 is a big loss if it turns out the note is fake.