Sounds like you're going to have confidence issues no matter where you go. My suggestion would be to deal with those immediately. No job, degree, girlfriend, (insert other status symbol here) are going to fill the void inside of you. If you look to the external world to fill that void then you're constantly going to be disappointed.
On a less philosophical level, I wouldn't worry about your level of intelligence in a masters program. There's plenty of "stupid" people that get degrees (and plenty of smart people who don't). It's statistically unlikely that you're going to be the "dumbest" person in your course. More realistically everyone, including yourself, will have certain strengths and weaknesses. That's how life works; a masters program is no different.