Really depends on the style of briefs you wear. You want to go for attractive designer brands like Andrew Christien if you're a briefs kind of guy and are trying to impress your partner, they make really top notch briefs for men and most importantly they're extremely comfortable.
There's really no shame in wearing whatever's most comfortable for you, I think it only matters if you're being intimate with your partner. Your gf might just be annoyed because she might have put in effort to look 'good' for you by wearing nice underwear/lingerie and you're just wearing a pair of crappy over-worn briefs, so that might put her off.
I'm a briefs/boxer-briefs kind of guy myself. For me, briefs are just a lot more comfortable because they're more 'snug' and I just find it more comfortable if I'm walking around a lot most of the day. I only wear boxers to sleep.
It's just a matter of what kind of briefs you wear, if you wear like 'tighty whities' or spiderman briefs then I can see how a girl might see it as childish seeing as that's what little boys wear but there are a lot of good/designer labels that specialise in top quality briefs for men. I personally only wear briefs/boxer-briefs by Andrew Christien and Calvin Klein.