For things like jeans and trousers and stuff anyway, unless they're the chavvy 'Bench'/'Hooch' 10ft logo down the sides, how many people will look at them and go, 'Ooh those are from Primark!' or 'Ooh those are from Armani!' because they can instantly tell? Not many, I don't think...
I think to a large extent, it's how you wear clothes that people judge you, for example if you look scruffy or you're wearing something really slutty or really mismatched. Otherwise, in the street, people won't give you a second glance.