There used to be a nice vintage stall in the market, no idea if it's still there, though. With the charity shops, like anywhere, it's a bit what your luck's like.
Not clothes wise, but for accessories, there are some nice antiques places down the barbican, where you can get some nice jewellery.
Totnes is worth a try too, but I disagree about Exeter. They have more "designer" shops, but I don't know about any vintage shops there.