While I'm not 100% sure on the specifics as I'm not looking into exotics, I don't think being just a panda vet is massively realistic. As said above, you could become a zoo vet, and maybe specialising in bears in general would be more realistic. But thinking logically there is not that many pandas in the world, much less if you are only based in one area, and these pandas are unlikely to need vet attention every day, and even when they do it's unlikely to take up a whole day, so I can't imagine you'd have much work as a panda vet if it were possible. If you are truly only wanting to be a panda vet with little interest in other animals I would not advise becoming a vet, at uni I can't imagine there is a lot of specific panda content (though if going into exotics you can do work in zoos) and a lot of the learning is focused around domesticated animals, like dogs, cats, sheep, cows, etc for 5+ years.
If you really want a career with pandas I'd look more into a conservation or zoo keeper direction, though I can't advise much about those.