Something to bear in mind is, your subject is fairly uncommon so applying "generic" notions of prestige probably isn't that relevant.
Outside of some sense of "the grass is greener" in terms of prestige, is there any actual concern or doubt about your current uni? Are you unhappy with the facilities it offers, the student body, or the lecturers (and their abilities)? Is there a serious problem with the teaching on the course? If the answer to those is no, I'm not sure trying to "trade up" is going to be that worthwhile. You may well find those things you take for granted now aren't present at the new university, so while it may be more "prestigious" you end up hating the course you're on.
Also, those feelings may not go away anyway; you could then just enter the same cycle thinking about some other university which is considered "more prestigious" that offers your course and eventually you'll get to the point where the "next best" uni you don't (or can't) get into for whatever reason, and then you're stuck somewhere totally different, still feeling the same way but with one or more years less funding available, having to settle into a new environment and get to know new people.
It's a significant decision to make and changing courses at this point is not really something to take lightly. Think very hard about why you are thinking about this and whether it's definitely something you should pursue (and if you come to the conclusion there are serious issues at your current location which will prevent you from achieving your best then by all means do pursue that with vigour!). You can always do a masters at the "other" university in question (or indeed, another university not currently in the question but which might become an option if you do excellently in your current course).