The only good reason to stop what you're doing now is if you're genuinely unhappy doing it.
What's interesting about your current stance is that you're willing to apply several times to undergraduate courses but somehow less willing to apply several times to graduate courses. Yes, GEM is more competitive but it's not to say it's impossible and people do get in. And you'd be doing it from the position of having a degree which gives you options for other things, not just medicine.
My perspective is that quitting your current degree ties you to medicine whether you find you want it or not because you'll have given up something so huge as a degree to do it. And for someone who's expressed some uncertainty about medicine and for valid reasons, this doesn't sound like a healthy option for you.
I don't subscribe to the notion that you should automatically drop everything for medicine if you think you want it. Or that if you're torn between medicine and something else, that you should only do something else. Or that you should never do another degree if you want to do medicine (although I do agree that specifically planning to do a degree when you know it's medicine you want is not the most sensible of choices).
The way I see it is that you have, in real terms, less than two years' worth of study left on your current degree and at least four vacations' worth of time to get work experience in a number of fields, plus two and a half years' of time to do part time work in settings that can count towards medicine. If you're doing well and you still have some uncertainty, the best thing to do is to get experience in a number of different areas to explore your options. Maybe look at another healthcare sector, like the AHPs, Pharmacy, or Dentistry, and consider those areas to see if they tempt you more. Maybe have a look at the NHS's clinical sciences programme and see if you can get work experience toward that. Maybe look at lab-based roles. Maybe something completely different. Make medicine an informed choice and an actual choice rather than simply narrowing your life down to a single lane.
Your degree is a valuable asset. You will be an attractive candidate to plenty of other employers in many pathways in life, including medicine. This is a point not made often enough and you can potentially apply to medicine from a secure base of either another career or from having other options.
Yes, finances come into it but the way it's put across on here is always a bit simplified. I know someone who started his second undergraduate degree in medicine last year. By living at home and living frugally for a year, he managed to put away £11,000 worth of living expenses. He pays his fees on a pay monthly basis, topping up a minimum second student loan he took out, is working 16-25 hours a week at enhanced rates as a care assistant and dispenser (less in exam time), does a couple of hours ad hoc tutoring here and there, occasionally some more specialist stuff for his former company, and does at least one night shift a week where he can study at the same time. He also saves on rent by living at home. He took the first offer he got and was not successful in applying for GEM but probably could have been if he wanted. But he ran the numbers and felt that this was doable and he'll graduate with quite a small amount of extra student debt. Difficult but not impossible.
Now I tutored this guy through his psychology A level and I know for certain that he would struggle to get a medicine offer. He'd already taken his A levels twice and he just struggled with A level style exams. He was dejected and lacked the confidence to get through interviews. He needed a break from medicine. His mother and I had a chat with him and persuaded him to take a look at a clearing course in pharmaceutical sciences that offered industry placements. He did that and absolutely loved his placements and secured three job offers but had also decided to apply for medicine. He applied to GEM in his but didn't make the cut but also applied to his local university that offered undergraduate medicine. They practically bit his hand off. He decided to defer his offer and take a job for a year (which allowed him to put away that money) and just allow him an option to look at other areas. He realised that he could do well in another sector but ultimately his heart lay with medicine.
He doesn't regret any decision he made. He had a lot of people telling him it was the wrong decision. He's doing well now. He was something of an inspiration for me to apply for medicine and he was just so sensible about it all. Very pragmatic and very open to new life experiences and to trying new things. He's not the type to drop out and he's better for having done that degree.
Now I could tell you plenty of other stories about people who've plugged away at medicine for years and got nowhere, and a couple about people who have dropped out and either regretted it or didn't. Everyone's story is different. I related that one because I thought it's important to see how even the most expensive option can work out not only okay but also very doable with the right choices. Don't let worst-case scenario finances dictate all your decisions.