I'm really glad that you're going. Not to have gone could have led to a lifetime of 'what ifs' and to be in a job that you hate at your age isn't a good prospect for the future.*
Now you've made the decision do try to embrace it. I can appreciate concerns about money, but remember that in the early years you get long holidays and should be able to earn a considerable amount. Think of it as a huge opportunity and adventure.
One 'older and wiser' thing I would add is that very few of us have a job we love all the time. I think you're lucky to have found something that you have some passion for - and that will also have highs and lows. *For most of us the jobs we have are down to a mixture of luck, opportunity and interest. If you like it most of the time you're fortunate. And it does sound as if medicine will give you a much better chance of that than pharmacy.*
Have a ball - and maybe store these posts to look back on in 10 years, when you're working as a doctor and have a family. *
(Edit. Don't know why all these asterisks appear - something to do with my iPad)*
What should I do?