Go for it, if medicine is what you want? If you decide to apply and achieve a place in medicine, you would still have 35-40 years of working life ahead of you. I personally would rather spend 30 years of my life working in a field I am passionate about, rather than spending 40 years of my life working as a depressed drone in a field I do not enjoy.
You have plenty of time to work hard later in your life (and earn money, as it seems to be a factor for you). Studying medicine in university is also hard, but a few years is nothing in your life. Preferably make the decision now, rather than wake up wondering after 30 years what your life would have been IF...