Nope, nope, nope.*
For one thing, Medicine is not an 'easier' option; the academic side may be less challenging than Physics, but the workload is greater (speaking as the partner of someone who has both a degree in Physics and a degree in Medicine). If your parents don't think you can cope with Physics, I'm surprised they think you can cope with Medicine.
For another, it's your choice and your life, not theirs. Any degree is difficult to see through unless your heart's really in it, and Medicine (especially considering the length of the course and the workload/long hours) is no exception. If you're interested in Medicine as well as Physics, then fine; if not, avoid it like the plague. You will be miserable if you force yourself to go down that route if it isn't really what you want.