Put yourself in the position of someone living in the United States - a society saturated with firearms - where your perception is that criminals are generally armed.
Your perception will be that if your legal firearms are removed, you will become figurative cattle to any criminal who will be completely unaffected by a ban on firearms.
Also, put yourself in the mentality of someone who is wedded to the idea of the Constitution. The 2nd Amendment is there to protect people from tyranny. This may seem bizarre to us - but they don't see it like that. It seems circular to us, but the sort of oppression they fear would be something like the Federal government moving to take people's weapons away - and they have these weapons for that very reason. The larger rationale (which I think has some merit) would be that if they would remove one Constitutional Freedom, there is nothing preventing them removing others.
Assume for a second that all legal firearms were removed. There would probably still be school massacres - although sure, there would probably be fewer - but every single day the news would be saturated with reports of murder, home invasion, rape and so on and the implication would be that people were left defenceless. A society can't tolerate that kind of constant barrage of bad news. It would be like getting rid of speed limits or making all drugs legal here- regardless of the long term, the short and medium term would be intolerable and political suicide.