Also, your arguments about where a gene has to be damaged don't necessarily hold up. Surrounding and interspersed with most genes are regulatory elements, and if those get messed up in the right way, they can cause a gene to no longer be expressed, or to be expressed enormously too much.
Finally there are some genes that are super mutagenic if they go wrong. This one takes a little explaining. Basically, the immune system (and specifically B cells) produces antibodies with a highly variable region at one end. Each B cell produces a different shape of antibody, and the shape of that part of the antibody is determined randomly. The way this is done is, there's a protein whose job it is to make random alterations to that part of the gene for the antibody. It's tightly regulated to only make random changes to that gene, and it's only turned on in B cells for a short time in their development from their progenitor cells. But again, if that protein's regulatory elements take a hit in the right way, it can be turned on in some random cell in your colon (for example) where it will just make random changes anywhere it likes in the genome, continuously.