A problem I have which I think is specific to me is that when I take one hand off the wheel to change gears, I start drifting even if I focus on the road. It's so annoying and I can't help it a lot.
Even worse, sometimes when I'm changing gears am not sure if I'm in 3rd, 4th, neutral or what and I panic and then stall - it happens rarely though, but if I make this mistake it knocks my confidence and I continue making other stupid mistakes.
I'm pretty much that person everyone looks at to feel better about themselves.