I have a chronic lack of patience. I can't go to lectures because I get what's going on at first, then it goes slowly and I lost interest, then when I reengage I don't know what's going on. I just learn by myself at home - it's so much easier. Same often happens with friends / parents, when you know what they're trying to say and you have to listen to them take five minutes to say it.
Unless you work alone, though, most people will have to learn this kind of patience. There will always be boring talks to listen to, and inarticulate friends you don't want to hurt.
You can develop coping strategies, like learning to solve a maths puzzle whilst also keeping track of where they have got to. Or just planning / fantasising. The former isn't as hard as it sounds. It's like people who play mental chess - is sounds hard, but most people can train themselves to do it.
Most of my friends also have a hard time watching TV without doing something else at the same time.