You seem to be talking about two different things: the rudeness of (some) interviewers and the fact that you "end up being rejected".
It may be that these two things are related - i.e. because the interviewer has decided within the first 5 minutes that you're not the ideal candidate, and they're just "going through the motions" to avoid cutting the interview too short - but the two things might also be entirely unrelated to each other.
I think them being "distracted by something in their room" or having "got up from their seat" is probably just a reflection of the way some people behave. I often see people on the phone (not video) with a newspaper in front of them, distractedly flipping from page to page whilst also trying to hold a conversation. I find that very odd behaviour, personally. I think what you're seeing is just another version of that. For some of them, it could also be a "power" thing - just trying to demonstrate that they're more important than you.
(By the way, I don't think you should worry about interviewers being late. Yes, it's rude. But it happens all the time.)