Most people don't maintain eye contact throughout a conversation. I find it quite weird and unnerving when they do, because it's not natural behaviour unless you acquire it by training yourself.
I often stare blankly into space if someone is talking at me and I'm not really thinking about responding, usually if I'm thinking about something else. I appreciate that it's quite rude but lots of people do it naturally. As a rule, the more engaged someone is, the more eye contact they'll make, but this applies to individuals only and everyone has their own limit.
There's also the 'shy' possibility of course. Or maybe he doesn't look anyone in the eye. Have you established any kind of baseline based on what he does with other people?