I think in conversations people naturally make eye-contact for about 60% of the time, so looking away sometimes is fine. I sometimes get it where I start thinking way too hard about how much eye-contact I'm making, but if that happens I take a few deep breaths and try to really immerse myself in what they're saying. You could also try looking at the bridge of their nose in-between their eyes, as that'll come across like you're making normal eye-contact.