Its because Oxygen possesses a half-full shell which makes it more stable and hence stronger attractive forces between the nucleus and electrons. Therefore stronger forces pulls the the atom together more tightly.
The electronic configuration of oxygen is 1s2,2s2,2p4 (4 away from being full-a half full shell)
The electronic configuration of Nitrogen is 1s2, 2s2, 2p3 (5 away from a full shell so less stable)
This is all based on spin-pair repulsion theory
This is what i put in the exam anyway, so I'm probably wrong