As long as you are taking the pill every day and restarting after a bleed on the correct day, you are unlikely to be pregnant. 3-4 hours difference isn't going to reduce protection, if you missed a day completely, that's when we consider it a missed pill.
Fatigue and stomach cramps could be caused by a huge amount of other issues - digestive issues can be confused with lady parts because everything is very close together down there. Stress can cause stomach discomfort and fatigue, as can diarrhoea (which you will need to bear in mind regarding your pill effectiveness if this is prolonged and severe).
If you are struggling with taking it late, perhaps pick a different time to avoid this disruption of routine.