Your body has what is known as a Circadian rhythm. It a 24-hour cycle of hormones and is regulated mostly by light-levels.
As such, you should really go to sleep when it's dark, otherwise you'll play havoc with your body's chemicals, and may end up being moody or tired at strange times of the day.
Having said that, we can all close our black-out curtains and make it nighttime during the day, and we could all go to work in a fluorescently-lit office at night. That wouldn't be too bad.
The "jet-lag" people feel is your body adjusting to a new Circadian rhythm because the pattern of light-levels changes when you take a trans-Atlantic flight, for example. This has been observed to happen with racehorses as well, believe it or not.
So if you keep adjusting your bedtimes by large amounts (like 10pm some nights, 8am others), you will be giving your body a tough time. You might not even eat properly, and this can lead to other problems.
Bottom line: get a decent amount of sleep at regular times!