Feeling guilty about quitting job

So I'm planning on handing in my notice at my part time job on Monday. This is for a multitude of reasons. It's a four mile drive away across the city, which can take me close to an hour in bad traffic and also means that I can't work weeknights, which would work far better for me than weekends because of having uni in the day. Plus I'm delivering pizzas, on my own to pretty rough areas of my city and I don't really feel safe doing that. I've got a shift on Sunday, my first after a break for Christmas and the thought of going back is really making me feel miserable.

Thing is, I still feel really bad about quitting because my boss is an incredibly nice guy, and I feel like I'm just going to drop this on him out of the blue. I've been thinking about it for a while, but I've never mentioned it to him. He's been really cool about letting me take time off to go do various weekends away with the clubs I'm part of. So basically I feel like I'm pulling a major dick move by up and quitting like this but I feel like it's the best thing for me personally.

Honestly I'm not really sure what response I'm looking for here, before anyone says it. Think I just needed to unload, and if anyone wants to weigh in on my situation, offer reassurance or tell me I'm being a prat, that would be grand.