I went home pretty much every weekend when I lived at uni, for the same reason as yourself, which was to work at my part-time job. I found it a nice change of pace; as others have said, things tend to be quiet on the weekends as most society events, nights out etc. happen during the week. The only potential problem could be if you join a sports club, some teams might train on a Saturday, or have matches on Sundays, say. I usually came back on Sunday evenings and had a movie night with friends, and invariably while catching up with them I learned they had just done mundane things like laundry, shopping etc. during the two days I was away, so I don't feel you'll miss much.
I also felt like it made me more organised and focused during the week. I knew that a big chunk of the weekend would be taken up with work and then I would want to relax and hang out with family etc., so I tried to make sure I got as much as possible done between Monday-Friday, which ultimately benefited me I think.