It might be a bit busy at that time as people have come back from work and they may want to go for a swim (but then again, they may be tired so it might be empty). Best time in my opinion is morning, or before 6Pm after coming home from school. There aren't many people there at that time.
It depends on how you swim e.g. casual, fast etc. You also have to learn different strokes like backstroke, breaststroke etc. You may also want to try getting a small float and practice just kicking lengths with it (I found this really difficult).
I used to go swimming for about 5 hours a week, but it was very rigorous as all I used to do was lengths. This, along with other sports kept me in shape.