The rowing machine in the gym should help a lot.
Look on YouTube vids for correct techniques for actual rowing and normal gym rowing.
Major muscles for rowing are core muscles, thighs, lats, and biceps as these are essential to the rowing motion. However, It is important to train all the major muscle groups because the body works as a unit and not as individual muscles in isolation. Do compound lifts.