The moment or angular force or torque is simply
linear force * distance
1) the moment (angular force) on the right gear from the mass must be
10*g*0.08 (clockwise)
2) In equilibrium, this must be balanced by the force from the left gear's teeth, touching the right gear's teeth. Assume the linear force is F, then on the right gear wheel, we must have
F*0.1 = 10*g*0.08
F = 8g
3) On the left gear wheel, we also have equilbrium
F*0.05 = M*g*0.02
Assuming I've not done a typo mistake, that should give you M. Note the direction of the "rotation" of the left gear must be clockwise, so the mass should occur on the right. If this isn't clear, which bit isn't?
As a short cut, as the gears' radii are in the ratio 2:1, then the angular forces will also be in the same ratio (the gear ratio). The angular velocities would be in the ratio 1:2.
Edit see
http://furthermaths.org.uk/docs/SimpleGearsandTransmissionStudentFinal.pdf