A voltmeter can do two things, it can measure the energy (per charge) transferred to a component as current flows, and measure the energy (per charge) transferred to the passing current by a cell, or would be transferred if a current was flowing.
The difficulty comes from the role of the internal resistance. When no current flows, there is no work done passing through it. When current flows then work is done and energy is lost and some of the emf from the cell is lost to this internal resistance.