It'll be the initial acceleration of the rocket the question is asking about, so you don't have to worry about changing mass.
Use F = ma, where F is the initial resultant force on the rocket (remember to consider the initial weight of the rocket!), and m is the initial mass of the rocket