Gravitational potential might best be looked at as a number related to a point in space, regardless of whether there is anything there. It is often used in questions relating to an object with mass but this does lead to misunderstanding.
GP is the energy per unit mass required to move an object from where it is (presumably in a gravitational field with some value) to infinity. The units being Joules per Kg.