Is the job fulfilling financially and for you personally? Like is it stressful or enjoyable? This is mainly focused on portrait and nature photography.
It takes a long time to build up a strong portfolio and a wide client base.
Jobs can be booked at the last minute and the clients with the highest budgets can be quite demanding.
A jewellery shop owner once made me work all night and then spend the next two days editing the images- recolouring products was not part of the job.
I mainly do nature, pets, property interior and product placement photography.
I rarely photograph people, only when a friend asks.
As with any freelance work, it will be stressful but equally, if you're passionate about it you should be able to enjoy the process. Be as proactive as possible and create as many of your own opportunities as you can to start establishing a trusted name for your self. You'll really need to grind to succeed but as you'll be your own boss the motivation should more then be there.
This article seems pretty decent in terms of providing clear steps that might help you on your journey-