Please do not get frustrated as your emotions will show it to all and possibly damage the relationship.
Besides suggesting you try and get a 15 minute freebi session with a subject experienced solicitor I am afraid to suggest that you have to live with their actions.
They will be trying to frustrate you to see if you and/or the relationship will last.
Remember, they are judged by society. I am not sure if your offence included violence and I am not suggesting you would be violent to your partner, but consider if that is what they are concerned with.
In all these matters, you will feel defensive but try to focus on yourself belief. be thankful you have a loving partner who can see past your previous life.
In regard to the 'time' point, you will of learned in prison that 'time expectation' for an offender is not a consideration.
Use your time at this moment wisely, work on your relationship, now that if it is to last you will face more and possibly greater challenges for sure. In other words do not worry about what you cannot control and concentrate on what you can - living an offence free life in a caring relationship.
Learn from yesterday, live for today, hope is for tomorrow else what is left if you remove a mans hope.