I understand it is worrying to face having restrictions put on your licence and I'm sure many people who should notify DVLA of a medical condition don't because they fear losing their licences.
That said, I think you should notify them as soon as possible- you have have a legal obligation to inform the DVLA and you should do so as soon as you are aware of the condition. Failure to let DVLA know could lead to a fine and potentially prosecution. You could lose your licence entirely/have to resit your test after a period of being banned.
Not only that but it will invalidate your insurance- in the event that you were unlucky enough to have an accident they may not pay out (if they found out about your condition) and you will be personally liable for all costs. In addition, most insurance companies require you to let them know if you have a notifiable condition (some insurers don't, but they are rarer) and in the terms of most policies it says you must inform them if you are later diagnosed with a notifiable condition. Failure to do either of these things will not only invalidate your insurance but your insurance company would be within their rights to cancel your insurance and this is likely to make it very expensive, or even impossible, to get insurance in the future (and not just car insurance, either).
I can understand the worry about losing your licence or having temporary restrictions put on you, and I can also understand how tempting it can be to not inform the DVLA as it can seem unlikely that they could find out (although your Dr could inform the DVLA themselves if they know that you have not informed the DVLA despite having been advised to- though they would have to tell you they are doing so and it's not something which is done lightly or frequently).
I would encourage you to do things properly because I have seen first hand the potentially serious impact of not doing so and being found out. I have a family member who had a notifiable condition who didn't bother to inform DVLA (or insurance company) because not being able to drive would have made life really difficult and they really didn't think they'd be found out. They thought even if they did it would be a minor thing and a small fine or something. Unfortunately, they had an accident.
Through this, the insurance company found out about their condition. The insurance company declared the insurance invalid and cancelled the policy. They then refused to pay for the cost of repair for either vehicle or the legal costs etc. This has been financially disastrous for them. The insurance company also passed the information on to the DVLA, who investigated and this led to a big fine and legal proceedings as they were deemed to be driving without insurance and also for non-disclosure. This led to licence being lost for a long period. Having an insurance policy cancelled has meant getting other kinds of insurance has been more difficult and expensive, with some companies refusing even to provide a quote. Just be aware of the potential consequences should you be found out.
Lastly, I would say that notifiable conditions are notifiable for a reason and this because it is possible that your medical condition could make it unsafe for you to drive. Imagine how you'd feel if you did cause serious harm to others or yourself when you were driving whilst unsafe to do so. Look at the Glasgow bin lorry driver and the grief he has caused- I do recognise that your condition is different and I'm guessing you don't have an HGV licence so the situation and degree of risk are different, but the principle still applies IMO.
Sorry for the long post. As I said, I've seen the devastating effect of not declaring and I wouldn't wish it on anyone.