I was the same & can totally relate to you, I had PCOS diagnosed at 15 and on the pill until 28 when I stopped to TTC. Like you you, i only had about 3 periods that first year and it was obvious I never ovulated. On hyper alert for pregnancy, I got very confused by all sorts of ailments. The cramping could be cysts forming on your ovaries once more?
Eventually I went to the GP, who referred me to a fertility specialist. After checking my tubes, the next step was Clomid, then if that didn't work after 3 months (ie 3 cycles), IVF (only 1 free round, though some health authorities offer up to 3 depending on age for child-less couples). While the referral went through I saw an endocrinologist, I was encouraged to lose weight, even losing 10-15% weight can kick start sluggish ovaries (which were thankfully fairly cyst-free due to the pill over a long period). I was also put on Metformin (the slow release one as it was kinder to my IBS gut) to help regulate my sugar levels, which also helped. I joined a gym, lost the weight, and just as I was about to start Clomid I got pregnant naturally just as I turned 30! Incidentally my first pregnancy symptom was really really sore boobs and tender nipples (that symptom was noticeable but less marked for baby 2 & 3) and my first pregnancy tests were all negative until 3 weeks after I conceived (I think I possibly ovulated late in a cycle) so dont hesitate to retest at weekly intervals if you still have symptoms.
After the baby, I refused to take the pill again and kept track of my cycles and tried to determine if and when I ovulated. As my periods returned with some regularity I had blood tests over 3 months that showed that I ovulated once, probably not the second time (probably tested too soon, my cycles were 25-38 days so hard to pinpoint mid cycle), and not the 3rd. I became really good at spotting ovulation (cm etc), and when we were ready to TTC number 2 I amazingly got pregnant first time. Same thing with number 3. Although our family is complete, I am never going back on the pill again!
At 35 neither age nor your ovaries are sadly on your side, so go to your GP now and say it has been a year, if not 2 (I was advised so by a nurse who thought I had been an idiot for waiting so long before seeking help). Referral takes time, use that time to TTC naturally. You are not infertile, just a lot less fertile than most, so get on track to get all the drugs you will need. Thankfully Clomid, though not the most pleasant, works well for PCOS sufferers like us. If you are overweight, try to lose some weight. You can take comfort that your first baby might "fix" you and you might still be able to have more than one.
Good luck!