The late afternoon naps are a problem. Also, in my experience it is unusual for a toddler to just drop a nap, just like that. There is a transition time that lasts a long time - 6 months to a year. You sound like you are in that transition period.
Once he has transitioned to no longer need a daytime nap at all, then his nights sleep will settle too. But currently, he isn't there yet. Often he still will need sleep - but not with the daily regularity that was happening for the first 2 years of his life.
So you have to consider that transition period and build it into your daily life. He was having a post-lunchtime nap so a good idea would be to established the same time as quiet, relax time.
When we are doing this transition, I would never, ever arrange to be busy post-lunch (just as was the case when it was naptime). However, I would no longer put child 'to bed' after lunch. But I used to settle toddler on the sofa. Make it as comfy as possible, pillows and even blankets if needed. Allow, even encourage, a sofa nap after lunch.
Moving naptime to the sofa means it is likely to be a less deep sleep than properly going to bed. More like a powernap. Even if he doesn't sleep (which he may well not lots of the time - he is in the process of dropping his nap afterall) at least he is having some quality rest time.
Then, mark the end of quite time very deliberately every day. Before, he would have 'getting up' (out of his cot) when he woke up. Now, don't let quiet time bleed into the rest of the day. Have a time when quiet time ends and do something very different at that time - go out, go to the park, go to the shops, whatever. Just to mark the end of quiet time.
Then, pay very close attention to keeping him going and active late afternoon. Especially on days when he has not napped during quiet time. It is not unusual that during transition to no nap that the child will fine it difficult to go all day without sleep. So he will need help getting through.
This wont last forever. he will gradually get better at coping without any daytime sleep and when that happens his night time will stabilise too.