I love travelling with DD, but from about one to three years old I wouldn't have gone anywhere long-haul. The flight would've been an ordeal, and the amount of actual enjoyment we'd have got from the trip wouldn't have counterbalanced the effort plus ticket cost.
But that was because DD was an early walker, extremely active, and a bolter: basically we needed somewhere small and childproof, or vast wide open spaces. Fortunately, she's a lot more chilled out now, and we took her to Japan when she was 5 and all had a great time.
I do get what people have said about missing out on the nightlife, though - many urban places like Osaka look a bit grey and uninspiring all day but come into their own at night: red lanterns, neon, giant glowing billboards, thousands of tiny speciality restaurants selling okonomiyaki, gyoza, takoyaki, stand-up ramen stalls, tiny alleyways between blocks of tall buildings absolutely packed with bars and karaoke booths and izakayas. There are always so many people out and about, and it's really atmospheric.
The good thing about coming from the UK is that you're all awake really late, so we did get to wander around much later than we usually would, but there's a limit to how much you can get into that with a small child. Nothing to be done about it, though, unless you want to ask your parents to come along and babysit...
FWIW, I would also avoid ambitious multi-stop tours: Tokyo, Osaka/Kyoto and Okinawa is a lot of travelling, a lot of packing and unpacking, and I don't think you really enjoy the later destinations as much as you might as it turns into a slog. I always want to cram as many places as possible into a long-haul trip, but since DD came along it ends up detracting from the experience rather than adding to it. (Not that I've been anywhere more ambitious than Sainsbury's this year.)