YANBU. I totally get where you are coming from.
I have a 17 month old DD and a 3.2yr DS. They are still at the intensive parenting stage so it hard work with both of them around. Plus DS won't play with DD most of the time, he grabs toys off her making her cry, pushes her, sometimes hits her, so they need constant supervision when they together, which means it is difficult to get anything done. I can't give either of them quality time, as I am bogged down with chores, stopping DS from hurting DD etc.
But when I have just one of them with me, it is sooooo much easier. DS especially is a totally different child when he is alone. When he is with DD, he is attention seeking, naughty, destructive etc, but when she is not there he is calm, will play with toys by himself, read book etc. I love going to a cafe with him and having a meal together, then read a book and have a chat. Things I can't do when DD is around.
When I have DD alone, I can make her laugh, read books to her, let her walk at her own pace, without having DS dominating our schedule.
At the weekend, me and DH spend half a day alone with each child. It works really well.
I think when they are older, hopefully they will play together, so might be easier than having one.