That lonely bit fades in a few days and families are far from perfect, you never know what is going on behind closed doors so stop looking at them. You are just seeing a snap shot of what could be a rare half decent moment. Scratch the surface and most replationship you will see will echo at least ONE of the threads on here.
You are not missing out on anything and neither are your children.
The evenings bit are the hardest, you just need to find something to fill that time with.
Studying, hobby, craft, box sets, books, pampering...anything. Once you find something you enjoy, you'll be begging the kids to go to bed early.
You need to sort out a contact area, either a relations house or a friends for now. Make set days where he can see them and stick to it. The boys need routine now more than ever and knowing that will see their dad on such and such a day will help them no end.
You will get a break, it's just really draining the first few weeks. You'll get through it, you have no other choice.