He's behaving like a 10 week old puppy, and he sounds like quite a distressed one at that.
Can't your partner help with some of the clearing up?
What breed is he? Did the breeder give you any advice on training, particularly relating to the specific breed, or have you read up on what to do? Crates can be great - contain the puppy and his mess (and he may be less likely to mess in his crate) and also provide a safe space for him. Make it cosy, warm, and cover with a blanket to make it enclosed and dark (comforting and less stimulating).
You need to be taking him into the garden every hour, and have a set phrase to encourage him to wee or a poo, eg "hurry up". Loads of praise when he does go.
He will need lots of cuddles and reassurance, and also lots of sleep at his age. Take him to the vet to check for any underlying issues.
Puppies are extremely hard work, as I'm sure you realised before you got him. However with time, love, routine and training things will soon change.
Presumably the breeder will be willing to have him back rather than him live a miserable existence, but don't you want to persevere?