You are doing great, OP. Please ignore the posters who are being nasty and judgemental. I've had two babies, one who slept fine as a newborn when I would put him down after nursing and one who fussed a lot and wanted to be held to sleep like yours. I expect some folks don't know how tough it is to stay awake with a baby who won't be put down.
For our fussy baby, my DH and I had a system for the first five weeks where he would hold her sleeping from 8pm to midnight. We would set up a station for him with snacks, telly and mobile to keep him awake.
At midnight we would switch and I would have her feeding and fussing for several hours with the same set up -- snacks, telly and mobile. I would try small stretches of putting her in the Moses basket and once she was able to stay down for an hour or so (around 5-6 weeks), I started the transition to nursing in bed and putting her back in the Moses basket after each feed.
You are trying to do what is best for your baby and so you are a good mother and on the right track. Find the setup that works for you and DH. This period won't last forever, but while you are in the thick of it, try to get at least a few hours sleep for yourself each day/night.
Do you have any other help? If someone can come hold the baby for two hours while you sleep during the day, that could really help. If you don't have any support but have a little extra cash, you can pay a postpartum doula to do this.