I had this with my last downstairs neighbour.
He used to whinge about anyone who walked in my flat. Just walking. If I was sat in the living room talking, he'd put music on so loud the floor would shake. He used to abuse anyone who walked up the stairs to get to my flat... the postman, visitors, any delivery people...as they went past his door he'd open it and shout abuse at them. I had so many complaints, it was unreal - like I could control what he did! I had carpets and thick rugs throughout the flat, I'm half deaf so never had music playing...there was literally NOTHING I could have done to limit the amount of noise I was making...I was in the flat on my own 90% of the time!
It all came to a head one New Year's Eve. I'd had my ex husband come around to visit and he'd left at about 10pm. Downstairs neighbour was so incensed that I'd had a visitor - and was talking to them - that he went outside and started throwing eggs at my living room windows. He was then in the communal hallway yelling abuse up the stairs, threatening to kill my cats and beat my boyfriend up. That went on for about an hour. I got my ex husband to come back, pick me and my boyfriend up and we spent the night at his flat. I called the Police and the Housing Association, who immediately said they'd take him to court (turned out he'd done the same to 4 previous occupants of my flat). As a result of the court case, an order with immediate powers of arrest was inforce for 3 years, however I demanded that the HA rehome me asap, and with the added support of my GP and psychiatrist, it happened within 6 months. I spent 3 of those months staying at my ex's flat and sent my cats 200 miles away to stay with my boyfriend.
Long story short, you need to nip this shit in the bud straight away. Tell them that, no, you will not stop talking at 9.45pm and if they don't like it, tough shit.