Oh I love fireworks, too (though it took me many years to get over my fear of them, and I can still feel anxious if I can hear but not see them) but there's a time and a place. In the 2-3 weeks around 5 November, the Tattoo, publicised displays etc, you can be prepared for them. But randomly, late at night in a residential area - no.
I used to live next-door-but-one to a 'conference hotel'. Most of the time we didn't even know it was there. Throughout the summer, though, parties would regularly end with fireworks. 10 minutes barrage of utter noise. Not a display - everything at once. At any time of the night.
One time I woke in terror at 1am, thinking there was a gunfight going on. I know what gunfire sounds like. It was horrific.
Anyway, last night's 'display' lasted 23 minutes. Then the music started. Loudly. Within 10 minutes there were angry voices and shouting, and the music was turned down.