It's probably gaps and cracks in the party walls. They might be due to age, building work, and wartime bomb damage, but even when new, interior brickwork is often very shoddy, because it won't show once plastered over.
Start with any visible cracks, rake them out, hoover out the loose material, fill in with sand and cement mortar, finish off, once dry, with easilfill plaster or joint plaster, which is easier to use and cheaper than filler
Then look for hidden gaps, behind the skirting and under the floor, especially where joists are built into the wall. You can inject expanding foam round joist ends, there is a pink fire resisting grade. The can comes with a short tube to inject into gaps. It is very messy. Clean out all rubble dust and dirt first, and spray with water to moisten.
After filling the cracks, you can fill the void under the floor against the party wall with brown mineral wool, which will muffle air movement and slightly muffle noise. You can also so this under the skirting, especially against external walls, and it will reduce draughts. Airbricks must not be blocked.
Taking up floorboards in old houses is easier than you think. You can see where they were previously taken up by plumbers and electricians. Refix with countersunk two-inch screws (not nails), drilling pilot holes. Mark the runs of pipes and cables on the tops of the boards and try not to drill or screw through many of them.