Is it worth us soundproofing our walls?

[MinimalistMommi]

We live in a victorian terraced cottage. We are back to brick in some areas ready for new plaster to go on. We can hear voices next door should we soundproof while we have the opportunity?

To complicate matters we are having lime plaster so it would need to be breathable sound proofing/blocking.

We are thinking of using cork : www.mikewye.co.uk/Secil-CORK-BOARD-application-internally.pdf

Are we wasting our time though as I'm assuming we would not be able to sound proof our fireplace as we are installing a woodburner? Would that let any noise through anyway.

The side where we have heard most noises (conversation) is on the wall without the fireplaces (we have two in our terraced cottage)

The cottage is teeny tiny, so we are not able to put bookcases etc against walls to absorb sound, there simply wouldn't be space.