Short answer (if you don't want to bother with the detail) is: unless you LOVE your current home, then move!
Long answer:
Has the work been looked at by someone with experience of the work e.g. builder, loft conversion specialist or a professional e.g. architect, surveyor?
Or is this just 'first thoughts'?
Quite often the staircase needed to reach the loft conversion usually takes out one bedroom unless the existing staircase is unusually large you would still have a 3 bed house.
As you are in a terrace you would have have no option to alter the roof height and limited options for dormer windows. If the roof pitch is not sufficient then headroom would be limited over a large area. Insulating the pitch rather than the loft floor is quite challenging.
The heating system would have to be altered quite extensively.
The EA's valuation is just that and you might not achieve the figure suggested.
The disruption and mess is usually horrendous.
As an architect, I welcome most commissions for domestic projects but am always very, very wary of loft conversions and have on several occasions actually turned down the work and recommended that the homeowner move.