Sorry, had to go in a hurry last night, child related issues...
As NQC says, it should make a difference but as it is residential only a building used for a relevant charitable purpose would qualify
(This is from public notice 708 - Buildings and construction)
3.1 The basic conditions for zero-rating the construction of a new building
3.1.1 Introduction
If you construct a new building you will normally have to charge VAT at the standard rate. You may, however, be able to zero-rate your supply if you are involved in constructing a new:
- eligible dwelling (referred to as a building ?designed as a dwelling? and explained at paragraph 14.2);
- building that will be used solely for a ?relevant residential purpose? ? see paragraph 14.6; or
- building that will be used solely for a ?relevant charitable purpose? (for non-business use or as a village hall ? see paragraph 14.7).
14.7 What ?relevant charitable purpose? means
14.7.1 The definition
?Relevant charitable purpose? means use by a charity in either or both of the following ways:
- otherwise than in the course or furtherance of business ? see sub-paragraph 14.7.2;
- as a village hall or similarly in providing social or recreational facilities for a local community ? see sub-paragraph 14.7.3.
Buildings typically seen as being used for business purposes:
- membership organisations where the organisation charges a membership fee
- school buildings where a fee is charged for the provision of education;
- offices used by charities for administering business activities, such as fund-raising events where an entrance fee is charged; and
- village halls and similar buildings, but see below.
I'm really sorry, like the others say, get more advice but I honestly don't think there is any way round it.
Might be worth having a chat with your builders to see if they can reduce anything as they can still claim back all their VAT, the difference between a cheaper job and no job might be worth it?