I have visited Carlisle Bay before for dinner whilst staying elsewhere in Antigua. Whilst the hotel is undoubtably plush, I would not say that the location is idyllic.
The hotel is in the middle of Old Road, a relatively poor village on the west coast. It feels rather isolated from the village due to the substantial security gates! Depending upon whether you regard Fort Knox style security as a positive feature this may or may not bother you
The main concern I would have is the hotel beach. I understand from speaking with locals (and my DH who has visited Antigua since the age of 2!), the beach can be plagued by seaweed, which may scupper your fantasy of crystal clear water.
Seaweed issues aside, if you are looking for luxury you would be hard pressed to find anywhere better in Antigua. There are however a couple of other possibilities you may like to consider. Have you looked at either Curtain Bluff or Hermitage Bay?
I know the island fairly well so am happy to help if you need any other info .
Sister in law and family just got back. They said it was great but not amazing and the food was v v expensive in the hotel, so much so that it took the shine of it a bit(and they are v v wealthy but not stingy IYSWIM). But that aside they had a fab time!