If you have a car, go to Dungenness, weird place, lighthouse at the end, see Derek Jarman's cottage and garden - Prospect Cottage.
For a longer trip you could get across to Hythe - the Howletts Zoo at Port Lympne is extremely good - massive gorilla enclosures with gorillas swinging about and beiing very entertaining, tigers, a safari trip through elephanst, giraffes etc. How far away if Battle? Battle Abbey and the Hastings Batle field, which is very well explained.
In Rye itself there is a nice little cafe called the Apothecary, god for soup-bsed lunches, and The Forge Restaurant is a reliable good quality steak / grill house. Beware - there is a wine bar that looks inviting for food on the quay, but they won't allow children!!
Walk up around the Citadel, through all the winding cobbled streets - you get great views fom all round the top.
Winchelsea beach and harbour