Name changed because it's outing.
We moved to Brighton 10 years ago with 3 kids under ten. Admittedly the city is a little rough around the edges in places, but I have not regretted it for one minute. I guess it's a bit of a marmite place in that you either absolutely love it or detest it with every inch of your being! Housing is very expensive and I don't know too much about Patcham so can't comment on that specifically.
Cons:
Parking is expensive & driving discouraged so it's better to get a bus to the seafront, but you have to be a bit more organised than if you live within walking distance
No decent department store
There are homeless people - it's attractive because there is quite a lot of support for homeless people and it's rarely extremely cold.
Housing is expensive.
There are hordes of tourists if you go down near the pier.
It's popular with drunken stag/hen idiots
Pros:
It's bloody lovely being near the sea and the South Downs.
There are a number of good schools and an excellent sixth form college.
People are generally friendly
It's very inclusive
There are nice restaurants & independent shops in a relatively small space
It's easy to go out and get a bus or taxi home in 15 mins
There is enough beach / coastline to get away from the very busy tourist hotspots
There are lots of free and paid for activities for children and adults
There are strong communities / interest groups
The eastern seafront is being regenerated slowly
My criteria was that I wanted to be near the seafront, and we are. I think that makes a big difference. The downside to that is that it takes 15 mins to get through the traffic and onto the A23.
Think about your needs - do you need to regularly drive out of Brighton? Will you take a train? Think about schools - would you be willing to move again for secondary? If you are Catholic there's Cardinal Newman which my friends have been happy with. Do you want to walk to coffee shops etc?
I'd say to look at Rottingdean / woodingdean / ovingdean but Longhill (the secondary school) doesn't have the best reputation but you could always ask to visit & see for yourselves.
Shoreham is lovely but I don't go there that often - I do know families who have moved to Worthing for a bigger house but traffic is hideous in and out.
We rented for a year before deciding on the area so in a way I would recommend that - but of course it's complicated by schools.
I go to Haywards Heath / Burgess Hill and the villages Hurstpierpoint / Ditchling / Hassocks regularly and all these areas have a very different feel. If I had to choose I would go to one of the above mentioned villages as there are some very good schools but it's a v different vibe from Brighton!!