A lot of people seem to take airport security searches very personally, as if the are somehow being individually judged. The fact is it is a necessary and essential part of the airport and air travel. I've been travelling up to 6 times a day through airport security as part of my job for the last 20+ years.
Like it or not, airports and aircraft are seen as huge potential terrorism targets, and each airport does it's absolute best to trap any attacks. The security staff will have been told exactly what they are looking for and that will sometimes be something very specific.
Security consists of you first of all submitting your APIS - your passport number and other personal details. This is compared against a long list of wanted people or people who are on a government 'watch' list. To get on this list you might have carried out a number of 'suspicious' activities that can be as innocuous as travelling on a one way ticket to a particular destination, buying a ticket in cash or at the airport - especially a one way ticket, making an unusual flight connection, or having a name, address and date of birth or any combination of the above that is similar to a known 'watched' person. It's an unfortunate fact but security don't single you out because you are 'brown,' but if your name is Middle Eastern there are a lot of people on watch lists who are from the Middle East, and without further information they cannot assume anything about you without further inspection. There are any number of other things they are looking out for, but the first stage is a very broad brush automated stage. Every single person who flies through an airport is subjected to these checks, they occur without your knowledge or presence being required.
Once you get to the airport another layer of checks is carried out by the airline when you check in. Airlines are required to check visas and travel permits, they are fined if they allowed you to travel without one so it is in their interests to check this carefully. A one way flight without checking in bags combined with other information could be a red flag. In all reality lots of people fly with no bags, but the layers of security are being built up early on.
By the time you get to the actual security check if they want to scrutinise you further they know exactly who they are looking for. However, they also have to check everyone for random threats and careless security breaches. The are a major line of defence against both targeted, specific attacks made by coordinated teams of attackers and lone wolf type attacks. Both kinds of attack are possible - the 9/11 attacks were orchestrated, the storming of the flight deck by a psychiatric patient on a Nairobi flight was a lone wolf attack. Both were very real threats to aircraft safety.
Terrorists are getting more and more sophisticated in their attempts. I am not allowed to give examples of things security have found, but some of them would surprise you (they surprise me! I am given regular briefings on aviation security as part of my work.) Things you may think harmless could be something they have been warned to look out for. They are looking for certain items in your baggage but they are also looking for the way you look, move and act.
Examples I can give are of an attack and an attempt from many years ago, but both show that the public cannot second guess what might be going on. The Lockerbie crash was caused by an explosive in the hold. A message had been sent to the FAA (known as the Helsinki Warning) that a Finnish woman would be an unwitting courier who would place a bomb on a flight out of Frankfurt. It stands to reason that this means they would pay particular attention to all Finnish women travelling out of Frankfurt, but at the time security staff could not allow the public to know this information. Sadly on this occasion they failed. A couple of years earlier a bomb was intercepted by security staff at Heathrow being carried by a passenger travelling to Israel. She was a 5 months pregnant, blonde haired Irish woman travelling to Israel to marry her fiancé. They had planned to travel together, but he had placed the bomb in their luggage and then told her he had decided to catch her up on a later flight.
We do not know what security are looking for, what inside information they have, and nor should we need to know. What we have to do though is accept that security is a permanent part of aviation. So far in this thread there have been people complaining that they are singled out because they are white, because they are brown, because they are fat, (all their words, not mine.) But no one is singled out unnecessarily. There will be a reason for it. My daughter recently complained she has been searched on every flight this year. She wears the same clothes each time, and on her most recent flight the friendly security staff told her that her sequinned t shirt was setting off their sensors. Most metal detector sensors also detect cumulative metal, so a lot of little pieces added together will set off the detector. In my case when I go to work I am wearing an underwired bra, have a metal last in my shoes, have metal buttons, belt and badge on my uniform (hoping the new airline uniform is a little less metal-y!) and a pen in my pocket. I know that if I take off my shoes and belt and pen the rest of it will not usually set off the detector.
Anything at all in your bag that should have been taken out will trigger a search, and the problem is that each airport is different as they will be dealing with their own internal knowledge and systems. Some places you can leave everything in your bag, other places leaving a Kindle in your bag will trigger a search. A polite request to staff about what they need out of the bag can usually avoid this.
There are also random searches - most of the arches are set to trigger a search on a sequence determined by random number generator. A certain proportion of all passengers must be searched, but not in an obvious pattern or sequence.
Once you are through security airport and airline staff are still trained to watch for anything unusual. The last line of defence stops with me, in charge of the aircraft itself, and I would rather no passengers will ill intent got so far I have to deal with them personally, but occasionally it does happen. Not to me, touch wood, but we always have to be vigilant.
People have mentioned immigration and customs too in this thread. They are a whole other topic. But they also have their triggers as to who to stop and question or search. They are looking for things that are not a direct risk to the aircraft or airport so are perhaps a little more direct in their line of questioning. Customs are looking for smuggling of drugs, alcohol, cigarettes, but also items like vegetables, plants, meat and animals (biosecurity is a big thing,) and goods which should have had import duty paid on them. They usually know which flights to target for this. Having had a passenger die on board with a stomach full of smuggled drugs it's not a pretty thing so customs are fairly essential. Immigration are looking for people who should not be travelling, either those without visas or right to travel, but there is also a big anti child and sex worker trafficking and anti slavery movement going on. A child travelling on their own will almost always be picked up on this (my daughter carries various documents explaining why she is travelling on her own) and also parents with a different name to their child if travelling together will be queried.
Hope the above explanation helps - it's not personal, and there's lots we can do to help security. If it were me running the airports I would ban hand luggage for a start, it would make the queues and numbers of people much easier to deal with. But people want to travel with all their stuff so for now we will have to put up with the queues and hassle that security entails.