In my experience, if it's a problem, cafe owners will bother people asking if they want anything else, or hurry them on to leave. However, good cafe owners know the regular customers, the sort that turn up in a large group, week in, week out, sometimes spending a lot, sometimes (like possibly this week) spending less, are the customers worth keeping. That if you open a cafe in an area where the only people around during the working day with money to spend are Mums with small children, it's worth taking the occasional 'loss' day in order to build a 'great place to go with children' reputation.
The 'one off drop ins' are great if you are located in an area with good footfall, but for most cafes, they need to win the 'at home in the week with money' local people - some areas, it's pensioners, some areas it's offering free wifi for the 'working from home' types who want a change of scene, and in others it's SAHMs with decent mat pay to spend.
There's a pub near us who've got the 'working from home' market covered, if you pop in some days it might seem pointless, they sit with laptops buying one or two cups of coffee for hours. Other days, they are full at lunchtime with people who've 'hogged a table for the last 3 hours' and now can't be arsed going home for lunch.
You went as a 'one off' in the week, and it was busy, it might not normally be busy on a Friday, it could well be these are valued customers and it was worth it to take the 'hit' on one week when in future weeks they will be the only customers in at that time. Few places can get a repuation for being just 'fast food' - not welcome to linger in the working day, yes they could meet at someone's house, yes they could do something else then pop in for a coffee afterwards, but they probably wouldn't pop in for a coffee afterwards if they had already caught up at a toddler group, they wouldn't on the way back from having a coffee at someone else's house.