The way mail servers treat messages is quite variable, and unfortunately it seems so many are set up by young kids people who never knew what it was like when it could take a few days to get through, so they seem to set very short (48 hours or less) limits before throwing away failed mail.
I think CruelAndUnusualParenting is optimistic about there being retries for up to a week... I think with the growth of e-mail (and spammers etc) there must me a lot of mail which is "return to sender" sitting in queues, and staff have perhaps reduced the number of hours/days to not allow their servers to get overflowing with e-mail - it runs to hundreds/ thousands of GB for a big ISP with a couple of million customers, and must be a big cost.
The sender should get a message (in days gone by it was common to get a 'mail has been delayed, still trying' message at the sending end, until it was either sent OK, or failed, and there'd be a 'sorry, given up on sending this' message).
The senders might try again, if the error message they get says "mailbox full", but if they just get some arbitrary "mail failed" then either they will think "too many applicants, too much hassle to chase this" or "one more go", so is partly down to the sender and the server they use, and you use (what message goes back).
Days gone by it was common to double the time between tries. So sending at noon on Wednesday, it might try a second time at 1400 then 1800 then 0200 Thursday, then 1800 Thursday, maybe 1800 on Friday or possibly give up. Depends on the server... You'd see "new" messages from old dates (and in random order, because one message might have 4 goes, another only 3, as it was sent after the other, but while the first was waiting 16 hours before trying again, the message sent an hour later had just finished waiting 8 hours and there was room in your mailbox when it tried to send to you...
I know many ISPs offer free e-mail accounts, but it is sometimes worth paying a fee - I've used Runbox.com and Fastmail.net for some time... they both do free trials, and the Fastmail service can be as cheap as US$5 a year (and can be made to work even from behind an office or ISP firewall that blocks all but web browser traffic). You can buy extra mail storage space at modest cost from Fastmail these days - much less than I paid 5+ years ago.
Runbox is more expensive but gives 10 GB of mail storage and 1 GB of file storage and can give you free web hosting of your own domain so you might have [email protected] (or surnamefamily.co.uk if 'surname.co.uk' is already taken)
Google Mail allows several GB of mail so won't 'fill up' but some people might not want to use a free mail account at Google, for job vacancies.
Back to Fastmail (for the benefit of anyone else, esp anyone stuck behind a firewall) - they use a proxy system whereby you can access each service (POP, IMAP, SMTP, etc) through any port number. See bottom of page for details. They charge $20/year for 600 MB of mail storage, and $40 for 6 GB of mail storage (though both reduced by $5 for signups).