There are some odd responses on here OP
I am sorry you have had such poor treatment by men so far, shaking your trust and belief in relationships generally I'm sure.
I get that you talk about your period, some couples do, its part and parcel of being in a relationship with a woman, she has periods, what is all the hand-wringing about blimey.
I also heard that theres a difference between answering his question and commenting that its normal for your relationship to talk this way.
Some are looking where there's nothing to see.
I also read your OP as him having lied to you by him apparently going off to work at the same time as you, yet when you drove past his house afterward he was back at his house. Now if he'd forgotten something (which I have to say at this point is what I would have assumed), he wouldn't say he was taking the day off without some sort of sensible reasoning as to why, especially as he's already spoken to you about work stresses.
So, for me, it doesn't add up.
I don't know if I would have stopped as I would have assumed he'd forgotten something, but I probably would have asked after his day to see what had happened when meeting or messaging later.
You have been abused in the past, and its always worth making autre you are emotionally and psychologically ready for the next one otherwise its not fair on either of you. I would not have checked his phone, that feels weird and stalky.
Thats a line very much crossed and I'm glad he caught you. This is something you have to have a grown up talk about and confront. How can he trust you now?
I hope this willl bring things out for you both so you can either discover he's not being straight with you, or that there's stuff going on for him that he might not want to talk about yet, but definitely that your trust issues are seeping into the relationship.
Its a horrible way to live and it may be that your senses are tingling for some as yet unclear reason, so see him.for what he is, whatever that may be, good or bad. I would recommend the freedom programme like pp, as part of your moving forward, it will give you perspective around the way you have been treated and very clear signs to look for which it sounds like you could benefit from.
I really hope this is a relationship worth working at and that you can both manage to do that.