My ds has severe allergy to milk and eggs. His reaction is anaphylaxis, ie. swelling of the mouth/throat, urticaria (hives). He also had colicky tummy and eczema as an exclusively breastfed baby, which reduced significantly when we discovered the allergy and I went dairy-free.
A friend's ds has tested allergic to egg and he gets the opposite reaction - low blood pressure so he goes very quiet & floppy, but is magically revitalised by antihistamine.
My understanding is that intolerance usually manifests itself with digestive symptoms eg. runny tummy, tummy ache, acid/reflux and also in adults (so I guess kids too) migraines or similar chronic and supposedly unexplained symptoms.
Probably someone else here has more experience than me but in the case of my ds it is potentially fatal for him to take in (or even touch) the substances he is allergic to, whereas if yours suffers nasty symptoms but not life-threatening, it's probably intolerance.
Of course the management may be similar (total avoidance being the main thing) but with intolerance you can afford to try out limited exposure (eg. cooked or raw, goats or cows milk, less or more of it).
BTW what did he have before cows milk? Formula is made from cows milk, and breastmilk also carries through some components of what you eat.
HTH and good luck in resolving it so he can be comfortable.