When Ds started having reaction to milk (started with redness around the mouth), I eliminated other things that we had recently started, incl. wheat until I got allergy testing. It was easy though, b/c we were just starting foods.
ETA: and yes, a few with milk allergy also can't have beef (and some with egg allergy also can't have chicken). FWIW, DS's milk (and mustard) allergy started with some flat redness around his mouth that I wiped off and it went away, but the 3rd exposure was full on all face hives, vommitting, etc. It could be just a skin irritation, but it could be allergies. Too bad your ped was so dismissive!