With me, it's more about just eating what I want to eat, and what makes me feel good. I'm not really concerned with the science of it or the "rules" per se.
I used to do a big milky cereal breakfast almost religiously for years, but tastes change. Now it just makes me feel gluggy and overfull. I've never been heaps into toast or sandwiches, and I've always drank coffee and tea black. Outside of that, there's not really a whole lot of situations where I'd be actively avoiding dairy or wheat anyway. I've never really "craved" dairy or wheat, and I don't consider them "treats" or "cheat foods".
I'm similar. I don't have any desire to drink milk, the only dairy product I have is grass fed butter and cheese every now and then. Wheat is the more difficult thing for me to personally give up. I can be very good for a long period of time but I always need to keep an eye on it. I love pasta, bread, pizza and the like, but I don't feel great after I've eaten it, especially when you compare it to how I feel normally.