Never mind the tone deafness of it that can only be found among those who are privileged. But, here's a common sense logic: diversity of race, ethnicity, background, etc. brings natural diversity of opinion to the table.
If everyone at the table is white, male, between 45-60, have middle class backgrounds, and all attended university, how does this breeds diversity of opinion? Not to say all white men will have the same opinion, but logic follows that people of the same age, race, background, etc. will have more similarities than differences.
A black woman from a poor background will have a different view of the world from those held by a white man from an affluent background. That is not to say they can't have things in common, but their approaches will be different.
In actuality, there's nothing wrong with the "like-minded" opinions of white men, but if you're arguing for diversity of opinions, having a table with an entirely white while body is defeatist to this cause.
We're all biased. We all have things we like and others we don't. We have standards we prefer and others we don't. Nothing wrong with that, but please beware that one's aesthetics/preferences are not superior to another's based simply on the fact that it is accepted/embraced by the majority that dominate that particular field.
Those who oppose diversity often create a false association. They argue that diversity lowers standards. So, to make room for the voices of others, one is lowering the standards. This assumes that a) the voice of the dominant group (e.g. white men) is naturally superior to those of the newcomers to the table (e.g. nonwhites). This is not so, not necessarily.
A reader who grew up reading Tolkienesque works of Speculative Fiction may have difficulty reading/liking fantasy stories that takes place in Africa or Asia. Does this mean the work is bad? No. Our tastes, biases and standards are idiosyncratic, and can be shaped by cultural experience, background.
Diversity will lead to discourse about what is good and isn't good. There are universal factors that can determine a story's qualities, such as voice, use of language and plot, etc.
For these reasons, I am wary of agents who write things such as "I just can't relate to this character" or "I'm not passionate about this character." I don't know what any of this means and I have to wonder: a middle aged white woman telling me she can't relate to a teenaged black girl from the inner city. Okay!
If only people are made aware of their prejudices/tone deafness and [lack thereof] empathy.