1. To be trusted at least minimally. Not to have it assumed, for instance, that because you may be X, you will surely oppose and work against Y.
2. To be respected/appreciated at least minimally. For example, not to be told that while your field needs representation for reasons having to do with accreditation, it is not actually welcome.
3. Not to be resented. Specifically, not to have people decide your degree itself or the institution from which it comes, or your latest publication, is a personal affront to them.
I would say these are the three main things. It is good not to be harassed or bullied but these things are more basic. It is alienating not to be on team but then also not, for strange and unimaginable reasons like these — reasons more destructive than any more rational ones.
Of course one’s research colleagues are elsewhere but the real life atmosphere one lives in every day matters, at least to me.