That kind of corrosive supiciousness is the problem. For the most part our administrators, those who are involved in backchannel operations, are the best and the most trustworthy we have.
I suspect that people are far more suspicious than is warranted by what actually goes on, but I think we can agree that suspicion is distracting and damaging. If admins made it an explicit practice that they would communicate privately only about certain, clearly defined issues; it might reduce some fo the mystery (though I'm sure some suspicion and paranoia would always persist).
I am curious, however, what is the measure of the "best and the most trustworthy" admins. My observation is that being elected to an admin role depends on not having too many detractors; and being deadmined is a result of truly egregious offenses. The corollary to that is that an editor who wants to be an admin must avoid contention in order to be elected; but can let loose once he/she is in. I'm not saying that it's common, but it bothers me to see how a lot of well-meaning editors can't bombarded when they ask to become admins by people they've had content disputes with; and then see that established admins more or less abandon the caution that got them their role to begin with.
Leif