So you're basically saying: The community is good enough to be called upon to express their trust initially, but they can never express a change of heart regarding that trust? Sounds weird.
It's not a concept without precedent. Lots of US judicial positions work that way, for instance.
The status quo in US politics may not be the *gold standard* we'd want to model Wikipedia after.
Not being able to easily fire a judge (in this example) is actually a quite common thing. The concept is that it goes towards the whole judicial independence thing where a judge don't have to worry about being fired for a correct but unpopular decision with the politician etc. i.e. Not making decision based on popular-ism, but rather whether the decision is right or not.
One can use the same line of argument regarding adminship. Namely that an admin shouldn't have to worry about axe-grinders when carrying out a decision.
KTC