So how to not make it a troll magnet?
That is the obvious problem, yes. My suggestion about making it so only admins can take part was designed to solve that problem, but it does introduce problems of its own.
It is quite common for groups to have rules about passing motions of censure against its members. It does seem appropriate for it to be admins that censure admins.