If you don't want to get involved, and I can appreciate that you have better things to do than be continually stamping out stupid little fires, why not get (say) half a dozen editors you trust to have sensible heads on their shoulders, point them in the direction of the stupid little problem and ask them to keep an eye on it. Pick a different half-dozen each time. Think of it as an unofficial jury duty. Solid, steady, experienced editors are going to do the right thing, quench the flames, monitor the situation and if things get out of hand, they will call for assistance from their own resources. In effect, they put the thing on their watchlists.