And again these are systems issues. I agree that MediaWiki is a complex
system. But one would think, given the significant focus on data
collection, any acceptance testing before a significant infrastructure
change would include a review to ensure that it will not impact data
collection. (This is the kind of thing I mean when I say "are we doing the
right tests".) We know that we've repeatedly seen implementation of
publicly visible/publicly editable content creation extensions that don't
work with Checkuser, aren't logged like content, aren't deletable or
suppressible, can't be edited by others and so on - all of which should be
standard requirements that should be tested for whenever such an extension
is deployed to a production wiki in the Wikimedia cluster (and I'm pretty
sure other MediaWiki users would consider those to be standard expectations
too). But these aren't tested for because they aren't part of the standard
testing suite.
[A little offtopic]
I don't think lack of testing has anything to do with "We know that
we've repeatedly seen implementation of publicly visible/publicly
editable content creation extensions that don't work with Checkuser,
aren't logged like content, aren't deletable or suppressible, can't be
edited by others and so on". Testing doesn't help if you intentionally
don't do something (To make a generalization. That might not even be
true. But my impression is most of the time those things are "known
issues" that people either decide aren't worth implementing, or are
leaving to later).
--
-bawolff