On Fri, Nov 7, 2014 at 2:19 PM, MZMcBride z@mzmcbride.com wrote:
I think that's unfair.
Wikis have a serious spam problem. People associate CAPTCHAs with spam prevention. On the English Wikipedia, one of the actions that results in the user being required to successfully enter a CAPTCHA is adding an(y?) external link to a page as a newly registered user. This, of course, in addition to the CAPTCHA presented when registering an account (consider that many new account creations only come about as the result of the requirement for an account to make a new page on the English Wikipedia).
Why not disable the extension for a week and see what happens? If you're wrong and there's a marked increase in wiki spam (account creations and edits), then you can help devise a better solution than a CAPTCHA. CAPTCHAs are clearly not a sustainable fix to the underlying problems they were implemented to address, or so I think you're saying. If you're right and the CAPTCHA is simply ineffective and unneeded, we've eliminated some code from production and we can move on. In other words: what exactly is stopping you from disabling the extension?
When we did usability tests of the new version and the old version, it was exceedingly clear that what Jon has said is true: the CAPTCHA is by far the most painful part of our signup form. Results and videos of those tests at: https://www.mediawiki.org/wiki/Account_creation_user_experience/User_testing
Back in the day, when we did the last redesign of the signup form, we considered running an A/B test of removing the CAPTCHA. Docs for that are at https://meta.wikimedia.org/wiki/Research:Account_creation_UX/CAPTCHA
The "turn it off for a week and see what happens with spam" test is the easiest implementation of the above, and even though normally it's a great way to produce garbage data (not being a controlled experiment) I'd be in favor of trying it. Every indication we have is that we'd get marked increases in signup conversion rates (ours are not great -- only 30% of visitors complete the form successfully IIRC).
For some more background, when we proposed something like that to Chris Steipp he was pretty iffy about it, and he's not wrong. At other sites that don't have a CAPTCHA on signup (like Facebook, Quora, others) they avoid a spam problem in part because they require an email address and confirmation. For some irrational reason, even in the era of throwaway email accounts from web services, not requiring email is some kind of sacred cow among Wikimedians, even if it would make it an easy choice to throw away our wretched CAPTCHA.
If we want to avoid spam bots signing up, there is going to be a hit in ease of use somewhere. Our network of sites is just too large to avoid being a target. It's just a matter of testing to see how much we can reduce that hit, and which method might be less easy.