On Mon, Aug 25, 2014 at 6:22 PM, Dan Garry <dgarry@wikimedia.org> wrote:
Honestly, I disagree with pretty much everything you just said. Even if we assume the bias has remained the same, we still don't understand how it transforms the underlying data, and without that understanding any conclusions you draw will be totally invalid and tell you nothing about the gender gap.

I agree that drawing any conclusions would be very premature. I just want to see what the data looks like and if it suggests any trends or sudden changes that may have happened over the past few years. I think we can all agree that the gender preference has some relation to actual gender. If the percentage of edits that came from people who identified as female suddenly increased or decreased at some point in time, wouldn't that be interesting regardless of our uncertainty that 100% of those people are actually female? If the percentage of edits that came from people who identified as female had been decreasing steadily over the past year, wouldn't that also be interesting regardless? I'm not suggesting we publish a paper about it. I'm just suggesting we look at the shape of the data and see if it suggests more specific questions. Right now, though, we have no information to go on, even to make educated guesses. Part of scientific investigation is forming a hypothesis, but that's difficult to do when you don't even have anecdotal evidence. There's nothing wrong with beginning an investigation with imperfect data. That's how most investigations begin.