It seems that people are well enough informed by the field studies that our
team runs to want us to continue to run them. In fact, demand has
sky-rocketed both within and outside of the Wikimedia Foundation. You hold
a minority opinion that testing software in the field is unnecessary. Yet,
field tests are considered a best-practice and have become a critical part
of our strategy for minimizing the disruption (and maximizing the benefits)
of software changes. I don't think that many people would appreciate your
proposed strategy of releasing the software and waiting for people to
complain. Given how difficult it is to develop good user-facing software,
it's likely that every major deployment would be disruptive if we adopted
that strategy. I can speak for a few disruptions that my research helped
prevent and some opportunities that it helped us explore.
Allow me to share a specific example. In this study, we found that
telling anonymous editors to register dropped their productivity by *25%.*
Yet we didn't identify substantial issues in user testing. If we had not
run this field experiment, we might have deployed the change thinking that
we were improving Wikipedia when we were really driving good editors away.
During the experiment, we received no substantial negative feedback
For a large collection of field experiments that were used to iterate on
Wikimedia software, see: https://meta.wikimedia.org/wiki/Growth
Really, what I want to say is this: If you want to improve privacy
protections, I am your ally. We're merely disagreeing about whether it is
good to assume that DNT means something it wasn't intended to mean or not.
However, when you say that my work has no value, it's hard to talk to you
productively because, honestly, I don't think your opinion is
On Thu, Jan 15, 2015 at 7:22 AM, Christian Aistleitner <
On Wed, Jan 14, 2015 at 12:07:57PM -0600, Aaron Halfaker wrote:
For example, not collecting usage data about
certain sections of our
population (e.g. IE10 users where DNT is set by default) means that we
don't know if our software works for them.
If WMF's main form of QA was through automated usage data collection,
you'd have a point.
But actually, I think WMF is doing better than that.
From my point of view, a central pillar in QA is “software getting tested”.
That's happening widely across WMF.
Both manually and automated.
It's great already and getting better every day.
And for me the main QA ingredient is listening to feedback from the
users. Besides studies and dog-fooding, WMF's bugtracker is a
testament to that and contains reports that “$X is not working on
browser $Y” or “$X needs to also do $Z”.
And that's really great!
To me, user behaviour data collection is a way to support and assist
the above two. But it is not a requirement when trying to determine
“if our software works for them”.
Users are sending us emails about issues, come to IRC to discuss
issues, file a ticket, or they just tell someone.
All without having their usage data collected.
I am convinced “IE10 users that do not want to unset DNT” are no
exception to that.
P.S.: I for one received bug reports from IE10 users. (But I do not
know whether or they used DNT.)
---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
Companies' registry: 360296y in Linz
Kefermarkterstrasze 6a/3 Email: christian(a)quelltextlich.at
4293 Gutau, Austria Phone: +43 7946 / 20 5 81
Fax: +43 7946 / 20 5 81
Analytics mailing list