Hi,
On Wed, Jan 14, 2015 at 12:07:57PM -0600, Aaron Halfaker wrote:
For example, not collecting usage data about certain sections of our population (e.g. IE10 users where DNT is set by default) means that we don't know if our software works for them.
If WMF's main form of QA was through automated usage data collection, you'd have a point.
But actually, I think WMF is doing better than that.
From my point of view, a central pillar in QA is “software getting tested”. That's happening widely across WMF. Both manually and automated. It's great already and getting better every day.
And for me the main QA ingredient is listening to feedback from the users. Besides studies and dog-fooding, WMF's bugtracker is a testament to that and contains reports that “$X is not working on browser $Y” or “$X needs to also do $Z”. And that's really great!
To me, user behaviour data collection is a way to support and assist the above two. But it is not a requirement when trying to determine “if our software works for them”.
Users are sending us emails about issues, come to IRC to discuss issues, file a ticket, or they just tell someone. All without having their usage data collected.
I am convinced “IE10 users that do not want to unset DNT” are no exception to that.
Have fun, Christian
P.S.: I for one received bug reports from IE10 users. (But I do not know whether or they used DNT.)