This is very interesting data.

One observation I would make is that like many education experiments, it does not control for (what I call) the "highly motivated researcher effect". What I've learned from a lifetime of "new ways to teach" is that the standard experiment is to parachute in a highly motivated researcher into the classroom to introduce the new method, collect data showing improved learning, and then advocate for the new method to be rolled out more widely. However, rolling out more widely involves taking regular teachers (good, bad, and in-between) to learn and apply a new approach, and techniques often fail in the face of teacher lack of enthusiasm to learn anything new, complaints it makes more demands on teachers to use the new method, etc. In this report it says " the program staff provide Wikipedia training and expertise so the faculty do not need to have any experience editing" which is a big red flag to me. It would be interesting to see the results in an experiment where you first train the faculty and then the faculty carry out the engagement with students. And then see the results in 3 years time when it's a case of "business as usual" rather than "the new thing".

As a general comment, students like the variety of someone new in their classroom. Students do tend to learn more from "real world" assignments than "lab" assignments because the real world is more complex. However, staff and students are often reluctant to have real world assignments significantly influence end-of-term marks/grades because of the uncontrollable variables in the real world assignment that makes it difficult to assess the relative achievement of the students. I would expect editing Wikipedia articles to suffer from this problem as each student will be working on different article(s) of different starting size and quality and with different levels of involvement and monitoring by other Wikipedians. It was not clear to me from the report if students were being assessed on this Wikipedia assignment and how important it was to their overall mark/grade.

-----Original Message----- From: Wiki-research-l [mailto:wiki-research-l-bounces@lists.wikimedia.org] On Behalf Of Zach McDowell Sent: Tuesday, 20 June 2017 8:30 AM To: wiki-research-l@lists.wikimedia.org Subject: [Wiki-research-l] Student Learning Outcomes using Wikipedia-based assignments

Hi Everyone,

For the last year I've been working on a fairly large (social science) research project studying student learning outcomes using Wikipedia based assignments with the Wiki Education Foundation. This was a mixed-methods study designed to address a variety of research questions and provide open data for researchers to dig through, analyze, and utilize in whatever way they deem fit.

Today I am happy to announce that the research report, the data, the codebooks, and many other supporting documents have been released under an open license.

The research report mostly summarizes the preliminary analysis (there were a LOT of questions) of some of the qualitative and quantitative data, but it is also meant to help understand the larger scope of the research project as well. Although this is just a preliminary report, I am working on a few journal publications with this data, so this should lead to more than the report (on my end at least).

If you are interested in student learning, new users, information literacy, or skills transfer, I hope this report and data set finds you well.

Blog post by LiAnna Davis on WMF Blog: https://blog.wikimedia.org/2017/06/19/wikipedia-information-literacy-study/

Full data set (zip file): https://github.com/WikiEducationFoundation/research

Research report (commons): https://commons.wikimedia.org/wiki/File:Student_Learning_Outcomes_using_Wiki...

best,

Zach

-------------------- Zachary J. McDowell, PhD www.zachmcdowell.com _______________________________________________ Wiki-research-l mailing list Wiki-research-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wiki-research-l