Hi all,
Thank you for your interest in the 2020 Wiki Comparison
<https://docs.google.com/spreadsheets/u/0/d/1a-UBqsYtJl6gpauJyanx0nyxuPqRvhzJRN817XpkuS8/edit>
dataset. We received questions regarding the differences between Wiki Comparison and the
monthly Wikimedia Movement Metrics that our team also publishes[1]. After investigation,
we found that they were due to:
A bug affecting editor counts, which was present in the published dataset for 4 days and
has been corrected
Differences between metric definitions
Further details are below.
We hope this is helpful in addressing the differences and their causes, and we apologize
for any inconvenience the bug affecting editor counts may have caused. Please feel free to
reach out should you have further questions or require further clarification.
Bug affecting editor counts
Five months of editor data were double counted, resulting in inflated numbers in three
metrics: monthly editors, monthly active editors, and monthly new active editors (Column
H, I, L in datasheet ‘Dec 2020’
<https://docs.google.com/spreadsheets/u/0/d/1a-UBqsYtJl6gpauJyanx0nyxuPqRvhzJRN817XpkuS8/edit>),
and understated numbers in one metric: unique devices per editor (Column G in datasheet
‘Dec 2020’
<https://docs.google.com/spreadsheets/u/0/d/1a-UBqsYtJl6gpauJyanx0nyxuPqRvhzJRN817XpkuS8/edit>
). The datasheet with the issue was published on Feb 22, 2021, and corrected on Feb 26,
2021. During the time when the bug was in place, any data reference to a year-over-year
comparison or an absolute number of the four metrics would have been impacted, whereas the
trends across wikis within the same year were consistent.
Differences between metric definitions
Unique devices
In Wikimedia Movement Metrics, unique devices are reported for only one project family:
Wikipedia. Wiki Comparison contains data across 16 project families.
For the Wikipedia family alone, the total number of unique devices in Wiki Comparison is
25% higher than that in Wikimedia Movement Metrics. This is because the calculations are
aggregated by different cookie definitions:
Wikimedia Movement Metrics: unique devices are aggregated based on the last global access
cookie across all projects.
Wiki Comparison: unique devices are aggregated based on the last access cookie in one
project.
Therefore, adding unique devices of each project together resulting in a 25% higher total
in Wiki Comparison than in Product Movement Metrics.
2020 monthly avg
Wikimedia Movement Metrics
Wiki Comparison
Wiki comparison / Wikimedia Movement Metrics
Unique devices (all Wikipedias)
1.65B
2.06B
1.25X
Editors
Total active editors in Wiki Comparison is 1.26x of active editors in Wikimedia Movement
Metrics. This is due to the metric definitions and how we count global editors:
Wikimedia Movement Metrics: the number of active editors is the number of registered users
who made at least 5 content edits across all projects
Wiki Comparison: the number of active editors is the number of registered users who made
at least 5 content edits in that project.
When adding the active editors of each project together into the total in Wiki Comparison,
global editors can be counted multiple times if they contributed to multiple wikis. In
Wikimedia Movement Metrics, global editors are counted only once no matter how many wikis
they contributed to. This results in 26% more active editors in Wiki Comparison if we sum
all wikis together.
This difference also applies to new active editors.
2020 monthly avg
Wikimedia Movement Metrics
Wiki Comparison
Wiki comparison / Wikimedia Movement Metrics
Active editors
92378
116202
1.26X
New active editors
19375
20821
1.07X
[1] Wikimedia Movement Metrics are published monthly on Commons; e.g. for December 2020:
https://commons.wikimedia.org/w/index.php?title=File:December_2020_Wikimedi…
<https://commons.wikimedia.org/w/index.php?title=File:December_2020_Wikimedia_movement_metrics.pdf>
Regards,
Jennifer & Kate
Product Analytics <https://www.mediawiki.org/wiki/Product_Analytics>
Wikimedia Foundation <https://wikimediafoundation.org/>