Hi Goran, We'll keep an eye toward consistency, but we have not made the data extraction into a fully automated process. We identified 3 columns that had slightly different names and we'll fix them: overall SIZE rank (2020) vs. overall size rank (2018, 2019) second month editor retention (2020) vs. second-month new editor retention (2018, 2019) monthly structured discussions messages (2020) vs. monthly structured discussions (Flow) messages (2018, 2019) The "project code" column was duplicated in 2020; the duplicate has now been removed. Finally, in 2019 we had added 3 new columns that we hadn't tracked in 2018: content pages, cumulative content edits, edits per content page. Please be aware that we may add or change columns in the future as needs evolve. Warm regards, Kate On Tue, Feb 23, 2021 at 12:37 PM Goran Milovanovic < goran.milovanovic_ext(a)wikimedia.de&gt; wrote:

Hi Kate, and thank you very much for your feedback. I think I've forgotten to mention how the Wiki comparison 2020 dataset is so great that I will start using it in my R programming language classes as of today to help people learn more about hypothesis testing and join operations across the dataframes : ) Thank you for all the hard work! extraction into a fully automated process. I have seen the code, I know the pain too well... All the work and then in the end there is always an additional detail that was maybe not considered in the beginning, similar things made me cry in the past in my work on Wikidata... I sympathise with you and the team and I wish you all the best in your future work! And by the way... The differences in column names are really not such a big deal, the variable semantics are so obvious so they match easily. Good work! With best wishes, Goran Goran S. Milovanović, PhD Data Scientist, Software Department Wikimedia Deutschland ------------------------------------------------ "It's not the size of the dog in the fight, it's the size of the fight in the dog." - Mark Twain ------------------------------------------------ On Tue, Feb 23, 2021 at 11:38 PM Kate Zimmerman &lt;kzimmerman(a)wikimedia.org&gt; wrote: > Hi Goran, > > extraction into a fully automated process. > > We identified 3 columns that had slightly different names and we'll fix > them: > overall SIZE rank (2020) vs. overall size rank (2018, 2019) > second month editor retention (2020) vs. second-month new editor retention > (2018, 2019) > monthly structured discussions messages (2020) vs. monthly structured > discussions (Flow) messages (2018, 2019) > > The "project code" column was duplicated in 2020; the duplicate has now > been removed. > > Finally, in 2019 we had added 3 new columns that we hadn't tracked in > 2018: content pages, cumulative content edits, edits per content page. > Please be aware that we may add or change columns in the future as needs > evolve. > > Warm regards, > Kate > > On Tue, Feb 23, 2021 at 12:37 PM Goran Milovanovic < > goran.milovanovic_ext(a)wikimedia.de&gt; wrote: > >> Well, it would be desirable to maintain consistent column names across >> the years... >> >> Best, >> Goran >> >> Goran S. Milovanović, PhD >> Data Scientist, Software Department >> Wikimedia Deutschland >> >> ------------------------------------------------ >> "It's not the size of the dog in the fight, >> it's the size of the fight in the dog." >> - Mark Twain >> ------------------------------------------------ >> >> >> On Tue, Feb 23, 2021 at 2:42 AM Jennifer Wang &lt;jwang(a)wikimedia.org&gt; >> wrote: >> >>> Hi all, >>> >>> For your reference we have updated wiki comparison dataset >>> <https://www.mediawiki.org/wiki/Product_Analytics/Comparison_datasets> >>> with 2020 data >>> <https://docs.google.com/spreadsheets/d/1a-UBqsYtJl6gpauJyanx0nyxuPqRvhzJRN817XpkuS8/edit?usp=sharing> >>> . If you have any feedback or suggestions, please let us know via >>> product-analytics(a)wikimedia.org. >>> >>> Regards, >>> Jennifer & Product Analytics >>> _______________________________________________ >>> Analytics mailing list >>> Analytics(a)lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >>

On Wed, 24 Feb 2021 at 20:04, Samat &lt;samat78(a)gmail.com&gt; wrote: Thank you very much for noticing this! It looks like we accidentally mixed up some of the definitions. I've restored the correct definition of overall size rank, which is "a composite ranking of wikis by monthly active editors and monthly unique devices (produced by taking the geometric mean of those two values)". I checked all of the other definitions and they should be correct. Size of the database is a good suggestion, but sadly my team doesn't have time to add any more metrics right now. If you'd like to note down your suggestion for the future, we have been keeping a list at mediawiki.org/wiki/Product_Analytics/Wiki_comparison_suggestions <https://www.mediawiki.org/wiki/Product_Analytics/Wiki_comparison_suggestions> . (We'd also be thrilled to help others expand the dataset directly, but this is likely to be difficult since our code <https://github.com/wikimedia-research/wiki-segmentation/tree/master/data-collection> currently relies on production data access <https://wikitech.wikimedia.org/wiki/Analytics/Data_access>, which is only available to staff of the Wikimedia Foundation or Wikimedia Deutschland or official research collaborators <https://www.mediawiki.org/wiki/Wikimedia_Research/Formal_collaborations>.) -- Neil Shah-Quinn senior data scientist, Product Analytics <https://www.mediawiki.org/wiki/Product_Analytics> Wikimedia Foundation <https://wikimediafoundation.org/>