https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-12-07
= 2016-12-07 =
== Product ==
=== Reading ===
==== iOS native app ====
* Last Week
** Shipped 5.3.1 with bug fixes
** Continued work on 5.3.2 - Dynamic text size, data layer update,
performance enhancements
* This week
** Finish work on 5.3.2, fix bugs from beta feedback
** Feature complete on 5.3.2 Friday 12/9
==== Android native app ====
* Last week:
** Continuing Q2 goals for Wikidata descriptions
** New beta v2.4.182
*** Fundraising and survey cards now configurable remotely (beta)
* Next week (https://phabricator.wikimedia.org/project/view/2352/):
** More Q2 goals for Wikidata descriptions (tutorial and polish)
==== Mobile Content Service (MCS) ====
* Remove obsolete mobile-summary endpoint (handled by RESTBase directly)
* Moved to node template version 0.5.0, moved to ES2015 and eslint
==== Reading Web ====
* Current sprint: https://phabricator.wikimedia.org/project/view/2362/
* So far:
** ReadMore has been enabled on Japanese and Spanish wikis (mobile site)
** Wikidata Page banner has been enabled on Finnish Wikivoyage
** Wikidata page descriptions has been enabled on French and German
wikipedias (mobile site)
** Lead image has been disabled from the mobile site, beta mode.
* Until we meet the next time:
** Create importable test pages that can be used to test various Reading
Web related changes
** Hovercards rewrite finish up
** Continue working on returning page images from the lead section only
==== Reading Infrastructure ====
* API i18n changes going out with next train:
https://lists.wikimedia.org/pipermail/mediawiki-api-announce/2016-December/…
** you can use the new errorformat, errorlang, errorsuselocal paramers to
get localized, parsed errors; if you don't, most API modules will behave
the same as before. Exceptions:
*** for query modules the error codes won't be prefixed anymore the way
parameter names are;
*** for a few modules (see announcement) which put errors in some
non-default location in the response data, that location might change
and/or the contents might change from a single error to an array of errors.
* could use some advice from Performance or Language on dealing with huge
JS data blobs: *https://phabricator.wikimedia.org/T32574#2836570*
<https://phabricator.wikimedia.org/T32574#2836570>
* MediaViewer tests failing, mw.language tests seem to be the cause:
https://phabricator.wikimedia.org/T152476
=== Editing ===
==== Collaboration ====
* Working on new filter UI for Special:Recentchanges; a bunch of backend
patches for this landed in MW core recently, more will land soon. Will be
behind a hidden preference for the time being.
* Converted Flow and Echo to extension.json. This has caused a few minor
issues for Flow and may cause some for Echo as well. Also, if you have open
patches, they'll probably conflict now.
* Going to convert PageTriage next; and FlaggedRevs after we clear out the
CR backlog a bit there
==== Parsing ====
* Blocked: Tim asked ops for someone to assist him with puppetizing
HTML5Depurate for a pilot deployment to the production cluster for our Q2
goal. No response yet.
** See "[Ops] Need Html5Depurate playmate" mail from Tim dated 11/24
* Ongoing investigation of https://phabricator.wikimedia.org/T151702 and
followups with Ops and Services
* Parsoid deploy today after almost a month (
https://www.mediawiki.org/wiki/Parsoid/Deploymentswill be updated later
today)
** Parsoid-native <gallery> rendering + a bunch of other changes going out
today (Most interesting for MCS: heading ids now assigned that match core
output).
** HTML version bump to 1.3.0; See
https://www.mediawiki.org/wiki/Specs/HTML/1.3.0#Changes_since_Specs.2FHTML.…
for
changes since 1.2.1
* Tim is close to finishing the first version of remex-html (
https://github.com/tstarling/remex-html ) -- a HTML5 parser in PHP
=== Discovery ===
* No blockers
* Started discussion on how to combine various models of user query
enhancements/transformations to improve search results:
https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/So_Many_Search_Optio…
* Continuing work on crosswiki searches, load test successful
* Published analysis of WDQS traffic to detect parallel connections:
https://phabricator.wikimedia.org/T149963
* WDQS now limits parallel connections to 5 per IP. Working fine so far.
== Technology ==
=== Analytics ===
* Blocker -- None
* Ongoing
** Wikistats 2.0 UI has kicked off with new contractor designer.
** RecentChanges via EventBus is on its way, we will soon have events
flowing in Kafka. Purpose is later to turn off Redis based producer.
** Mediawiki history reconstruction is at final review stage (will be
long, heavy code) -- Plans to communicate early next quarter
** Old logs on stat1002 have been *deleted*. More at
https://phabricator.wikimedia.org/T84618. Now working on removing some old
generation processes in the cluster.
** Cross team collaboration in migrating limn editor-engagement
dashboards to dashiki
=== Release Engineering ===
* '''Blocking''':
** None?
* '''Blocked'''
** None
* '''Updates'''
** Reminder that deployments are weird between now and the New Year:
https://lists.wikimedia.org/pipermail/wikitech-l/2016-November/086936.html
** Gerrit is updating, pretty much right now
=== Security ===
* Security Reviews
** Recommendations API continues from last week
* Continuing work on CSRF protection for anonymous users (
https://phabricator.wikimedia.org/T40417)
* Question for Parsing: Can we limit the amount of nesting in lists?
** https://phabricator.wikimedia.org/T151834
* Question for Reading: Should page requests containing invalid characters
in title still result in 200 OK if curid is valid?
** https://phabricator.wikimedia.org/T144100
=== Services ===
* Blockers: none
* Updates:
** PDF render service and API endpoint deployed and completely
operational:
***
https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_pdf_title
** Working on followups for the API cluster outage
*** https://phabricator.wikimedia.org/T152074
=== Technical Operations ===
* '''Blocked''':
** Community Tech
*** https://phabricator.wikimedia.org/T152080 page_assessments creating
duplicate key errors
* '''Blocking''':
** None
* Updates:
** Still working on API cluster failure.
https://phabricator.wikimedia.org/T151702
** new labsdbs are almost ready to start being used
** AB testing design still under discussion
https://phabricator.wikimedia.org/T143694
** Goals for next quarter being drafted, if you have any dependencies on
Ops contact us
== Wikidata ==
* I still can not join the Hangout when not being a WMF employee. I know
multiple people already looked into this. Please forgive me when I'm not
trying any more. I still find the information given in this Etherpad very
helpful, and love sharing relevant bits via a WMDE mailing list.
** Grace to look into this ^
* Blocked on security reviews of:
** Cognate extension: https://phabricator.wikimedia.org/T149082
** InterwikiSorting Extension: https://phabricator.wikimedia.org/T149083
== Fundraising Tech ==
* Investigating banner impression drop from 08:00-09:00 UTC on 12/1 and
12/2: https://phabricator.wikimedia.org/T152122
* Fixing CiviCRM reports & internal dashboard
* Tweaking mailing list export
* Planning for next year
** Estimating effort for next year's required payment processor integrations
** Potential extra-curricular work supporting ORES:
https://www.mediawiki.org/wiki/User:Adamw/Study_abroad_proposal
Google Code-in (GCI) has been running for 8 days and students have
already resolved more than 50 tasks!
Some highlights:
* Several extension pages on mediawiki.org display screenshots
(AbuseFilter, InputBox, Cite, MobileFrontend, …)
* Kiwix ZIM files can now be opened from the Android file manager
* Pywikibot received additional tests
* The celery logging of the ORES service does not include
TimeoutErrors
* WikiEduDashboard does not allow to create programs with url-encoded
spaces in the title
* Wikidata has a special page to list all available badges
* MediaWiki's Newsletter extension received numerous code fixes
* All deprecated Linker::link() usage got replaced in MediaWiki core
* html-metadata received support for Twitter cards metadata
* Numerous MediaWiki extensions specify extension dependencies in
their maintenance scripts
* Translatable messages in several projects show $1 as an insertable
* Huggle got support for Ukrainian and Russian language added
* ConfirmEdit's system messages aren't CAPTCHA module specific anymore
* action=checktoken in MediaWiki's API warns if the token ends in " \"
* The MobileFrontend hooks documentation got updated
* The PageImages API offers a "Help" link
* VisualEditor's user guide has a new section about musical scores
* …and many more.
Congratulations to all our students and thanks to all our mentors!
You also have an idea for a task you'd like to mentor? Read
https://www.mediawiki.org/wiki/Google_Code-in_2016
Please contact us if you need help or have questions!
Thanks,
andre
--
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/
Hello everyone,
Last Thursday, a change went live affecting the page images API, which
cased images that previously were appearing in many Wikimedia projects
including the Wikipedia apps, the Wikimedia portal and mobile web search
results to disappear for about 3 hours. The full incident report can be
found here
<https://wikitech.wikimedia.org/wiki/Incident_documentation/20161202-2016120…>,
the original bug report, here <https://phabricator.wikimedia.org/T152155>.
This issue was temporarily fixed by allowing projects to display both free
and non-free images by default for the short term. This will be reverted
once a maintenance script creating the new list of images completes
running, which should take a couple of weeks. After this, the default will
change back to displaying free images only. We will send an update again
once this is deployed.
Thanks and sorry for the inconvenience and chaos!
- Olga
--
Olga Vasileva // Product Manager // Reading Web Team
https://wikimediafoundation.org/
Google Code-in 2016 just started:
https://www.mediawiki.org/wiki/Google_Code-in_2016
In the next seven weeks, many young people are going to make their
first contributions to Wikimedia. Expect many questions on IRC and
mailing lists by onboarding newcomers who have never used IRC or lists
before. Your help and patience is welcome to provide a helping hand!
Thanks to all mentors who have already registered & provided tasks!
You have not become a mentor yet? Please do consider it.
It is fun and we do need more tasks! :)
* Think of easy tasks in your area that you could mentor.
Areas are: Code, docs/training, outreach/research, quality
assurance, and user interface. "Easy" means 2-3h to complete for
you, or less technical ~30min "beginner tasks" for onboarding).
* OR: provide an easy 'clonable' task (a task that is generic and
could be repeated many times by different students).
* Note that you commit to answer to students' questions and to
evaluate their work within 36 hours (but the better your task
description the less questions. No worries, we're here to help!)
For the full info, please check out
https://www.mediawiki.org/wiki/Google_Code-in_2016/Mentors
and ask if something is unclear!
Thank you again for giving young contributors the opportunity to learn
about and work on all aspects of Free & Open Source Software projects!
Cheers,
andre
--
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/
It looks like the page was deleted/restored thus giving it a new page ID.
Originally when pages where deleted the page_id was not kept, which caused
a new page_id to be issued when it was restored. This phenomenon has since
been fixed, and should no longer happen.
On Sat, Dec 3, 2016 at 8:47 AM, Renato Stoffalette Joao <joao(a)l3s.de> wrote:
> Hi all.
>
> Firstly, apologies for eventual duplicates or posting the question in the
> wrong mailing list.
>
> Secondly, could anybody kindly explain to me if some Wikipedia pages
> changed their IDs from the past ? Or if so point to me where this might be
> documented ?
> I have Wikipedia pages-articles XML dumps from the years 2006 and 2008
> and when I was parsing those dumps I ran across some situations
> such as the following one. In the dumps from 2006 and 2008 I found that
> the South Africa page has the ID 68854, while in the most current Wikipedia
> pages-articles XML dump (i.e. 2016) the same article has the ID 17416221.
> I am trying to match some Wiki pages by IDs across time, but the example
> above is not helping.
>
> Much appreciated in advance for any help.
>
> --
> Renato Stoffalette Joao
> - PhD Student -
> L3S Research Center / Leibniz Uni.
> 15th Floor, Room:1519
> Appelstraße 9a
> 30167 Hannover, Germany
> +49.511.762-17759
>
>
> _______________________________________________
> Xmldatadumps-l mailing list
> Xmldatadumps-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>
Hello,
The Wikimedia Developer Summit 2016 videos have been published to YouTube
(and Commons!). [1] [2]
These videos cover the event earlier this year that saw MediaWiki and
Wikimedia contributors come to gather to discuss the future of our shared
technologies. Please enjoy and share.
[1] https://www.youtube.com/playlist?list=PLeoTcBlDanyNOFQmVMVwpKUXeuw_-896K
[2] https://commons.wikimedia.org/wiki/Category:Wikimedia_De
veloper_Summit_2016
A big thanks to the people who helped get these videos online to share.
Particularly Husn Shujaat and Andre Klapper.
Interested in more? Join us at the 2017 Wikimedia Developer Summit in
January!
https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit
--
Yours,
Chris Koerner
Community Liaison - Discovery
Wikimedia Foundation
Original version: https://www.mediawiki.org/wiki/Topic:Td5wfd70vptn8eu4
The Wikimedia Developer Summit 2017 is 13 weeks away!
PARTICIPANTS
* 68 people have requested an invitation.
* 19 of them have requested travel sponsorship as well.
Join us: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit.
The deadline to request travel sponsorship is Monday, October 24th.
PROPOSALS
* 8 proposals submitted - https://phabricator.wikimedia.
org/project/view/2205/
** 1 in backlog
** 2 in Unconference
** 2 missing basic information
** 3 to be pre-scheduled
https://www.mediawiki.org/wiki/Wikimedia_Developer_
Summit/Call_for_participation
The deadline for submitting new proposals is Monday, October 31.
MAIN TOPICS
https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Program
On track:
* A plan for the Community Wishlist 2016 top results
* Building on Wikimedia services: APIs and Developer Resources
* How to manage our technical debt
Missing basic information
* A unified vision for editorial collaboration
* Building a sustainable user experience together
* How to grow our technical community
Missing two facilitators
* A unified vision for editorial collaboration
* How to grow our technical community
Missing one facilitator
* Handling wiki content beyond plaintext
* Building a sustainable user experience together
* Artificial Intelligence to build and navigate content
ORGANIZATION
* Meet the Program committee. https://www.mediawiki.org/
wiki/Wikimedia_Developer_Summit/Program_committee
--
Quim Gil
Engineering Community Manager @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil