Cross-post.
---------- Forwarded message ---------
From: Adam Baso <abaso(a)wikimedia.org>
Date: Mon, Aug 27, 2018 at 4:19 PM
Subject: Wikimedia Foundation Readers Roadmap Review - 27 August 2018
To: <wikitech-ambassadors(a)lists.wikimedia.org>
Hello -
We recorded a roadmap review this morning for the Wikimedia Foundation's
Readers team. This covers material about efforts on the web, native apps,
multimedia, infrastructure, and parsing. Enjoy!
YouTube*
https://www.youtube.com/watch?v=MZeL0rBDJ3s&t=20
PDF
https://commons.wikimedia.org/wiki/File:Wikimedia_Foundation_Readers_Roadma…
As a reminder, you can find our department (Audiences) annual plan on Meta:
https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2018-2019/…
.
Thanks!
Adam Baso
Engineering Director
Wikimedia Foundation - Readers
* I'm converting the video for WebM VP9 on Commons as well.
Hello,
If you don't use ORES API, please ignore this email.
If you are using wp10 models in your tool, gadget, or research, please note
that these models are now renamed to "articlequality" to better reflect
what they are (in comparison to "editquality"). articlequality models are
deployed on English, Russian, French, Persian, Turkish, and Basque
Wikipedia languages.
So URLs like this:
https://ores.wikimedia.org/v3/scores/enwiki/855137823/wp10
Need to be changed to something like this:
https://ores.wikimedia.org/v3/scores/enwiki/855137823/articlequality
Same goes with parsing the results.
The "wp10" still exists as an alias and if you don't determine models
(meaning you want scores for all models) we respond with wp10 and
articlequality data duplicated [1] but we might pull the plug after four
weeks after this announcement.
[1]: For example see: https://ores.wikimedia.org/v3/scores/enwiki/855137823
For more information see: https://phabricator.wikimedia.org/T196240
Best
--
Amir Sarabadani
Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
In an attempt to identify abandoned VPS projects, I've created a wiki
page that lists all existing projects, here:
https://wikitech.wikimedia.org/wiki/Cloud_VPS_2018_Purge
Currently 85 projects[2] on that list are unclaimed. If you are a VPS
user, please visit that page and mark any projects that you use as
{{Used}}. Note that it's not necessary for you to be a project admin to
mark something -- if you know that you're currently using a resource and
want to keep using it, go ahead and mark it accordingly. If you /are/ a
project admin, please take a moment to mark which VMs are or aren't used
in your projects.
When October arrives, I will shut down and begin the process of
reclaiming unused projects.
If you think you use a VPS project but aren't sure which, I encourage
you to poke around on https://tools.wmflabs.org/openstack-browser/ to
see what looks familiar. You can also log in to
http://horizon.wikimedia.org which will provide you with a handy menu of
projects that you are currently a member of.
Thank you!
-Andrew and the WMCS team
[1]
https://phabricator.wikimedia.org/phame/post/view/112/neutron_is_finally_co…
[2] Here, for good measure, is that list. Every one of these projects
is currently a candidate for deletion:
aicaptcha
analytics
bots
chicotestproject
ci-staging
codereview
collection-alt-renderer
commonsarchive
community-labs-monitoring
contributors
dashiki
deep-learning-services
discourse
discourse-wam
download
etytree
fastcci
getstarted
glampipe
globaleducation
hat-imagescalers
hound
iiab
iiifls1
ircd
jupyter
kubernetes-testing
maps
maps-team
math
matrix
mcr-dev
mediahandler-tests
mediawiki-docker
multimedia
mw-api-testing
mw-extension-ids
mwfuzz
mwoffliner
newsletter
nonfreewiki
openocr
ores
orig
otrs
pagemigration
paws
phlogiston
piwik
project-smtp
rcm
rdfiodev
reading-lists
reading-web-staging
recommendation-api
redirects
reportcard
sciencesource
sentry
services
services-testbed
t136871
test-twemproxy
thumbor
tor
traffic
twl
utrs
video
wcdo
wdq-mm
wikibase-nearest-neighbors
wikibrain
wikidata-federation
wikidata-page-banner
wikidata-primary-sources-tool
wikidumpparse
wikifactmine
wikimetrics
wikisource-tools
wikistream
wikitolearn-dev
wildcat
wlmjudging
wmf-research-tools
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/447099 changes the
signature of ContentHandler::getUndoContent from <Revision, Revision,
Revision> to <Revision|Content, Revision|Content, Revision|Content[, bool]>
(with Revision soft-deprecated). This is a breaking change as
ContentHandler is meant to be subclassed, and any subclass that typehints
Revision will raise a warning at compile-time, and a fatal when called with
Content.
Searching Gerrit, Github and Google only yielded a single instance of a
subclass overriding the method (which is being fixed) and providing
backwards compatibility in a robust way does not seem feasible, so we are
changing the method signature without a deprecation period (in the sense of
subclass compatibility; with respect to calling the method with Revision,
we follow the normal deprecation process).
For more information on the MediaWiki deprecation policy see
https://www.mediawiki.org/wiki/Deprecation_policy#Removal_without_deprecati…
Hi all!
The state of the Wikimedia error logs makes determining the health of a
deployment difficult. This week there were a number of log messages that
made determining the health of this week's train difficult.
This email is a request for help for a couple of troubling messages
currently showing up on our error logs; please help to investigate these
log messages if you are able:
* Exception thrown for failure to save settings appears ~ 1000
times/day[0]
* "Falling back to DifferenceEngineSlotDiffRenderer" Logspam[1]
Thank you for your help!
<3
-- Tyler
[0] <https://phabricator.wikimedia.org/T202149>
[1] <https://phabricator.wikimedia.org/T202686>
Hi All,
A reminder that TechCom is hosting a RFC discussion via IRC on
Wednesday. Introduce a new namespace for collaborative judgments about
wiki entities <https://phabricator.wikimedia.org/T200297>
This meeting is to discuss the use of JADE to record the judgments and
discuss concerns related to this approach.
The meeting is scheduled for 22 August at 2pm PST(21:00 UTC, 23:00 CET)
in #wikimedia-office
If you haven't joined a #wikimediaoffice meeting before more information
can be found here:
<https://meta.wikimedia.org/wiki/IRC_office_hours#How_to_participate>
More information regarding the TechCom RFC process is available here:
https://www.mediawiki.org/wiki/Wikimedia_Technical_Committee/Processes#RFC_…
Thanks,
Kate
--
Kate Chapman TechCom Facilitator (Contractor)
https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-08-22
= *2018-08-22* =
== Callouts ==
* SRE:
Switchover/Switchback dates finalized
Services: Tuesday, September 11th 2018 14:30 UTC
Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC
Traffic: Tuesday, September 11th 2018 19:00 UTC
MediaWiki: Wednesday, September 12th 2018: 14:00 UTC
Switchback:
Traffic: Wednesday, October 10th 2018 09:00 UTC
MediaWiki: Wednesday, October 10th 2018: 14:00 UTC
Services: Thursday, October 11th 2018 14:30 UTC
Media storage/Swift: Thursday, October 11th 2018 15:00 UTC
IMPORTANT: For the duration of the switchover (1 month), please refrain
from large db schema changes and please avoid deploying any kind of new
feature that requires creation of tables.
IMPORTANT: There will be a train freeze in the week of Sept 10th and Oct
8th.
* Release Engineering blocked by: Feedback needed (*how problems could have
been prevented*) from many people/teams on a recent MediaWiki train related
incident report.
** 1.32.0-wmf.13, 9 blockers, feedback needed for 8 of them:
https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train
** Aaron Schulz (Performance), Adam Wight (Scoring Platform), Bartosz
Dziewoński (Contributors), Brad Jorsch (MediaWiki Platform), C. Scott
Ananian (Contributors), Daniel Kinzler (Wikimedia Deutschland), Timo Tijhof
(Performance), Prateek Saxena (Audiences Design)
* Release Engineering looking for more participants to join the Code Health
Metrics working group. This group's purpose is to define and later
implement a set of core metrics that we will use to asses the health of our
code base. More info:
https://www.mediawiki.org/wiki/Code_Health_Group/projects/Code_Health_Metri…
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by: -
* Blocking: -
* Updates:
**6.0.0 released (
https://phabricator.wikimedia.org/tag/ios-app-v6.0-walrus-on-a-unicycle/ )
** 6.0.1 to be released next week (
https://phabricator.wikimedia.org/tag/ios-app-v6.0.1-walrus-on-a-golf-cart/
)
** 6.1 up next, release date TBD (
https://phabricator.wikimedia.org/tag/ios-app-v6.1-narwhal-on-a-bumper-car/
)
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
** Finishing up polling Echo notifications (limited to certain types of
notifications initially)
==== Readers Web ====
* Blocked by:
* Blocking:
* Updates:
** Mobile website (MinervaNeue / MobileFrontend):
*** Page issues UI and instrumentation: T191528 T201124 T197932 T196150
*** Invest in the MobileFrontend & MinervaNeue frontend architecture:
T188937
*** Avoid abrupt and jarring white flash in media viewer: T197110
*** Other fixes and hygiene: T197499 T199000 T197497 T202021 T201956
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** PCS:
*** Separating pagelib CSS from base CSS this week
*** Adding simple header to mobile-html (title, description, page image)
** Maps:
*** Nothing to report.
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Nothing to report
==== Multimedia ====
* Blocked by:
* Blocking:
* Updates
** SDoC: Discussion with search team about searching for related statements
** SDoC: prototyping depicts search
** SDoC: design artwork
** No longer blocked by MCR
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
** No update.
==== Anti-Harassment Tools ====
* Blocked by:
* Blocking:
* Updates:
** Continuing work on Partial Blocks:
https://phabricator.wikimedia.org/T2674
** Updated the TechComm RFC to include Multiblocks:
https://phabricator.wikimedia.org/T199917
==== Editing ====
* Blocked by:
* Blocking:
** Updates:
** Improved/bug fixes: collapsed tables in visual diffs; insertion of
horizontal rules; pages with lots of tables in WTE 2017; switching between
visual editor and source mode; VE linking in IE11; regressions with icon
displays; etc.
** Compact multiple transactions into one:
==== Growth ====
* Blocked by:
** SRE: Query about plausibility of measuring e-mail traffic to our
no-reply addresses https://phabricator.wikimedia.org/T202329
* Blocking:
* Updates:
** No update.
==== Language ====
* Blocked by:
* Blocking:
* Updates:
** No update.
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
** No update.
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
** No update.
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** Changes to hash and salt (every 90 days) appinstallIDs are on, these
will be retained (hashed) forever
**Working with performance in making eventlogging javascript module lighter
so it can be loaded on 1st page load.
**Working on computing “top contributors” metric per project so it is
available in wikistats2, this is one of the most coveted reports by our
community.
**Spark available now in Jupyter notebooks.
**Meetings around modern event data platform to define what parts will we
start developing next quarter.
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
** No update.
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
** Put off upgrade of mediawiki on payments servers
** testing payment notification listener on debian stretch and php 7.0
** Bugfixes and UI tweaks for new card processor API integration
** Nearly done new system for ingesting EventLogging data about banners and
FR landing pages
*** Activated EventLogging for all banner campaigns, sampled at 1% on the
front end.
** Working on email Opt-In page
** CiviCRM:
*** reviewing final group of patches for donor data export and deletion
*** bugfixes on imports
*** Looking at performance of donation imports
=== Core Platform ===
* Blocked by:
* Blocking:
* Updates:
** LoadBalancer overload investigation (T180918)
** TitleValue performance (T201801)
** Awaiting review: Security review for Guzzle 6.3.3 (T202143)
** In testing: Create GuzzleHttpRequest class as new default for
HttpRequestFactory (T202110)
** Ready for testing: Convert MultiHttpClient to use Guzzle (T202352)
** Platform Evolution: developing sessions for TechConf
** REST mobile endpoints removed from projects other then wikipedia and
wikivoyage
** New event type coming to EventBus on this train:
mediawiki.revision-tags-change
=== Performance ===
* Blocked by:
* Blocking:
* Updates:
** Deferred jQuery loading is rolling out to all wikis on this week's train
** All wikis using mcrouter
** nutcracker writes have been disabled
** Mcrouter broadcast operations enabled for all wikis
** patched a bug in Html::inlineScript
** Removed use of StartProfiler.php from mw-init
=== Release Engineering ===
* Blocked by:
** Feedback needed (*how problems could have been prevented*) from many
people/teams on a recent MediaWiki train related incident report.
*** 1.32.0-wmf.13, 9 blockers, feedback needed for 8 of them:
https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train
*** Aaron Schulz (Performance), Adam Wight (Scoring Platform), Bartosz
Dziewoński (Contributors), Brad Jorsch (MediaWiki Platform), C. Scott
Ananian (Contributors), Daniel Kinzler (Wikimedia Deutschland), Timo Tijhof
(Performance), Prateek Saxena (Audiences Design)
* Blocking:
** MediaWiki 1.29 final release and EOL; was due in June:
https://phabricator.wikimedia.org/T197669 (w/ Security)
* Updates
** New general purpose CI job that builds and runs test containers via
Blubber/Docker based on config provided in each project (think
`.travis.yml` file)
*** Read more about Blubber here:
https://wikitech.wikimedia.org/wiki/Blubber
*** See recent builds at
https://integration.wikimedia.org/ci/blue/organizations/jenkins/blubber-tes…
*** Gives developers one major benefit of the CD pipeline work now, having
control over their pre-merge and gating tests without having to mess with
integration/config
*** Only scheduled to run for a few repos at the moment, but will
eventually be expanded to many more projects (we need to tune CI infra
around it first)
** Looking for more participants to join the Code Health Metrics working
group. This group's purpose is to define and later implement a set of core
metrics that we will use to asses the health of our code base. More info:
https://www.mediawiki.org/wiki/Code_Health_Group/projects/Code_Health_Metri…
=== Research ===
* Blocked by: None
* Blocking: None
* Updates:
** Deployed a new UnsourcedStatements campaign (en, fr, it):
https://labels.wmflabs.org
** Finished the first phase of the "Elicit new editor interests" research.
Starting the second phase soon.
** Looking into using Cassandra as a storage engine of Recommendation JS
API.
=== Scoring Platform ===
* Blocked by: SRE, RelEng
* Blocking: None
* Updates:
** All SP repos have been moved to wikimedia organization in github.
** Git LFS support for ores models is going live, some help from RelEng
would be great: https://phabricator.wikimedia.org/T192042
** JADE scalability discussion continues, TechCom meeting is scheduled for
today.
** PoolCounter support to limit number of external connections to ORES has
started. SRE input would be appericiated:
https://phabricator.wikimedia.org/T201824
** Wikilabels robustness is being worked on, some puppet changes will need
review. The node is being reimaged to stretch
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
** Published blog post about search tokenization:
https://wikimediafoundation.org/2018/08/07/anatomy-search-token-affection/
** Added support for haswbstatement without value (any statement with given
property) https://phabricator.wikimedia.org/T199886
** Re-indexed wikis in Malay, Indonesian, and Polish, enabling Malay
stemming and fixing errors in Polish stemming (
https://phabricator.wikimedia.org/T196780,
https://phabricator.wikimedia.org/T186046)
** Created ElasticSearch plugin for Esperanto analysis:
https://phabricator.wikimedia.org/T200099
** Migrated ElasticSearch cluster to RAID0 and upgraded to Debian stretch:
https://phabricator.wikimedia.org/T198391
** Working on migrating communications from analytics to search cluster to
Kafka: https://phabricator.wikimedia.org/T198490
** Working on running multiple Elastic instances on the same hardware:
https://phabricator.wikimedia.org/T193654
** Working on ES 6.3 upgrade: https://phabricator.wikimedia.org/T197960
** Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
=== Security ===
* Blocked by:
* Blocking:
** MediaWiki 1.29 final release and EOL; was due in June:
https://phabricator.wikimedia.org/T197669 (w/ RelEng)
* Updates:
**
=== Site Reliability Engineering ===
* Blocked by:
* Blocking:
* Updates:
** No update.
== Wikidata ==
* Blocked by:
** none
* Blocking:
** none
* Updates:
** Work in Lexicographical data continues:
*** adding senses
*** improving the editor workflow
*** https://phabricator.wikimedia.org/project/view/2292/
** Experiment about not using secondary SQL table for formatting item links
being gradually deployed starting Thu Aug, 23rd
*** https://phabricator.wikimedia.org/T201831
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
** No update.
== Multi-Content Revisions ==
* Blocked by:
* Blocking:
* Updates:
** MCR schema migrations are in progress
*** Finished running populateContentTables.php except for a few small wikis
*** Filed T202032: duplicate ar_rev_id values causing
populateContentTables.php to fail
** Phase 2 (file caption support) development tasks nearing completion
** Phase 3 (statement support) development tasks commencing
== SoS Meeting Bookkeeping ==
* Updates:
**
Hey,
ORES repositories used to exist in wiki-ai organization in github.com. For
example you could reach them at github.com/wiki-ai/ores or
github.com/wiki-ai/wikilabels
They all are now moved to wikimedia organization. So they exist in
github.com/wikimedia/ores or github.com/wikimedia/wikilabels. You can get
list of all of scoring platform team repos in [1]
This is needed for better organizational hierarchy and maintenance and also
better support for git LFS [2]. The old URLs redirect to the new ones so
you should not face any issues in developments but at one point in really
far future these might stop working, so please make sure you update the
urls. Also, if you encounter any issues, let me know.
Sorry for any inconvenience and thank you for your contribution.
[1]: https://github.com/orgs/wikimedia/teams/scoring-platform/repositories
[2]: https://git-lfs.github.com/
Best
--
Amir Sarabadani
Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland – Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Hello everyone,
I'm pleased to announce that we've released OOUI v0.28.0 this Tuesday.
Key highlights of this release:
- New OOUI “Tutorials” section[0]
Special thanks to GSOC student Hagar Shilo, who has provided us an excellent
addition to our documentation.
The tutorials simplify stepping into development with the library.
- Final code cleanup step after major icon refinement[1], see below.
Possible breaking changes:
The following changes might cause breaking of your interface, please carefully
consider if they affect your code.
- Drop 'find' icon, deprecated in v0.26.2
- Drop 'settings' icon, deprecated in v0.27.0
- Drop cite icons, renamed and deprecated in v0.27.0
- Remove 'clip' & 'unClip', deprecated in v0.26.1
- Remove 'comment', deprecated in v0.26.1
- Remove deprecated 'userActive'/'userInactive', deprecated in v0.26.2
All six icon changes above have been deprecated and announced in former
versions. With this release we've removed them completely.
You can find details on additional new features, code-level and accessibility
changes, styling and interaction design amendments, and all
improvements since v0.27.0 in the full changelog[2].
I'd also like to give a shoutout to Alangi Derick and Tim Eulitz for
their contributions.
If you have any further queries or need help dealing with breaking
changes, please
let me know.
As always, library documentation is available on mediawiki.org[3],
there is comprehensive generated code-level documentation and
interactive demos and tutorials hosted on doc.wikimedia.org[4].
OOUI version: 0.28.0
MediaWiki version: 1.32.0-wmf.18
Date of deployment to production: Regular train, starting Thursday 16 August
[0] - https://doc.wikimedia.org/oojs-ui/master/demos/tutorials/index.html
[1] - https://phabricator.wikimedia.org/T177432
[2] - https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md
[3] - https://www.mediawiki.org/wiki/OOUI
[4] - https://doc.wikimedia.org/oojs-ui/master/
Best,
Volker
--
Senior UX Engineer
Wikimedia Foundation
volker.e(a)wikimedia.org | @Volker_E