https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-03-21#Multimedia
= 2018-03-21 =
== Callouts ==
* Fundraising campaigns
https://meta.wikimedia.org/wiki/CentralNotice/Calendar - next up:
Netherlands 2018-04-03 through 2018-05-01
* WMDE needs a Wikidiff2 review: https://gerrit.wikimedia.org/r/404293
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
** Increased rollout of Reading List sync to 50% of Beta audience.
** Blocked on a few minor client-side issues, but should continue full
rollout by end of week.
==== Readers Web ====
* Blocked by:
* Blocking:
* Updates:
*Interrupt work for Hindi Wikipedia campaign (
https://phabricator.wikimedia.org/T190101)
*Discussion around DNT (https://phabricator.wikimedia.org/T187277)
*Improvements and some refactoring to Popups extension
*Quarterly goal dependency update:
** Increase learning by lowering the cost of exploration (Page previews)
*** Turning on virtual page view logging for all wikis (
https://phabricator.wikimedia.org/T189906)
*** Pushing for full-deploy in next 3 weeks. (
https://phabricator.wikimedia.org/T154635)
*** This goal was impacted by interrupt work.
** Continue improving the ways that users can download articles of interest
for later consumption
*** Proton work was delayed until next quarter due to external dependencies
in standing up a production endpoint
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** New /page/media endpoint is exposed via RESTBase, so apps can test it
and provide feedback.
** /page/metadata and /page/references to follow this week.
===== Maps =====
* Blocked by:
* Blocking:
* Updates:
==== Multimedia ====
* Blocked by: N/A
* Blocking: N/A
* Updates
** Search: Patch for MediaInfo caption indexing nearly complete, WMDE and
Cormac working diligently to get things merged, Search supporting (thanks,
both)
** File page prototyping: Progressing, slowly. Mark to meet with Thiemo
from WMDE to clear some things up.
Quarterly goal dependency update
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
** Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and
exposure
*** SDC: Research/Multimedia
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
* GlobalPreferences going live today
* Work on TemplateWizard progressing well
==== Anti-Harassment Tools ====
* Blocked by: None
* Blocking: None
* Updates:
* Work continues on Blocking tools
* Bug fixes to Interaction Timeline
* Research on Harassment Reporting Tool
==== Editing ====
* Blocked by:
* Blocking:
* Updates:
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Tidy has been replaced on all wikiversities today. About 300 more to go
before the end June deadline.
** Whitespace trimming in headings, list items, table cells, headings and
captions (wikitext versions only) has been pushed to production. Found a
bug for which there is a fix in gerrit
** Arlo has been working on the heredocs syntax RFC (
https://phabricator.wikimedia.org/T114432 ) and has a WIP patch @
https://gerrit.wikimedia.org/r/#/c/418198/ if someone wants to test and
play with it on your local dev wikis.
*Quarterly goal dependency update:
** Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
*** Parsing:Mediawiki PF/Services
*** No new updates or blockers.
==== Collaboration ====
* Blocked by:
* Blocking:
* Updates:
** Flow uninstalled from wikis where it was unused; can still be requested
by communities, will be re-enabled with local consensus
** Almost done setting up maps test servers in labs
==== Language ====
* Blocked by: None
* Blocking:None
* Updates:
** CX2 work continue;
** Migration to recommend service from Production is planned; WIP.
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
** OOUI – v0.26.0 breaking change release
https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v…
*** 7 breaking changes, mostly already deprecated icons got removed
*** WikimediaUI theme's new icon set in place
https://phabricator.wikimedia.org/T177432 !
*** Unified VE in Vector/WikimediaUI interface base font-size to usage of
OOUI in core/extensions elsewhere https://phabricator.wikimedia.org/T97631
!
** Style Guide
*** Continued work on v1 goals
https://phabricator.wikimedia.org/tag/wikimediaui_style_guide/
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** 504s in Pageview API RESTBase layer, problem with RESTBase running out
of TCP ports, applying some fixes now to prevent the problem
** notebook1003 is replacing notebook1001, so we need to migrate folks
using Jupyter, watch for communication on the lists
** responsive Wikistats work is ongoing
** porting geowiki to Hadoop is done, building a dashboard on top of it now
** more analytics processes migrated to Kafka Jumbo cluster (webrequest and
EventLogging are fully migrated now)
** improvements on automatic refining of EventLogging topics to Hadoop
* Quarterly goal dependency update:
**Improve, adjust, or create features geared at the needs identified in New
Editors research project.
*** New Editors Experience:Analytics
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
** Upgrading CiviCRM install to 5.0
** More work on main CC processor new API, including redesigning recurring
contribution charges to fit better with Civi
** Making sure all the unintentional recurring donor refunds happened as
they were supposed to
** More work on using EventLogging to get banner impression and fundraiser
landing page stats
** Looking into applying CSP to banner preview pages, but need to decide
how to make it fit with pending core patch:
https://gerrit.wikimedia.org/r/253969
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
** Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
*** Parsing:Mediawiki PF/Services
* Reduce product and technical debt to modernise our tools and
technologies, and to make future changes more effective and efficient
*** Parsing/Mediawiki PF
**1.1 It is possible to store structured data within wiki pages, in
particular on media file pages on Commons. We will enable the MediaWiki
storage layer to correctly store and process structured data elements
within wiki pages.
*** SDC: Mediawiki PF/Wikidata
=== Performance ===
* Blocked by: None
* Blocking: None
* Updates:
** Collecting a bunch of oversample data from countries in Asia/Oceania,
prep for Singapore to go live.
** Thumbor 100% done
** Expecting to spend a bit of time helping with reviews or ResourceLoader
changes related to Hindi wiki front page stuff
** auto_prepend_file being used to load profiler
** Re-did coal changes to use python-kafka, confluent_kafka doesn't work
** Deploying RUMSpeedIndex this week, essentially emulates Speed Index
calculation using the browser resource timing API
=== Release Engineering ===
* Blocking
**
* Blocked
**
* Updates
** Minor Gerrit upgrade planned for this week (2.14.6 -> 2.14.7)
** Incident analysis started last week of the last year’s worth of
incidents reports
** Scap 3.7.7 should be rolled out to production this week
*Quarterly goal dependency update:
** Continue improving the ways that users can download articles of interest
for later consumption
*** Reading Web: Tech Ops/RelEng (work is currently blocked on
https://phabricator.wikimedia.org/T187821 which is part of a larger epic
https://phabricator.wikimedia.org/T181084)
** Talked about in team meeting Monday
** is there a task?
=== Research ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
** Wikimedia Technology/Goals/2017-18 Q3#Segment 2: Search integration and
exposure
*** SDC: Research/Multimedia
=== Scoring Platform ===
* Blocked by:
** Releng is helping us with a git-lfs pilot deployment.
** Waiting for Scap 3.7.7 for a fast-rollback process.
* Blocking:
* Updates:
** The JADE extension is in beta.
*** https://www.mediawiki.org/wiki/Extension:JADE
*** This is for auditing ORES. It adds "Jade" and "Jade_talk" namespaces
to MediaWiki. We're using JSON content with schema validation. Prateek is
contributing some volunteer design time.
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
** Update:
*** Trey published new blog about supporting languages with multiple
writing systems:
https://blog.wikimedia.org/2018/03/12/supporting-languages-multiple-writing…
*** AB test for query explorer feature for LTR finished & analyzed:
https://phabricator.wikimedia.org/T189843 Results are looking pretty good,
seems to be improving things.
*** Wikidata search patches merged. The new search not enabled yet, but
improved search result format is.
*** Fixed performance issue with LTR cache:
https://phabricator.wikimedia.org/T188015
*** TermLookup implementation for ElasticSearch merged:
https://phabricator.wikimedia.org/T143706
*** Fixed deepcategory keyword to not return results if keyword has failed,
e.g. because there are too many categories:
https://phabricator.wikimedia.org/T189331
*** Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
*** Working on Slovak analysis: https://phabricator.wikimedia.org/T178929
*** Started implementation work on Lexeme search:
https://www.wikidata.org/wiki/User:Smalyshev_(WMF)/Lexeme_search
=== Security ===
* Blocked by:
* Blocking:
* Updates:
=== Services ===
* Blocked by: none
* Blocking: none
* Updates:
** New REST endpoint to get lint errors for page/revision
***
https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_lint_title_re…
** New REST endpoint to get page media
***
https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_media_title_r…
** refreshLinks topic is now partitioned in kafka according to MySQL
shard, should decrease connection spikiness
=== Site Reliability Engineering ===
* Blocked by:
** None
* Blocking:
** None
* Updates:
** Firewalling of misc database hosts successful. Some delays in OTRS email
delivery experienced
** Consistent logical backups in codfw goal completed successfully
** mathoid deployed in kubernetes. Production traffic is partly served
(~50%) from the new kubernetes cluster in both eqiad+codfw
** eqsin turn-up planning procedure meetings starting.
** EtcdConfig in production goal almost completed
** Some varnish issues ongoing https://phabricator.wikimedia.org/T189892
*Quarterly goal dependency update:
** Continue improving the ways that users can download articles of interest
for later consumption
*** Reading Web: Tech Ops/RelEng
** Update: No updates, was postponed for early next quarter
** Audiences DesignStandardise our user interfaces to match user
expectation of quality from our products
*** Audiences Design: Ops
** Update: Done AFAIK.
== Wikidata ==
* Looking into current Lua usage to see where we can improve the Lua
functions we provide: https://phabricator.wikimedia.org/T189506
* Optimizing a heavily used database table (wb_terms):
https://phabricator.wikimedia.org/T188279
* Polishing a lot of things for lexicographical data first deployment
*Quarterly goal dependency update:
**1.1 It is possible to store structured data within wiki pages, in
particular on media file pages on Commons. We will enable the MediaWiki
storage layer to correctly store and process structured data elements
within wiki pages.
*** SDC: Mediawiki PF/Wikidata
** Prepare backend infrastructure for structured data search
*** SDC: Search PF/Multimedia/Wikidata
== German Technical Wishlist ==
* Blocked on external review for the most recent Wikidiff2 optimizations:
https://gerrit.wikimedia.org/r/404293
* Working towards an MVP of the FileImporter extension:
https://phabricator.wikimedia.org/tag/move-files-to-commons/
* Finishing touches on AdvancedSearch:
https://phabricator.wikimedia.org/tag/advanced-search/
== SoS Meeting Bookkeeping ==
* Updates:
Hi,
As part of auditing MediaWiki initialisation code[1], I noticed the state
of our OutputHandler logic. It is currently implemented through a series of
global functions in a dedicated includes file (separate from
GlobalFunctions.php) which are then ad-hoc loaded (as needed) in
WebStart.php.
I wouldn't consider these part of the public interface, because they were
only conditionally included, and not in any autoloader. Stable usage would
require a consumer to directly require the file from $IP/includes.
However, given that on many wikis they will appear to be consistently
available, it is not impossible someone somewhere might use it somehow.
I found zero callers apart from the primary call in WebStart.php (plus some
Api comments, and a false-positive match in a unit test).
Not usage in MediaWiki core or elsewhere in Wikimedia Git (per
CodeSearch[2] and GitHub[3]).
As such, per our policy [4] I propose to refactor these without leaving
back-compat aliases.
Feedback welcome at https://gerrit.wikimedia.org/r/420223
-- Krinkle
[1] https://phabricator.wikimedia.org/T189966
[2] https://codesearch.wmflabs.org/search/?q=wfOutputHandler
[3]
https://github.com/search?q=org:wikimedia+wfOutputHandler+-repo:wikimedia/m…
[4] https://www.mediawiki.org/wiki/Deprecation#Removal_without_deprecation
I'm refactoring MonoBook, starting with MonoBookTemplate. The current
change gets rid of the entire immediate print/html soup approach and
instead assembles a giant string and prints that in one statement at the
end. See: https://gerrit.wikimedia.org/r/#/c/420154/ and
https://gerrit.wikimedia.org/r/#/c/420161/
Pending reviews and assurances that I have indeed not totally broken
everything, we'll probably be merging this in the next week or so. If
anyone would like to point out particular reasons why this is a terrible
idea, please do so now.
Future plans include putting that ridiculous getPortlet into core
BaseTemplate, making a less dumb BaseTemplate::getFooter without
breaking anything already using it so MonoBook can lose its silly
replication thereof, organising all the files in MonoBook better
(putting them in resources, includes, etc according to standard skin
practices), and making MonoBook responsive.
There are also some problems that need addressing down the road: that
I'm not sure how safe it is for caching and the like to just go moving
images/css files around willy-nilly, that there are no 'standard' skin
practices as far as anyone can tell, and that some people seem to think
MonoBook is bad and not worth this. But MonoBook is not bad. It is a
delightful skin. We should preserve it, and not just in formaldehyde.
-I
Hi there,
I'm a final year Mathematics student at the University of Bristol, and I'm
studying Wikipedia as a graph for my project.
I'd like to get data regarding the number of outgoing links on each page,
and the number of pages with links to each page. I have already
inquired about this with the Analytics Team mailing list, who gave me a few
suggestions.
One of these was to run the code at this link https://quarry.wmflabs.org/
query/25400
with these instructions:
"You will have to fork it and remove the "LIMIT 10" to get it to run on
all the English Wikipedia articles. It may take too long or produce
too much data, in which case please ask on this list for someone who
can run it for you."
I ran the code as instructed, but the query was killed as it took longer
than 30 minutes to run. I asked if anyone on the mailing list could run it
for me, but no one replied saying they could. The guy who wrote the code
suggested I try this mailing list to see if anyone can help.
I'm a beginner in programming and coding etc., so any and all help you can
give me would be greatly appreciated.
Many thanks,
Nick Bell
University of Bristol
What ways are there to include user-edited JavaScript in a wiki page?
I ask because someone put this revision in (which is now deleted):
https://fa.wikipedia.org/w/index.php?title=%D9%85%D8%AF%DB%8C%D8%A7%D9%88%D…
You can't see it now, but it was someone including a JavaScript
cryptocurrency miner in common.js!
Obviously this is not going to be a common thing, and common.js is
closely watched. (The above edit was reverted in 7 minutes, and the
user banned.)
But what are the ways to get user-edited JavaScript running on a
MediaWiki, outside one's own personal usage? And what permissions are
needed? I ask with threats like this in mind.
- d.
Hi,
Sometimes I find adding assert() calls in my code very handy for various
reasons:
- failures in development mode on some complex code where exposing all the
details to unit tests is sometimes hard and/or pointless
- readability of the code
But I worry about the perf implications of these lines of code. I don't
want these assertions to be used to track errors in production mode.
PHP7 introduced expectations which permit to have zero-cost assert() [1]
Looking at the MW codebase we don't seem to use assert frequently (only 26
files [2] ).
Are there some discussions about this?
Is assert() a good practice for the MW code base?
If yes would it make sense to benefit from zero-cost assertions in WMF
appservers?
Thanks!
[1]
http://php.net/manual/en/function.assert.php#function.assert.expectations
[2]
https://codesearch.wmflabs.org/search/?q=%5Ctassert%5C(&i=nope&files=php%24…
[x-posted announcement]
Hello,
Wikimedia Foundation’s Language team would like to invite you for an online
office hour session scheduled for Wednesday, March 21st, 2018 at 13:00 UTC.
This will be an open session to talk about our work, and in particular the
changes to interlanguage links, which were recently rolled-out on the
English Wikipedia.
The new option shows a list of up to 9 languages instead of a long list
that can have more than 200 items, and a panel with all the links that can
be looked up in any language using a search box. The purpose of this
feature is to make articles in all languages easier to find. We recently
published a blog post about this feature and the thoughts behind the
development:
https://blog.wikimedia.org/2018/03/08/compact-language-links-launch.
This session is going to be an online discussion over Google
Hangouts/Youtube with a simultaneous IRC conversation. Due to the
limitation of Google Hangouts, only a limited number of participation slots
are available. Hence, do please let us know in advance if you would like to
join in the Hangout. The IRC channel will be open for interactions during
the session.
Please read below for the event details, including local time, youtube
session links and do let us know if you have any questions.
Thank you
Runa
== Details ==
# Event: Wikimedia Foundation Language office hour session
# When: March 21st, 2018 (Wednesday) at 13:00 UTC (check local time
http://www.timeanddate.com/worldclock/fixedtime.html?iso=20180321T1300)
# Where: and on IRC #wikimedia-office (Freenode) and
https://www.youtube.com/watch?v=RmZcL6zVcTA
# Agenda:
Discussion about Compact Language Links, and Q & A.
--
Engineering Manager, Language (Contributors)
Wikimedia Foundation