Hi all,
I was happy to be invited by the Wikimedia Foundation to the Wikimedia
Technical Conference 2019 in Atlanta~[1]. At this conference, I represented
the technical needs of the mathematical community~[2]. Apart from a lot of
great achievements for the whole Wikimedia movement with regard to the five
focus areas of the conference~[3], there were also several math-specific
achievements. In this message, I will focus on those aspects:
1) Bold italic capital greek symbols are now possible. Please join me in
thanking Petr Kadlec (aka. Mormegil) for his perennial effort to make this
possible~[4].
2) We will - very soon - have a demo on the Wikimedia beta cluster enabling
links from formulae to a dedicated special page that displays definitions and
explanations regarding mathematical objects of interest, i.e., identifiers,
symbols, terms. Thank you André Greiner-Petter for the implementation of this
feature~[5].
3) In a session on 'Integrating contributions from other teams or volunteers`
organized by Christoph Jauera (aka. Fisch). We derived definitive action items
on improving the participation opportunities to the Math on wikis~[6].
4) We discussed the future of Math rendering with Petr Pchelko (WMF) which
will simplify the setup of mathoid and eventually get rid of fallback images
while maintaining support for MathML disabled browsers~[7].
I am grateful to the Wikimedia Foundation and the organizers of the event, in
particular, Rachel and Greg to get the chance to enjoy this well-organized
conference with amazing people and a wonderful program.
All the best
Moritz (physikerwelt)
http://moritzschubotz.de | +49 1578 047 1397
[1]:https://www.mediawiki.org/wiki/Wikimedia_Technical_Conference/2019
[2]:https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math
[3]:https://phabricator.wikimedia.org/T238406
[4]:https://phabricator.wikimedia.org/T218295
[5]:https://phabricator.wikimedia.org/T208758
[6]:https://phabricator.wikimedia.org/T234662#5660102
[7]:https://phabricator.wikimedia.org/T237516
Some more details on item 4, from my personal, biased perspective:
I started working on the Math extension in 2012 and implemented the
main functionality
of the new rendering method which is based on MathJax rather
than on LaTeX in 2013 at the Wikimedia Foundations Headquarter in San
Francisco. At that time the vision was to replace the monolithic PHP based
framework MediaWiki, with a large number of small dynamic JavaScript modules.
The idea was that those modules are developed as isomorphic platform-independent
components using interfaces of a management framework that takes
care of caching and efficient execution. The long-term goal was that the
functionality could be executed either on the client or on the server and that
the management layer would figure out the best execution strategy based on the
current prerequisites. In the first step, a framework based on HTTP requests was
set up to handle services such as math rendering. Mathoid the math rendering
service was one of the first instances of this service template. From the
retro perspective that might have been too early. Neither a convenient type and
schema description language existed, nor a way to specify rich metadata on the
execution characteristics existed. However, a fine-grained I/O schema seems
desirable for implementing robust and durable services. Moreover, rich
metadata on the execution characteristics such as runtime, memory footprint,
I/O data distributions seem required to allow an execution management layer
for effective execution strategy planning. After the services went into
production schema improvement was difficult and never happened.
Today, it seems pretty certain that MediaWiki will be PHP based for the
foreseeable future. Given this situation, we did now plan to improve Math
support by making the best use of the build-in MediaWiki core functionality. We
will rely on the MediaWiki core caching functionality to continue providing an
instant user perception of math rendering and continue using a stateless
node-based math rendering service. Our hope is that by incorporating the 'new`
MathJax 3 rendering mode 'common HTML` also MathML disabled browsers will be
able to display high-quality mathematical formulae without to rely on
disturbing images. We plan to enable this rendering mode as opt-in in a first
step and thereafter have a community vote if the new imageless rendering mode
should become the default. If that won't work, we will need to
evaluate inline SVG
images that would require either SVG or JavaScript support on the client.
Given the situation that only a very small number of visitors use browsers
that neither support SVG images nor allow JavaScript this second alternative
seems to an ethical option as well.
I will update the associated ticket phabricator ticket~[7] as soon as we have
derived a more detailed plan for the implementation.
Hi,
for HTML version see
https://www.mediawiki.org/wiki/Scrum_of_scrums/2019-12-18
Željko
--
= 2019-12-18 =
== Callouts ==
* SRE:
** Kubernetes staging and codfw cluster were bootstrapped from scratch
today. All apps have been redeployed in the exact same state as before and
everything seems to be nominal, but if you notice anything reach out to SRE
** EQIAD kubernetes cluster will happen after the holidays
** Product Infrastructure needs +2 rights for all PI engineers in
deployment-charts ([[phab:T232794]])
* Release Engineering - December 23-January 3rd - No deploys (including
train), Holiday break
== Product ==
=== Editing ===
* Updates:
** ArticleTargetLoader: Check apiXhr exists
** Don't bother fetching metadata for diffs
** Wikitext links: File/Category support
** Support wikitext link editing by Ed Sanders
** Error handling
*** ve.init.mw.ArticleTargetSaver: Fix exception when handling errors
*** ve.init.mw.MobileArticleTarget: Don't close overlay when showing error
*** ve.init.mw.DesktopArticleTarget: Remove dead error handling code
** ve.init.mw.ArticleTargetLoader: Fix promises to be abortable again
** Exactly match collapsible logic of jquery.makeCollapsible (task
[[phab:T236247]])
** ApiVisualEditorEdit: Normalize newlines in wikitext (task
[[phab:T154977]])
** ve.ui.MWEducationPopupTool: Respect default preferences for anon users
(task [[phab:T231762]])
=== Growth ===
* Updates:
** build: Updating mediawiki/minus-x to 0.3.2 by libraryupgrader
** Inject PageViewService into Impact and SpecialImpact (task
[[phab:T240444]])
** Make PageViewInfo a soft dependency (task [[phab:T240444]]) by Kosta
Harlan
** Suggested Edits
***Remove close button from widget, close on hover out (task
[[phab:T238165]])
*** Use unpadded (i) icon on mobile (task [[phab:T236854]])
*** Consistently use suggested-edits-, not suggestededits- (task
[[phab:T236854]])
*** Do not update state until user clicks "Done" (task [[phab:T238459]])
(task [[phab:T238172]])
*** Record pre-activation as a preference (task [[phab:T238888]])
*** Don't fail so hard if pcs/aqs data are undefined
*** Use placeholder icon instead of larger image (task [[phab:T238282]])
*** Update copy for pageviews component (task [[phab:T238322]])
*** Restore no-repeat on task cards
=== Android native app ===
* Updates:
** Minor update released to production (tweaks requested by Analytics)
** Finishing up mobile-html integration.
=== Web ===
* Updates:
** Summary: Planning for desktop improvements, last minute bug fixes, and
wrapping up advanced mode.
** [[Reading/Web/Desktop_Improvements|Desktop Improvements Project (Vector
/ DIP)]]:
*** Planning, research, and task creation
*** [[phab:T240622|<nowiki>[Technical debt payoff] Remove
InlineDiffFormatter and InlineDifferenceEngine from
MobileFrontend</nowiki>]]
*** [[phab:T232140|<nowiki>Separate out logo handling into square image
logos and long text/wordmark banner logos</nowiki>]]
*** [[phab:T237561|<nowiki>[SPIKE 10hrs] Investigate how skin suboptions
can be presented in Special:Preferences</nowiki>]]
*** [[phab:T237635|<nowiki>[SPIKE] What should feature management look like
in desktop improvements?</nowiki>]]
** Page previews (Popups):
*** [[phab:T236589|<nowiki>Page Previews Storybook not available on
doc.wikimedia.org and documentation a bit broken</nowiki>]]
** Mobile website (MinervaNeue / MobileFrontend):
*** [[Reading/Web/Advanced_mobile_contributions|Advanced mode]]:
**** [[phab:T240644|<nowiki>Regression: Label is visible on Logout button
for logged in users on all wikis (non-AMC) and contribution icon on user
page</nowiki>]]
**** [[phab:T240487|<nowiki>XSS in MobileFrontend</nowiki>]]
**** [[phab:T238364|<nowiki>[Bug] Simplified Talk Page Does Not Support
Sections With Non-ascii Characters/Several ascii characters aren't
supported either</nowiki>]]
**** [[phab:T232653|<nowiki>Updates to user pages for all users</nowiki>]]
**** [[phab:T232652|<nowiki>Add History to article toolbar for all
logged-in users</nowiki>]]
**** [[phab:T232594|<nowiki>Add article and discussion tabs to all
logged-in users on mobile</nowiki>]]
**** [[phab:T235681|<nowiki>[AMC] Toolbar on User talk page should match
toolbar on User page</nowiki>]]
*** Miscellaneous:
**** [[phab:T214647|<nowiki>[EPIC] Re-define the contract for displaying
drawers and overlays in MobileFrontend</nowiki>]]
**** [[phab:T237290|<nowiki>Disable mobile beta mode (for now)</nowiki>]]
**** [[phab:T214045|<nowiki>CTA, reference and Red link Drawers should be
removed from DOM after close</nowiki>]]
**** [[phab:T177260|<nowiki>Rewrite disabled MobileFrontend browser tests
to phpunit tests and Node.js browser tests</nowiki>]]
**** [[phab:T198265|<nowiki>Special:MobileOptions is empty for third
parties and this is confusing</nowiki>]]
** WikidataPageBanner:
*** [[phab:T191689|<nowiki>PAGEBANNER "origin" parameter calculation is not
working at all.</nowiki>]]
*** [[phab:T122096|<nowiki>[EPIC] Cleanup WikidataPageBanner
architecture</nowiki>]]
** Miscellaneous:
*** [[phab:T211985|<nowiki>Improve language icon discoverability in mobile
by using icon in desktop</nowiki>]]
=== Product Infrastructure ===
* Blocked by:
** SRE on providing buster node images for proton
** SRE/RelEng? on adding +2 rights for all PI engineers to
deployment-charts ([[phab:T232794]])
* Updates:
** Computer-Aided Tagging is "soft-launched," on Commons, please report any
issues.
** Investigating job queue lag caused by fetchGoogleCloudVisionAnnotation
jobs ([[phab:T241072]])
=== Structured Data ===
* Blocking:
** Search Platform: Data dumps for SDC: [[phab:T221917]]
* Updates:
** see above for computer-aided tagging
** some new input types available for statements on commons
** dumps unblocked
=== Parsing ===
* Updates:
** We are looking to enabled Parsoid/PHP for the language variants this
week which will effectively mark the end of Parsoid/JS in production.
=== Inuka ===
* Updates:
** KaiOS app: responsive layout (for landscape-oriented devices)
[[phab:T240790]]
** KaiOS app: links preview (still working on references) [[phab:T234625]]
** KaiOS app: various bug fixes
== Technology ==
=== Fundraising Tech ===
* Updates:
** CiviCRM
*** On-demand sending of end-of-year summary receipt emails
*** Better deadlock handling in donation queue consumers
** Improving visibility of recurring donations and endowment donations on
internal dashboard
** CentralNotice
*** Fixing bug in status code when campaign fallback is active
*** Revising UI elements of region-level targeting
** Paymentswiki
*** Investigating charging NL bank donations through a different payment
processor
=== Core Platform ===
* Blocking:
** Search Platform: MW Job consumers sometimes pause for several minutes
[[phab:T224425]]
* Updates:
** Virtual offsite last week
** Finishing last sprint for the year this week
=== Engineering Productivity ===
==== Release Engineering ====
* Blocking:
** Product Infrastructure - SRE/RelEng? on adding +2 rights for all PI
engineers to deployment-charts ([[phab:T232794]])
* Updates:
** Train Health
*** Last week: 1.35.0-wmf.10 - [[phab:T233858]]
*** This week: 1.35.0-wmf.11 - [[phab:T233859]]
*** Next week: No Train - 2 weeks!
=== Search Platform ===
* Blocked by:
** Structured Data: Data dumps for SDC: [[phab:T221917]]
** Core: MW Job consumers sometimes pause for several minutes
[[phab:T224425]]
* Updates:
** CirrusSearch writes are split into per cluster kafka partitions to
isolate clusters from each other [[phab:T235832]]
=== Site Reliability Engineering ===
* Blocking:
** PI in providing buster docker images for proton
** Product Infrastructure - SRE/RelEng? on adding +2 rights for all PI
engineers to deployment-charts ([[phab:T232794]])
* Updates:
** Kubernetes staging and codfw cluster were bootstrapped from scratch
today. All apps have been redeployed in the exact same state as before and
everything seems to be nominal, but if you notice anything reachout to SRE
*** EQIAD kubernetes cluster will happen after the holidays
The Extension:VisualEditor page on Mediawiki.org still guides people to use Parsoid/JS. Where is the best place to find instructions for migrating a wiki installation from Parsoid/JS to Parsoid/PHP? If documentation doesn't exist, I can help create it once I understand the steps. I've also opened a topic on the Extension_talk:VisualEditor page if that's a more convenient place to discuss: https://www.mediawiki.org/wiki/Topic:Vd9tmhoaax7re3it
Thank you!
Michael
Summary
-------
Parsoid/PHP, the PHP port of Parsoid is now live everywhere for all
products on all wikis. Parsoid/JS is still deployed on the Wikimedia
cluster but doesn't receive any traffic and will be decommissioned
in the new year.
Context: Making Parsoid the default MediaWiki wikitext engine
-------------------------------------------------------------
In 2018, we completed the replacement of HTML4 Tidy from MediaWiki
with RemexHTML, a HTML5 parser.
The port of Parsoid to PHP is the next step along the way to integrate
Parsoid and the MediaWiki wikitext parser into a single wikitext engine.
That final product will let us bring the benefits of Parsoid's approach
to a wider set of products and let us start work on improving templates
and wikitext or other features without having to implement that in two
wikitext engines with different processing models.
See a Feb 27 tech talk for a full context [1]. A future blog post will
provide more details about the porting project and process.
Performance
-----------
Parsoid/PHP on the wikimedia cluster seems to be about 2x faster than
Parsoid/JS for the wikitext -> HTML and HTML -> wikitext endpoints.
This performance bump is a pleasant surprise given that going in, we
anticipated to incur some performance penalty. While we have various
theories about the factors contributing to this, we haven't had the
opportunity yet to investigate fully. The load on the MediaWiki Action API
endpoint was also significantly reduced on Dec 13 when we stopped
processing mirrored traffic on Parsoid/JS (Parsoid/PHP accesses the
database directly instead).
Timeline
--------
We started porting in earnest in Feb with some preparatory work over
the previous few months. We deployed Parsoid/PHP as a passive mirror
of the full volume of wikitext -> HTML requests in October & November
to discover and fix problems early. By Dec 2, we’d enabled Parsoid/PHP
for the majority of Parsoid clients on all wikis. On Dec 18, we switched
everything over to Parsoid/PHP.
Compared to an earlier estimate of 9 months, we took about 2 months longer
to get to this milestone.
Thanks to all the testing in place, in the end, this Parsoid/JS to
Parsoid/PHP switch went fairly smoothly with some minor glitches.
Credits
-------
This project to port Parsoid from Javascript (Node.js) to PHP was a
multi-team collaboration. The Parsing, Core Platform, Product
Infrastructure,
Service Ops, Security teams from the Wikimedia Foundation were the relevant
teams. We also benefited with some contractorhelp from Wikiteq. Thanks to
everyone involved!
[1]
https://www.mediawiki.org/wiki/Wikimedia_Technical_Talks#Episode_1:_The_lon…
Hello everyone,
As of this week, the Readers Web team has expanded the deployment of a
small set of features from the mobile "advanced mode" users to all
logged-in users. All logged-in users will now have access to article and
talk tabs, as well as direct access to history pages.
We hope that these changes will allow new contributors to access these
parts of the interface with more ease. For more advanced contributors, we
do recommend using the "advanced mode", which includes the above features
and more!
Thank you!
Olga
--
*Olga Vasileva* // Reading Product Manager // Web
https://wikimediafoundation.org/
*Imagine a world in which every single human being can freely share in
the sum of all knowledge. That's our commitment. Donate
<http://donate.wikimedia.org/>. *
Hello colleagues,
This is a reminder that the December 2019 Wikimedia Café meeting will start
approximately one hour from now.
Pine
( https://meta.wikimedia.org/wiki/User:Pine )
On Fri, Nov 29, 2019 at 10:50 PM Pine W <wiki.pine(a)gmail.com> wrote:
> Hello colleagues,
>
> For the next Wikimedia Café online video meeting, agenda items include:
> * Documentation for Wikitext <https://en.wikipedia.org/wiki/Help:Wikitext>
> * The NavWiki <https://outreach.wikimedia.org/wiki/NavWiki> project
>
> Last month's Wikimedia Café had a problem with an inaccurate time
> conversion from USA Eastern time to UTC. People attempted to join the
> meeting at different times based on whether they referred to UTC or to US
> Eastern time. I apologize for the error, and I think that it has been
> corrected for December.
>
> The meeting style for the Café will emphasize discussion rather than
> presentation. People are welcome to participate as listeners only if they
> prefer.
>
> Please see the page on Meta
> <https://meta.wikimedia.org/wiki/Wikimedia_Caf%C3%A9> for more
> information about the Café. Please watch the page for any updates,
> particularly to the schedule or the agenda. Signing up for the meeting is
> optional, but is helpful to the organizers so that we can estimate how many
> people will attend. Signing up for the meeting also informs us who we
> should notify individually if there are significant changes, such as to the
> schedule.
>
> If there are any problems with connecting to the meeting or if you have
> any questions or comments then please write on the Meta talk page.
>
> Regards,
> Pine
> ( https://meta.wikimedia.org/wiki/User:Pine )
>