Hi all,
(Reposting https://www.mediawiki.org/wiki/Topic:Tyvfh19mba4pway9 here to
garner more input.)
I'm working on Extension:GlobalPreferences and trying to figure out how
best to do things with all preferences, after they've been defined (in
order to show various extra Preferences-form bits and pieces that depend
on knowing about all preferences). At the moment, we're using
$wgExtensionFunctions and hacking the $wgHooks global to add a new
callback at the end of $wgHooks['GetPreferences'].
One idea is to add a new MediaWiki service called 'PreferencesFactory',
that can be used to retrieve a new Preferences object. Extensions would
then be able to use the MediaWikiServices hook to redefine the
PreferencesFactory (with MediaWikiServices::redefineService()). Of
course, only one extension would be able to do that (which maybe is a
bit odd).
Apart from being able to override the Preferences class, a service for
this would also mean the Preferences class could be refactored
(gradually?) to not be such a collection of static methods.
The proposed patch is: https://gerrit.wikimedia.org/r/#/c/374451/
I'd love to hear anyone's ideas about this, including completely
different and better ways to do things. :-)
Another idea is to add a new hook, after GetPreferences. This wouldn't
be as flexible as the PreferencesFactory idea, but is a lot simpler.
Thanks,
Sam.
Hi all!
I'm working on the database schema for Multi-Content-Revisions (MCR)
<https://www.mediawiki.org/wiki/Multi-Content_Revisions/Database_Schema> and I'd
like to get rid of the rev_sha1 field:
Maintaining revision hashes (the rev_sha1 field) is expensive, and becomes more
expensive with MCR. With multiple content objects per revision, we need to track
the hash for each slot, and then re-calculate the sha1 for each revision.
That's expensive especially in terms of bytes-per-database-row, which impacts
query performance.
So, what do we need the rev_sha1 field for? As far as I know, nothing in core
uses it, and I'm not aware of any extension using it either. It seems to be used
primarily in offline analysis for detecting (manual) reverts by looking for
revisions with the same hash.
Is that reason enough for dragging all the hashes around the database with every
revision update? Or can we just compute the hashes on the fly for the offline
analysis? Computing hashes is slow since the content needs to be loaded first,
but it would only have to be done for pairs of revisions of the same page with
the same size, which should be a pretty good optimization.
Also, I believe Roan is currently looking for a better mechanism for tracking
all kinds of reverts directly.
So, can we drop rev_sha1?
--
Daniel Kinzler
Principal Platform Engineer
Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.
Handling of usernames in imported edits in MediaWiki has long been weird
(T9240[1] was filed in 2006!).
If the local user doesn't exist, we get a strange row in the revision table
where rev_user_text refers to a valid name while rev_user is 0 which
typically indicates an IP edit. Someone can later create the name, but
rev_user remains 0, so depending on which field a tool looks at the
revision may or may not be considered to actually belong to the
newly-created user.
If the local user does exist when the import is done, the edit is
attributed to that user regardless of whether it's actually the same user.
See T179246[2] for an example where imported edits got attributed to the
wrong account in pre-SUL times.
In Gerrit change 386625[3] I propose to change that.
- If revisions are imported using the "Upload XML data" method, it will
be required to fill in a new field to indicate the source of the edits,
which is intended to be interpreted as an interwiki prefix.
- If revisions are imported using the."Import from another wiki" method,
the specified source wiki will be used as the source.
- During the import, any usernames that don't exist locally (and can't
be auto-created via CentralAuth[4]) will be imported as an
otherwise-invalid name, e.g. an edit by User:Example from source 'en' would
be imported as "en>Example".[5]
- There will be a checkbox on Special:Import to specify whether the same
should be done for usernames that do exist locally (or can be created) or
whether those edits should be attributed to the existing/autocreated local
user.
- On history pages, log pages, and the like, these usernames will be
displayed as interwiki links, much as might be generated by wikitext like "
[[:en:User:Example|en>Example]]". No parenthesized 'tool' links (talk,
block, and so on) will be generated for these rows.
- On WMF wikis, we'll run a maintenance script to clean up the existing
rows with valid usernames and rev_user = 0. The current plan there is to
attribute these edits to existing SUL users where possible and to prefix
them with a generic prefix otherwise, but we could as easily prefix them
all.
- Unfortunately it's impossible to retroactively determine the actual
source of old imports automatically or to automatically do anything about
imports that were misattributed to a different local user in
pre-SUL times
(e.g. T179246[2]).
- The same will be done for CentralAuth's global suppression blocks.
In this case, on WMF wikis we can safely point them all at Meta.
If you have comments on this proposal, please reply here or on
https://gerrit.wikimedia.org/r/#/c/386625/.
Background: The upcoming actor table changes[6] require some change to the
handling of these imported names because we can't have separate attribution
to "Example as a non-registered user" and "Example as a registered user"
with the new schema. The options we've identified are:
1. This proposal, or something much like it.
2. All the existing rows with rev_user = 0 would have to be attributed
to the existing local user (if any), and in the future when a new user is
created any existing edits attributed to that name will be automatically
attributed to that new account.
3. All the existing rows with rev_user = 0 and an existing local user
would have to be re-attributed to different *valid* usernames, probably
randomly-generated in some manner, and in the future when a new user is
created any existing edits for that name would have to be similarly
re-attributed.
4. Like #2, except the creation (including SUL auto-creation) of the
same-named account would not be allowed. Thus, an import before the local
name exists would forever block that name from being used for an actual
local account.
5. Some less consistent combination of the "all the existing rows" and
"when a new user is created" options from #2–4.
Of these options, this proposal seems like the best one.
[1]: https://phabricator.wikimedia.org/T9240
[2]: https://phabricator.wikimedia.org/T179246
[3]: https://gerrit.wikimedia.org/r/#/c/386625/
[4]: https://phabricator.wikimedia.org/T111605
[5]: ">" was chosen rather than the more typical ":" because the former is
already invalid in all usernames (and page titles). While a colon is *now*
disallowed in new usernames, existing names created before that restriction
was added can continue to be used (and there are over 12000 such usernames
in WMF's SUL) and we decided it'd be better not to suddenly break them.
[6]: https://phabricator.wikimedia.org/T167246
--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
As was previously announced on the xmldatadumps-l list, the sql/xml dumps
generated twice a month will be written to an internal server, starting
with the November run. This is in part to reduce load on the web/rsync/nfs
server which has been doing this work also until now. We want separation
of roles for some other reasons too.
Because I want to get this right, and there are a lot of moving parts, and
I don't want to rsync all the prefetch data over to these boxes again next
month after cancelling the move:
********
If needed, the November full run will be delayed for a few days.
If the November full run takes too long, the partial run, usually starting
on the 20th of the month, will not take place.
*********
Additionally, as described in an earlier email on the xmldatadumps-l list:
*********
files will show up on the web server/rsync server with a substantial
delay. Initially this may be a day or more. This includes index.html and
other status files.
*********
You can keep track of developments here:
https://phabricator.wikimedia.org/T178893
If you know folks not on the lists in the recipients field for this email,
please forward it to them and suggest that they subscribe to this list.
Thanks,
Ariel
Hello all,
It's that time of year again where the density of holidays increases and
at the same time our plans for fundraising also increase.
Per our usual practice we will be not doing deployments at various
points in the next few months. Here's the full outline:
Reminder for all on we did last year:
* No MW train the week of Thanksgiving (but SWAT deploys were open for
high-priority things).
* No deploys (at all) the last two weeks of December. People were happy
with this.
* The first week of January was normal (minus Monday being Jan 2nd, our
observed New Year's Day holiday) deployment wise.
* The second week was weird due to Dev Summit/All Hands: No MediaWiki
train, only SWATs as needed on Mon/Tues/Wed. No SWATs/deploys during
All Hands.
I imagine we'll do similarly. In that case (looking at the calendar....)
* No MW train the week of Thanksgiving (Nov 20th), SWATs open for high
priority things
* No deploys weeks of Dec 18th and 25th (last two weeks)
* Normal week week of Jan 1st (minus no deploys that Monday)
* The Dev Summit and WMF All Hands is the week of January 22nd, so that
will be a "No Train but SWATs OK on Mon/Tues/Wed" week.
* The following week (week of January 29th) the Release Engineering team
will be on an offsite, so a week of "No Train, but SWATs and service
deploys OK".
This is now on-wiki at:
https://wikitech.wikimedia.org/wiki/Deployments#Upcoming
Best,
Greg
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| Release Team Manager A18D 1138 8E47 FAC8 1C7D |
https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-10-25
= 2017-10-25 =
== Callouts ==
* Services are running a new JobQueue test in beta cluster, so if you see
anything strange in beta - please notify
* Mukunda broke scap in beta, will get it fixed soon!
== Audiences ==
=== Readers ===
==== Multimedia ====
* Blocking: None
* Blocked by: None
* Updates: MediaInfo/Wikibase on Commons work is continuing,
https://phabricator.wikimedia.org/T177022
* UploadWizard changes for 3D legal considerations,
https://phabricator.wikimedia.org/T178513
* and discussions about MP3.
* Heads up: Mark on vacation next week, plus three days in the following
week - will respond but may be AFK
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
** Apps offsite last week
** 5.7.0 will be released this week (
https://phabricator.wikimedia.org/project/view/2899/ ) - Visual updates to
onboarding, iOS 11 support, iPhone X support
** Starting work on 5.7.1 -
https://phabricator.wikimedia.org/project/view/3047/ and 5.8 -
https://phabricator.wikimedia.org/project/view/2913/
==== Reading Web ====
- currently offsite-ing
==== Reading Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** Reading lists available in beta & labs:
https://restbase-reading.wmflabs.org/en.wikipedia.beta.wmflabs.org/v1/ +
http://readinglists.wmflabs.org/api/rest_v1/#/Reading_lists
==== Discovery ====
* Blocked by:
* Blocking:
* Updates:
* continuing work on portal automation, general clean-ups, adding SVGs
=== Contributors ===
==== Parsing ====
* Blocked by: Parsoid debian package upload blocked by change needed to the
deb-upload script (Daniel Zahn replied on IRC y'day as he was rushing to
catch a flight that the release server has changed). But, not sure if
anything is needed on our end or will Ops fix this? I can create a phab
ticket if required.
* Blocking:
* Updates:
** Parsing team back from offsite (last week). Nothing significant to
report for purposes of SoS. Focusing on some perf work this week.
==== Services ====
* Blockers: none
* Updates:
** All jobs in beta cluster are processed via kafka as a test. Please
tell us if you notice anything weird
** Deprecated /titile/ and /title/{title}/ listings in REST API has
been removed
=== Community Tech ===
* Preparing for a new wishlist survey
* Deploying Unicode sections soon
=== TechOps ===
* Blocked by:
** Flow isAllowed gets actual revision text before it is needed
https://phabricator.wikimedia.org/T172025
* Blocking:
* Updates:
** Procurement for Asia datacenter has started
** We have IP addresses allocated
** Work on unifying production and CI build pipeline ongoing
https://phabricator.wikimedia.org/T177276
** Database shard s5 split into s8 ongoing
https://phabricator.wikimedia.org/T172679
** Work on porting varnish request stats scripts to Prometheus
https://phabricator.wikimedia.org/T177199
** Puppet modernization ongoing https://phabricator.wikimedia.org/T177254
=== Search Platform ===
* Blocked by: none
* Blocking: none
* Updates:
* Finished removal of messaging fallbacks from Elastic indexing (
https://phabricator.wikimedia.org/T177871)
* Relaxing phrase query filter A/B finished, analyzing (
https://phabricator.wikimedia.org/T177956)
* Checked language analyzers to see whether we want to research other
morphological libraries (https://phabricator.wikimedia.org/T171652)
** See the plan in https://phabricator.wikimedia.org/T171652#3707331
* Looking at top abandoned queries (
https://phabricator.wikimedia.org/T176997)
* Extended a set of default search namespaces on several wikis (
https://phabricator.wikimedia.org/T170473)
* Enabled stricter throttling on WDQS to deal with clients that issue tons
of short requests very fast
* Enabling ElasticSearch prefix search on Wikidata today
* Working on porting Selenium tests from Ruby to JS
* Working on upgrade to Elastic 5.5
* Working on indexing Wikidata descriptions and adding them to fulltext
search
=== Release Engineering ===
* Blocked by: None
* Blocking: None
* Updates:
** Mukunda broke scap in beta, will get it fixed soon! (
https://phabricator.wikimedia.org/T179013 )
** Deployment logspam is mostly quiet, except one non-deployment related
log from the tidy migration (Parsing/MW Platform know)
** Zeljko paired with Elena T. to get some Echo notification browser tests
written (nodejs/mocha framework).
** Zeljko scheduled a Tech Talk on Oct 31st to discuss/teach the nodejs
browser testing framework.
** Scap tech-debt project well underway, merged many improvements last week
** new/docker based CI: the phan job is now migrated
** Gerrit was upgraded last week (minor version)
** Working on git-lfs (large file store, for big binary blobs) support in
Gerrit for ORES and Reading teams
** Redesigned (simplified) the Phabricator login screen <
https://phabricator.wikimedia.org/D831>, should be deployed soon.
=== Security ===
* Blocked by: None
* Blocking: whowever is waiting for security reviews
* Updates:
* Node Security Project 3.0.0 being released soon (available as nsp@next);
please consider upgrading if you work on Node.js-based projects
* Reviews:
* pdfrw
* more will be scheduled soon
== Wikidata ==
* Preparing WikidataCon:
https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017
* We will live-stream the main tracks on Saturday and Sunday:
https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Program/Remote
Hi, Moriel. All of them were in the old wikitext editor.
Igal
On Oct 29, 2017 05:03, "Moriel Schottlender" <mschottlender(a)wikimedia.org>
wrote:
Igal, I'm not sure what this is but a suspicion was raised as to a
potential reason.
Can you see if there's a difference in this between edits you're making in
VisualEditor versus edits that are made through the wikitext editor?
On Sun, Oct 29, 2017, 3:21 AM יגאל חיטרון <khitron(a)post.bgu.ac.il> wrote:
> Hi. FYI: There were a lost of my own edits in the past week that were
> marked as unseen in watchlist (and [0]). I can't open a phab ticket,
> because it's not reproduceable. So if you know what to do with this,
please
> do, otherwise just ignore this letter. Thank you.
> Igal (User:IKhitron)
>
> [0]:
>
> https://he.wikipedia.org/w/api.php?action=query&list=
watchlist&wlshow=unread&wlallrev=1&format=xmlfm&wllimit=100&wlprop=ids%
7Ctitle%7Cflags%7Ccomment%7Ctimestamp
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
_______________________________________________
Wikitech-l mailing list
Wikitech-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hello all,
I would like to announce the release of MediaWiki Language Extension
Bundle 2017.10. This bundle is The bundle is compatible with MediaWiki
1.28 and 1.29 or above and requires PHP 5.5.9 or above.
Next MLEB is expected to be released in 3 months. If there are major
changes or important bug fixes, we will do intermediate release.
Please give us your feedback at
[[Talk:MLEB|https://www.mediawiki.org/wiki/Talk:MLEB]].
* Download: https://translatewiki.net/mleb/MediaWikiLanguageExtensionBundle-2017.10.tar…
* sha256sum: 75eefff3bbf50f1f84232f45ab169d2eedea5f7a18994f85b43237f2145403b1
Quick links:
* Installation instructions are at: https://www.mediawiki.org/wiki/MLEB
* Announcements of new releases will be posted to a mailing list:
https://lists.wikimedia.org/mailman/listinfo/mediawiki-i18n
* Report bugs to: https://phabricator.wikimedia.org/
* Talk with us at: #mediawiki-i18n @ Freenode
Release notes for each extension are below.
-- Kartik Mistry
== Highlights and upgrade notes ==
== Babel ==
=== Noteworthy changes ===
* Style updates.
== cldr ==
=== Noteworthy changes ===
* Localisation and maintenance updates only.
== CleanChanges ==
=== Noteworthy changes ===
* Localisation and maintenance updates only.
== LocalisationUpdate ==
=== Noteworthy changes ===
* Localisation and maintenance updates only.
== Translate ==
=== Noteworthy changes ===
* Special:Translate now accepts showMessage URL parameter that allows
linking to specific messages. This will be better accessible in a
future release.
* Special:LanguageStats and Special:MessageGroupStats now show the
total number of shown languages.
* Special:LanguageStats and Special:MessageGroupStats now handle the
sortable parameter correctly.
* Numerious compatibility and stability fixes:
** Translation editor shortcuts were not showing up in recent version
of MediaWiki.
** Compatibility fix for PHP 7.1.
** Compatibility fix for translatable pages and Visual Editor wikitext
editing mode.
** Compatibility fix for Elastica that broke translation memory.
** Compatibility fix for CxserverWebService which changed API.
* Style updates.
== UniversalLanguageSelector ==
=== Noteworthy changes ===
* Language searching is improved with new language name data and
better "no results found" view.
* Popup positioning should now work better in various skins.
* Style updates.
--
Kartik Mistry/કાર્તિક મિસ્ત્રી | IRC: kart_
{kartikm, 0x1f1f}.wordpress.com
Hi,
I just moved the npm jobs in the MediaWiki queue over to the new,
faster, docker based CI system. Instead of requiring building a fresh VM
for each 20-30 second job, it uses a docker container. I'm expecting
this to make a noticeable improvement in CI response times.
For the most part everything should just work, if something doesn't,
please file a bug in the #ci-config project and CC me (legoktm).
If something has gone terribly wrong and I'm not around, you can ask a
CI team member in #wikimedia-releng to revert [1] to put everything back
to the way it was.
[1] https://gerrit.wikimedia.org/r/#/c/387500/
-- Legoktm
# Who 👨💻
Željko Filipin, Engineer (Contractor) from Release Engineering team. That's
me! 👋
# What 📆
Selenium tests in Node.js. We will write a new simple test for a MediaWiki
extension. An example: https://www.mediawiki.org/wiki/Selenium/Node.js/Write
# When ⏳
Tuesday, October 31, 16:00 UTC
# Where 🌍
The internet! The event will be streamed and recorded. Details coming soon.
# Why 💻
We are deprecating Ruby Selenium framework:
https://phabricator.wikimedia.org/T173488
See you there!
Željko Filipin