Hello all,
There are a few new log messages that have crept their way into the
1.30.0-wmf.9 train release currently winding its way down the track - on
group0 and group1 wikis[0]. I've halted the train for the time being due
to these new messages[1].
1. T170599[2] - Wikibase: $idSerialization must match /^Q[1-9]\d{0,9}\z/i
2. T170596[3] - Could not acquire lock 'LinksUpdate:job:pageid:xxx'
3. T170597[4] - Wikidata/extensions/Constraints:
InvalidArgumentException:$itemId must be either ItemId or string
RelEng makes an attempt during train to keep the introduction of new log
messages per-release to a minimum. Logspam masks real problems and can
make the job of deployers and developers unpleasant.
Any help or guidance on any of these three tasks would be very much
appreciated!
Thanks!
-- Tyler
[0]. <https://tools.wmflabs.org/versions/>
[1]. <https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#Logspam>
[2]. <https://phabricator.wikimedia.org/T170599>
[3]. <https://phabricator.wikimedia.org/T170596>
[4]. <https://phabricator.wikimedia.org/T170597>
Hey folks!
I just posted a new update to the blog. This update covers roughly the
last month.
https://phabricator.wikimedia.org/phame/post/view/58/status_update_july_11t…
As of July 1st, we are officially the Scoring Platform team. We're
welcoming Adam Wight to the team officially. There will be a nice
announcement that we'll post to the Wikimedia Blog in a few days.
The last ~month was very productive, but we had two major production
issues[1,2]. As you will see in the blog post, there's a series of tasks
that address problems that were related to these issues.
Despite dealing with production issues, we've been able to get a very
substantial change to the revscoring library merged. This change will make
accessing information about models (build environment, test statistics,
scoring thresholds, etc.) much easier. This will cause a breaking change in
ORES UI so we'll be making an announcement when we roll it out. Stay tuned.
We've also increased our language and model coverage substantially. We even
built and deployed a totally new type of model to help out French
Wikisource!
See the post for more details :)
1. https://wikitech.wikimedia.org/wiki/Incident_documentation/20170613-ORES
2. https://wikitech.wikimedia.org/wiki/Incident_documentation/20170623-ORES
-Aaron
Principal research scientist
Lead of the Scoring Platform team
Defender of the universe
Eater of toast
Wiki of the media foundation
TL;DR:
* "Tool Labs" is being renamed to "Toolforge"
* The name for our OpenStack cluster is changing from "Labs" to "Cloud VPS"
* The prefered term for projects such as Toolforge and Beta Cluster
running on Cloud VPS is "VPS projects"
* "Data Services" is a new collective name for the databases, dumps,
and other curated data sets managed by the Cloud Services team
* "Wiki replicas" is the new name for the private-information-redacted
copies of Wikimedia's production wiki databases
* No domain name changes are scheduled at this time, but we control
wikimediacloud.org, wmcloud.org, and toolforge.org
* The Cloud Services logo will still be the unicorn rampant on a green
field surrounded by the red & blue bars of the Wikimedia Community
logo
* Toolforge and Cloud VPS will have distinct images to represent them
on wikitech and in other web contexts
In February when the formation of the Cloud Services team was
announced [0] there was a foreshadowing of more branding changes to
come:
> This new team will soon begin working on rebranding efforts intended
> to reduce confusion about the products they maintain. This refocus
> and re-branding will take time to execute, but the team is looking
> forward to the challenge.
In May we announced a consultation period on a straw dog proposal [1]
for the rebranding efforts [2][3]. Discussion that followed both on
and off wiki were used to refine the initial proposal [4]. During the
hackathon in Vienna the team started to make changes on Wikitech
reflecting both the new naming and the new way that we are trying to
think about the large suite of services that are offered. Starting
this month, the changes that are planned [5] are becoming more visible
in Phabricator and other locations.
It may come as a surprise to many of you on this list, but many
people, even very active movement participants, do not know what Labs
and Tool Labs are and how they work. The fact that the Wikimedia
Foundation and volunteers collaborate to offer a public cloud
computing service that is available for use by anyone who can show a
reasonable benefit to the movement is a surprise to many. When we made
the internal pitch at the Foundation to form the Cloud Services team,
the core of our arguments were the "Labs labs labs" problem [6] and
this larger lack of awareness for our Labs OpenStack cluster and the
Tool Labs shared hosting/platform as a service product.
The use of the term 'labs' in regards to multiple related-but-distinct
products, and the natural tendency to shorten often used names, leads
to ambiguity and confusion. Additionally the term 'labs' itself
commonly refers to 'experimental projects' when applied to software;
the OpenStack cloud and the tools hosting environments maintained by
WMCS have been viable customer facing projects for a long time. Both
environments host projects with varying levels of maturity, but the
collective group of projects should not be considered experimental or
inconsequential.
[0]: https://lists.wikimedia.org/pipermail/labs-l/2017-February/004918.html
[1]: https://en.wikipedia.org/wiki/Straw_man_proposal
[2]: https://lists.wikimedia.org/pipermail/labs-l/2017-May/005002.html
[3]: https://lists.wikimedia.org/pipermail/wikitech-l/2017-May/088184.html
[4]: https://wikitech.wikimedia.org/wiki/User:BryanDavis/Rebranding_Cloud_Servic…
[5]: https://phabricator.wikimedia.org/T168480
[6]: https://wikitech.wikimedia.org/wiki/Labs_labs_labs
Bryan (on behalf of the Wikimedia Cloud Services team)
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Manager, Cloud Services Boise, ID USA
irc: bd808 v:415.839.6885 x6855
Hello, This is the Discovery update for last week. Apologies for the
delay in getting it out.
== Discussions ==
=== Search ===
* Created a method for the Kafka consumer to take 'learn to rank'
queries from a queue and run them against ElasticSearch to generate
relevance labels [0]
* Added in the ability to use kafka in our LTRank feature generation
queries and pushing them into ElasticSearch for analysis [1]
* Added ability to extract TF and IDF based features in the
ElasticSearch 'learning to rank' plugin [2]
* A/B test still in progress 'explore similar' links, but we're
running into a few bugs that will be sorted out next week [3]
* Fixed a bug where searching for phrase queries did not highlight
page content [4]
=== Analysis ===
* Fixed a bug with the sister project snippets and eventlogging [5]
* Finished up analysis for determining what is a reasonable per-IP
ratelimit for maps [6]
* Fixed a minor dashboard bug (splines) [7]
[0] https://phabricator.wikimedia.org/T162059
[1] https://phabricator.wikimedia.org/T162072
[2] https://phabricator.wikimedia.org/T167437
[3] https://phabricator.wikimedia.org/T164856
[4] https://phabricator.wikimedia.org/T167798
[5] https://phabricator.wikimedia.org/T168916
[6] https://phabricator.wikimedia.org/T169175
[7] https://phabricator.wikimedia.org/T169125
Yours,
Chris Koerner
Community Liaison
Wikimedia Foundation
https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-07-12
*= 2017-07-12 =*
contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering
== Callouts ==
* Reading Web found a browser bug that might be interesting to others:
Firefox fires onload twice in certain circumstances
https://phabricator.wikimedia.org/T170018
* RelEng/Performance: 3D thumbnails broken on beta, Multimedia could your
input https://phabricator.wikimedia.org/T170444
** Parsing: Need input about naming of a property in Parsoid's DOM output
for language variants ( see
https://lists.wikimedia.org/pipermail/mediawiki-i18n/2017-July/001129.html )
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by: none
* Blocking: none
* Updates: 5.5.1 Released. Working on 5.6 (Dark mode, on this day in the
feed) : https://phabricator.wikimedia.org/project/view/2701/
==== Android native app ====
* Blocked by: n/a
* Blocking: n/a
* Updates: (all ongoing)
** reading list syncing UI work is nearly done; on deck: ZIM support
** Stephen working on cross-platform CSS/JS consolidation
** engineer interviews in progress
==== Reading Web ====
* Blocked by:
* Blocking:
* Updates:
**Discovered a browser bug that impacts EventLogging (
https://phabricator.wikimedia.org/T170018 ) If your projects are relying on
it you may want to check that.
**Planning to roll out Page previews to all wikis except German and English.
**Minerva and MobileFrontend will be split this week
**Updating Vector's print styles
==== Reading Infrastructure ====
* Blocked by: N/A
* Blocking: N/A
* Updates:
** Discussions about Summary 2.0 Spec and title properties.
** Measuring payload reduction of MCS transformations with removal of
reference lists.
** working on Reading Lists
==== Multimedia ====
* Blocked by: N/A
* Blocking: N/A
* Updates
** Beta thumbnails are looking better, but 3D thumbnails are still broken -
could use input from releng/performance about what might be causing this
https://phabricator.wikimedia.org/T170444
** Patch to properly label webm audio files as audio/webm (currently
video/webm)
==== Discovery ====
* Blocked by: N/A
* Blocking: N/A
* Updates:
** (maps) Ops out this week, ops + developers out next week
** (search) running A/B test on Special:Search "explore similar" feature
=== Community Tech ===
* Blocked by:
* Blocking:
* Updates:
** HTML5 section IDs (https://phabricator.wikimedia.org/T152540) are still
coming
** Preparing CodeMirror and LoginNotify for wider deployment
=== Contributors ===
==== Editing ====
* Blocked by:
* Blocking:
* Updates:
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Need input about naming of a property in Parsoid's DOM output for
language variants ( see
https://lists.wikimedia.org/pipermail/mediawiki-i18n/2017-July/001129.html )
==== Global Collaboration ====
===== Collaboration =====
* Blocked by:
* Blocking:
* Updates:
** Better UI for Echo notification blacklist feature
** RCFilters
*** Initial implementation of live updates!
*** Bug fixes
** Article Reminder notification
*** Initial implementation behind feature flag
==== Contributors Design ====
* Blocked by:
* Blocking:
* Updates:
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
** Building CentralNotice admin button to clear banner cache for specific
banners / languages: https://phabricator.wikimedia.org/T168673
** Upgrading integration with our main credit card provider:
https://phabricator.wikimedia.org/T163946
** Deployed initial version of pulling 3rd-party mailer data into CiviCRM:
https://phabricator.wikimedia.org/T161758
** Fixing stubborn PayPal Express Checkout / dead session bug:
https://phabricator.wikimedia.org/T167923
** Optimizing thank you email sender job
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
=== Performance ===
* Blocked by:
* Blocking:
* Updates:
=== Release Engineering ===
* Blocked by:
* Blocking:
* Updates:
** 1.29-rc.1 is out, hopefully release out this week (Friday if nothing
comes up)
=== Research ===
* Blocked by:
* Blocking:
* Updates:
=== Scoring Platform ===
* Blocked by:
* Blocking:
* Updates:
** Thank you to the people who helped with the June 23 outage, especially
AKosiaris and SSastry!
***https://wikitech.wikimedia.org/wiki/Incident_documentation/20170623-ORES
***
https://github.com/wiki-ai/revscoring/commit/ab11dc4c30e8b9824949b05b7d1200…
***
https://github.com/wiki-ai/revscoring/commit/c8d5c3a863059b83647e739fdcb8ce…
***
https://github.com/wiki-ai/ores/commit/2657d8a8ab4e8fa517922fcef07a598b01c5…
***https://phabricator.wikimedia.org/T168965 (Good demo in the last post)
**We're unblocked and unblocking on coordinating the Extension:ORES handoff
with Collaboration, https://phabricator.wikimedia.org/T167911. Looks like
we're within negotiating range of feature parity.
**Having a conversation about "ethical" AI, which is a major focus of our
work. I'll be mailing out soon to expand the circle to anyone interested
in that.
=== Search Platform ===
* Blocked by: none
* Blocking: none
* Updates:
** Continuing work on ML-assisted search ranking
** Working on Japanese & Vietnamese analyzers (
https://phabricator.wikimedia.org/T166731,
https://phabricator.wikimedia.org/T170423 )
** Wikidata search fixed remaining problems, waiting for deploy to do
production testing
** Database phrase search fixed (https://phabricator.wikimedia.org/T167798 )
** Reindexing wiki to enable archive search everywhere (
https://phabricator.wikimedia.org/T163235 )
** Reloading WDQS data for encoding fix (
https://phabricator.wikimedia.org/T131960 )
=== Security ===
* Blocked by: none
* Blocking: none
* Updates:
**Auditing labs views and sanitarium redaction
**Review reccomendation api service
** review *Vue.js library review*
** review AdvancedSearch extension
=== Services ===
* Blocked by: none
* Blocking: none
* Updates:
** Enabled automatic blacklisting of unrenderable articles for all
use-cases in Change-Propagation
** Woking on the next generation RESTBase storage model implementation
** FYI: Grafana alerting capabilities:
https://wikitech.wikimedia.org/wiki/Grafana.wikimedia.org#Alerts_.28with_no…
=== Technical Operations === (apologies for not being around)
* Blocked by: none
* Blocking: none
* Updates:
** Services running in Kubernetes Pods will have different network policies
than the status quo. Feel free to comment in
https://phabricator.wikimedia.org/T170111
** WP Zero users blocked from accessing media content due to abuse
https://phabricator.wikimedia.org/T168142
** RCStream will be disabled this week.
** Recommendation API to e deployed this week
https://phabricator.wikimedia.org/T148129
** Due to a spam attack some people may have been unsubscribed from
ops(a)wikimedia.org. If you care about that content, please make sure you are
subscribed
=== Analytics ===
* Released a new snapshot of mediawiki_history today, it has cummulative
revision counts! See the announcement on analytics-l
* Deploying an alpha version of the new Wikistats 2.0 UI working with real
data
* Lots of work on purging data from EventLogging according to our data
retention guidelines and privacy policies
* Preparing for the Kafka 0.10 upgrade
* Getting ready to decommission stat1002 and stat1003 and replace them with
new boxes
* Joseph is on parental leave until the end of July
* Dan is going on parental leave at some point within the next few weeks
== Wikidata ==
* Blocked by:
* Blocking:
* Updates:
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
Hello all,
I failed to send this notice out before when the change was made; mea
culpa.
Background:
* We start a new 1.XX-wmf.XX series after each MW 'tarball' release. For
example right now we're in the 1.30-wmf.XX series now that 1.29 is
nearing release.
The change:
* Instead of only incrementing the wmf.XX portion when a new branch is
actually deployed to Wikimedia production servers, we will increment
that number each week regardless.
** For example, last week we did not push a new branch out to production
due to the short work week. That week would have been 1.30-wmf.8. We
thus skipped wmf.8 and are now on wmf.9 this week.
Why?
We hope to make the creation of the weekly deployment branch
(1.XX-wmf.XX) automatic in the near future. This will allow us to put
that on a cron and not worry about special cases (another special case
being when we hold back the train due to a bad regression). This (every
week gets a wmf.XX number) should simplify logic in many places.
Thanks!
Greg on behalf of the Release Engineering team
PS: Yeah, we could have just gone with ISO week numbers, but we didn't
want to change too much in the version string to reduce the chance of
breaking too many other tools.
PPS: And yes, my failure to pre-announce this instead of post-announce
caused at least the ReleaseTaggerBot to break this week. Mea cupla.
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| Release Team Manager A18D 1138 8E47 FAC8 1C7D |
