Hi all,
Gerrit decided to crap itself this morning while doing garbage collection
on the repositories. As a result, you may have some issues when fetching
the latest objects from the repo. Right now MediaWiki core seems to be the
only repo noticeably affected, but I'm currently checking into the status
for all repositories.
Fresh clones are not affected. If you end up hitting a problem in fetching,
the following workaround should get you back in shape:
cp .git/config .git/config.backup
git remote remove origin
mv .git/config.backup .git/config
git fetch
Investigation is ongoing. If you hit any other repos than MW that seem to
be having issues, please chime in on
https://phabricator.wikimedia.org/T151676
Have a good weekend!
-Chad
Hi all!
I thought I'd resume the practice of sending a summary of the Architecture
Committee meeting to this list.
The minutes of Wedensday's meeting can be fond at
<https://www.mediawiki.org/wiki/Architecture_committee/2016-11-23>.
See also the ArchCom status page at
<https://www.mediawiki.org/wiki/Architecture_committee/Status> and the RFC board
<https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>.
Here are the minutes, for your convenience:
* ArchCom soon to meet Victoria (perhaps Dec 7)
* Robla resigns from ArchCom in the wake of leaving WMF. Daniel is doing the
chairing for now.
* Ongoing activity:
** Brion: mobile video
** Gabriel: thumbnail API (Gabriel), ServiceWorkers for the discovery portal
** Tim: replacing tidy
** Timo: oldime table refactoring
** Daniel: experimenting with the Interwiki overhaul, thinking about db schema
optimization
* Talking with Faidon about ArchCom scope, progess, and impact. Some key points
that came up:
** ArchCom should be more arcive in providing visions and guideleines, such as
high level architecture documents.
** ArchCom should be involved in more of the ongoing developemnt at WMF. People
tend to ask ArchCom only if they have to.
** ArchCom should not only get active when asked. ArchCom scruteny should not be
limited to RFC discussions.
** For RFC discussions, we too often focus on easy-to-decide topics with limited
impact
** Lack of ops perspective in ArchCom
** A clearer scope / charter would be helpful
** Role of ArchCom needs to be discussed with our new CTO
** ArchCom has no clear connection to WMF org chart, and no power in resourcing.
Platforms (mostly MediaWiki itself) are not well represented in the org chart.
* Up for next week’s IRC discussion: ''Per-language URLs for multilingual wiki
pages'' (T114662).
--
Daniel Kinzler
Senior Software Developer
Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.
Please comment on whether to approve the "Conflict of interest" section
of the draft Code of conduct for technical spaces. This section did not
reach not reach consensus earlier, but changes were made seeking to
address the previous concerns.
You can find the section at
https://www.mediawiki.org/w/index.php?title=Code_of_Conduct/Draft&oldid=229…
You can comment at
https://www.mediawiki.org/wiki/Talk:Code_of_Conduct/Draft#Finalize_new_vers…
.
A position and brief comment is fine.
You can also send private feedback to conduct-discussion(a)wikimedia.org .
I really appreciate your participation.
Thanks again,
Matt Flaschen
Hi all,
I would like to propose that we separate the bot right into bot and
boteditinterface, where the bot right only allows you to mark edits as
bot edits for non-mediawiki namespace edits and you would need the
boteditinterface right to mark any mediawiki namespace edits as a bot
edit. Furthermore, I would propose that nobody on Wikimedia wikis ever
get the boteditinterface right (But if I end up having to cave on that
requirement, at least we could restrict it to people who really need
it)
The idea being that any potential edit to site javascript should be
highly scrutinized and shown on the RC by default. Additionally, most
bots do not make edits to the MediaWiki namespace so this should not
cause an undue burden. Those that do make edits, do not make a super
large number. This proposal does not include edits made by
centralnotice extension, which will continue to be marked bot.
If you think this is a horrible idea, please object now :) For more
information see https://phabricator.wikimedia.org/T151408 . If there
are no major objections from the wikitech-l community, I also plan to
notify the owners of bots who make edits to mediawiki namespace.
Thanks,
Brian.
I was reading http://stateofjs.com/2016/introduction/#sections and
could not avoid noticing that the frameworks or technologies we use
are not among the most popular or most liked among the participants of
this survey.
Examples:
* Frontend frameworks: We use jQuery and OOjs UI. The latter does not
appear at all in the list, jQuery is not in the top ten. This question
might be biased though on what people perceive as a framework.
* Testing framework: We mostly use qUnit, Cucumber, Selenium. Of these
only Cucumber appears in the top 6 and it has very low satisfaction
(people who have used do not like it).
* CSS tools: We use plain CSS and Less. Less has considerable lower
satisfaction than SASS/SCSS and is less popular.
* Build tools: We don't use these in core to my knowledge, but many
extensions seem to use Grunt for running linting tools. Again, Grunt
has very low satisfaction compared to other tools.
It is natural, that as large and complex project we do not jump to the
latest cool thing. I am not advocating to change tools that work well
for us, but I don't remember seeing a public discussion whether they
work well or not. Though, I am seeing some changes, for example
jscs+jshint being replaced with eslint.
We could possibly go faster or write better software with better tools
(of course this would need a careful evaluation). And while doing that
we could perhaps lower the barrier for new developers by using
something they already know. The topic of how to attract new
developers to our movement has been popular lately (for example [1]).
[1] https://phabricator.wikimedia.org/T148911
----
For me one pain point is automated testing of JavaScript code. It
seems that testing frameworks, development practices and the way code
is written could all be improved to make automated testing easier.
Would there be interest in sharing comments how you do this and does
what you do work well for you?
-Niklas
https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-11-23
= 2016-11-23 =
== Technology ==
=== Analytics ===
* Blockers: none
* on track for quaterly goals
* main project about edit data (mediawiki edit history reconstruction)
progressing,
* we are now calculating standard edit metrics for all wikis since the
beginning of time using denormalized edit history:
https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Denormalized_edit_h…
experimental dataset on pivot: https://goo.gl/XAMVsh
* working on productionizing infrasructure for event streams
* waiting for hardware for pageview API
* owning now statsv together with ops (utility that can consume kafka data
and report to graphana)
* Thanking discovery for contributing to our metric reporting tools
Upcoming:
Start design work to revamp information architecture of
http://stats.wikimedia.org
=== Performance ===
Not blocking, not blocked
* thanks everyone who attended the active/active DC meeting after I flagged
it here, it has helped getting the ball rolling on two blockers
* hidden tabs confirmed as messing with timing data, now excluded from perf
metrics
* investigating little-known legacy features in mediawiki thumbnailing to
decide whether we continue supporting them on Thumbor (302 redirects)
* second view tests added for firefox and IE in WebPageTest (was previously
only looking at Chrome)
* still active on thumbnail URL/API RFC discussion
* briefly discussed witth multimedia team setting up Thumbor for them to
leverage in their ImageTweaks extension
=== Security ===
* Security Reviews
* Linter review complete
* LoginNotify schedule for this week
* Continuing work on wiki account compromise remediation (T150554)
* Assistance needed -- e-mail to engineering@ is forthcoming with request
=== Services ===
* Blockers: none
* Updates:
** PDF render service deployed in codfw, eqiad and public exposure next
week
** New version of service-template-node: ES6 and ESLint are coming
=== Technical Operations ===
* '''Blockers'''
** IOS native app
*** Requesting timeline for Wikipedia iOS app requesting 0px thumbs:
https://phabricator.wikimedia.org/T147648https://phabricator.wikimedia.org/T151078
iOS 5.3.0 was shipped last week
** Performance ?
*** MW fix to return 400 on 0px thumbs
https://phabricator.wikimedia.org/T147784
* '''Blocking'''
** None
* Updates
** jobqueue woes https://phabricator.wikimedia.org/T151196
** kubernetes/calico work ongoing, goal on track
** dropping varnish 3 compatibility code from our puppet repos
** labsdb goals on track as well
=== Release Engineering ===
* Blocking
**
* Blocked
**
* Updates
** Mediawiki 1.28 tarball release this week!
== Product ==
=== Reading ===
==== Mobile Content Service (MCS) ====
* Board: https://phabricator.wikimedia.org/tag/mobile-content-service/
* Added announcements feed endpoint (public now). More info and request URL
at
https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/RESTBase_services_for_ap…
:
==== Android native app ====
* Last week:
** Continuing Q2 goals for Wikidata descriptions
** New fundraising announcement explore feed card in progress
** Now building against Android Nougat 7.1 API 25
** Fixing login and editing issues
** Lots of unit tests
* Next week (https://phabricator.wikimedia.org/project/view/2352/):
** More Q2 goals for Wikidata descriptions (tutorial and polish)
==== iOS Native App ====
* Last Week:
** Shipped 5.3.0 (In the news notifications & feed content, MCS backed
feed, language variant support, other bug fixes and enhancements)
https://phabricator.wikimedia.org/project/view/2220/
** Added announcement card to the feed (for user research and fundraising)
** Started update of data layer to fix issue with data access &
modification from widgets and notifications
** Started dynamic font size updates to the app
* This week: https://phabricator.wikimedia.org/project/view/2357/
** Finishing data layer update
** Continuing dynamic font size updates
** Other minor bug fixes for 5.3.1
==== Web ====
* Current sprint:
https://phabricator.wikimedia.org/tag/reading-web-sprint-86-%F0%9F%94%AA%F0…
** Stopping HoverCards A/B tests from Russian and Italian wikis
** New readers work
** Make PageImages return the image in the lead section
** MobileFrontend tech debt
** Trending service
** Hovercards rewrite
* Next week: probably the same stuff as the current week.
==== Reading Infrastructure ====
* not blocking
* Still looking for reviews on the API error/warning i18n patches
** https://gerrit.wikimedia.org/r/#/c/321402/ - Improve handling of Message
objects as Message parameters
** https://gerrit.wikimedia.org/r/#/c/321404/ - Add Message::listParam()
** https://gerrit.wikimedia.org/r/#/c/321405/ - Fix MediaTransformError
message handling
** *https://gerrit.wikimedia.org/r/#/c/321406/*
<https://gerrit.wikimedia.org/r/#/c/321406/> - API: i18n for warnings and
errors
** The extension patches depended on by that change are next in importance.
These are for OAuth, TitleBlacklist, GlobalBlocking, Translate, and
ConfirmEdit.
** All other WMF-deployed extensions affected by this change have patches
too, see *https://gerrit.wikimedia.org/r/#/q/topic:api-error-i18n/T47843*
<https://gerrit.wikimedia.org/r/#/q/topic:api-error-i18n/T47843>.
Non-WMF-deployed extensions are (mostly) not touched at this time, the
worst that should happen to them is wfDeprecated warnings eventually.
* https://gerrit.wikimedia.org/r/#/c/312865/ is blocked on review by
Security
=== Community Tech ===
* Not blocking
* Blocker: Need a security review for
https://phabricator.wikimedia.org/T150832 to proceed with exposing a couple
of table views on tool labs db
* Community Wishlist survey proposal phase over.
https://meta.wikimedia.org/wiki/2016_Community_Wishlist_Survey
* Did bug-fixing for Copypatrol (plagiarism detection tool) and launched
French version: https://tools.wmflabs.org/copypatrol/fr
* Ongoing RfC about changing default collation on Meta:
https://meta.wikimedia.org/wiki/Requests_for_comment/Switch_default_categor…
* Ongoing RfC about abandoned labs tools takeover:
https://meta.wikimedia.org/wiki/Requests_for_comment/Abandoned_Labs_tools
* Ongoing work with programs dashboard
=== Discovery ===
* BM25 scoring enabled on 10 larges wikis
* Discovery mission & roadmap presentation:
https://docs.google.com/presentation/d/1ctlqdLA__0OxDuO7mJEIDLP-xt9a7E4jv4I…
* Load-testing crosswiki searching backend code
* Portal updates per-language article count stats, dewiki joins the
lucrative 2M+ club :)
=== Editing ===
==== Language ====
* Blocked: T150512: WikiBase Repo tests failing with UsageException
** This is making it difficult to merge Translate patches. Issue seems to
be in database clearing in tests. QA/RelEng?
==== Collaboration ====
* Blocked: None
* Blocking: None
* Updates
** No deployments this week. Ongoing work on:
*** Mobile support for left nav of Special:Notifications
*** RecentChanges filters and filter framework for Edit Review Improvements
=== Fundraising Tech ===
* Big English fundraiser starts next week!
** repeating Greg's emailed plea:
https://lists.wikimedia.org/pipermail/engineering/2016-November/000331.html
** Stuff that could impact the fundraiser: GeoIP, ResourceLoader,
MessageCache, EventLogging, Hive webrequest tables
* CentralNotice: reviewing Aaron Schultz's latest MessageCache patch:
https://gerrit.wikimedia.org/r/#/c/318489
** We want to understand it really well before we deploy anything that
could affect banners
** If anyone with deep knowledge of MessageCache (Aaron, Gilles?) has time
for a quick video chat, Andrew Green has a few questions
** As always, more scrutiny and comments are welcome.
* Deployed Nirzar's mobile CSS fixes, looking great so far
* Minor caching optimizations for CiviCRM jobs
Hello everyone,
I have proposed a 2017 Dev Summit session titled "Improved editability,
tooling, reasoning, and performance by adopting DOM-based semantics for
wikitext" [1].
The TL:DR; summary is to treat a wikitext article as made up of document
fragments that are composed together instead of a concatenation of
strings which may (but often does not) yield well-formed HTML markup.
Rather than repeat what is already there on the task, I'll mention a
couple of benefits if you are an editor:
1. If you are a template editor, and you edit a template, it would be
much more efficient to update all affected articles. For the common use
case, you would simply replace the old output in an article with the new
output without having to reparse the article.
2. It is possible to think of edit tooling that lets you edit the
document at much finer granularities than sections without stepping on
each other's toes.
There are more details in a draft note [2]. Please don't be alarmed by
the "wikitext 2.0" term there. That is an umbrella term that we've been
throwing around in the Parsing Team for a bunch of related but different
proposals related to wikitext syntax and semantics. This phab task and
the wiki page is one proposal that concerns itself primarily with
semantics of wikitext markup and less about syntax. The only syntactic
feature considered is the heredoc-style syntax for some transclusions
[3] which would immensely benefit this approach.
Please comment on the task with your thoughts, ideas, concerns,
approval, criticism, or award tokens or subscribe to indicate interest. :-)
Thanks,
Subbu.
[1] https://phabricator.wikimedia.org/T149282
[2] https://www.mediawiki.org/wiki/Parsing/Notes/Wikitext_2.0
[3] https://phabricator.wikimedia.org/T114432
FYI to those who use these hosts during SWAT deploys.
----- Forwarded message from Giuseppe Lavagetto <glavagetto(a)wikimedia.org> -----
> Date: Tue, 22 Nov 2016 14:53:50 +0100
> From: Giuseppe Lavagetto <glavagetto(a)wikimedia.org>
> To: Operations Engineers <ops(a)lists.wikimedia.org>
> Subject: [Ops] mw1017, mw1099 are gone
>
> Hi all,
>
> due to an unfortunate chain of events, mw1017 and mw1099, the MediaWiki
> test appservers in eqiad, had to be decommissioned this morning. Their
> substitutes are mwdebug1001 and mwdebug1002
>
> So, not to break the deployers workflow, what is happening is what follows:
>
> - when you select mw1017 from the browser extension, the request will go do
> mwdebug1001
> - when you select mw1099, the request will go to mwdebug1002
>
> the extensions will be updated as soon as possible (in a way that will
> allow us an easy change of the server list without any extension release),
> and I am in the process of upgrading the documentation on wikitech.
>
> I apologize for the inconvenience,
>
> Giuseppe
> --
> Giuseppe Lavagetto, Ph.d.
> Senior Technical Operations Engineer, Wikimedia Foundation
> _______________________________________________
> Ops mailing list
> Ops(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/ops
----- End forwarded message -----
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| Release Team Manager A18D 1138 8E47 FAC8 1C7D |