Dear Wikipedia data/API user,
The WMF’s Engineering, Product and Partnerships teams are conducting a
short survey to help us understand how organizations are pulling and using
data from our projects. This information will inform future features and
improvements to our data tools and APIs.
We would appreciate a few minutes of your time. The link to the Survey
below will take you to a Google Form - there is no need to sign up to fill
out the survey. The survey should take no more than 10 minutes.
https://docs.google.com/forms/d/1yUrHzyLABN419RCDbzepjoRWCbaWYV4wbtbKPa95C4…
Thank you for your input and feedback!
Warm wishes,
Sylvia
PS-- Apologies for the cross posting, you might see this note on a couple
of other lists.
--
*Sylvia Ventura **Strategic Partnerships **Wikimedia Foundation **+1 (415)
839 6885 x6788 *
Hi,
Recently in the discussions about PHP version support, there have been
some brief tangents about whether MediaWiki should continue to have LTS
releases and what exactly those releases should entail. I'm writing this
to document the current state of LTS in MediaWiki, explain why I use the
LTS release, and what I'd like to see going forward.
The status quo is that we have an LTS release every 4 versions/2 years
(1.19, 1.23, 1.27, etc.). It receives bug and security fixes for 3
years, so people have a year to upgrade from one LTS to the next.
There's also some other stuff on-wiki[1] about special release notes
handling.
>From a quick perusal of the 1.23 release notes[2], nearly all of the
backports are security related, with a few major bug fixes (e.g. fixing
InstantCommons). I'd make a quick guess that there plenty of bug fixes
that have happened to master since the branchpoint that could be easily
backported, but aren't.
All of the non-dev wikis (and wiki farms!) that I help run currently run
1.23.x. My main reason for doing so is that upgrading major versions
usually takes a few hours for each wiki. They have custom (often
non-published) code or non-Wikimedia deployed extensions that typically
break and require some kind of fixing. Then there's schema changes,
JS/CSS changes, etc. It adds up to something that I don't want to do
every 6 months. (Maybe it would be easier if I did it every 6 months?
I'm not sure.)
Really all I want is a longer support for some arbitrary predetermined
version with regards to security issues so I have to upgrade major
versions less often. :-)
I think LTS is a significant factor to consider when deciding our PHP
version requirements as there is a huge difference between saying we're
going to support PHP 5.4 (or whatever) for 1 year versus 3 years.
I think it would be helpful if other people who use LTS could share
their motivations for doing so, and if the release/security teams could
share what issues make LTS release support problematic or difficult (a
few things have been mentioned on IRC, but I'll let those people speak
for themselves).
[1] https://www.mediawiki.org/wiki/Version_lifecycle#Release_policy
[2]
https://phabricator.wikimedia.org/diffusion/MW/browse/REL1_23/RELEASE-NOTES…
Thanks,
-- Legoktm
In last week's RFC meeting, it was proposed that we require PHP 5.5
for MediaWiki core git master (to be 1.27.0), and this was approved
without objections.
After the meeting, Mark Clements (HappyDog), a long-term valued
contributor to the project, expressed strenuous objections to this
decision on the ticket: <https://phabricator.wikimedia.org/T118932>
On the ticket, I proposed a process whereby the RFC will be reopened
for review if any existing Phabricator user will second the motion. If
you do object, please register your objection on Phabricator.
In the meantime, please do not merge any changes which require PHP 5.4+.
-- Tim Starling
Hi all!
If you are interested in code quality and you are attending the Dev Summit,
please have a look at <https://phabricator.wikimedia.org/T119032>.
I would appreciate any input on which additional sessions you think are
important, or how you would prioritize and group them. Any session proposed at
<https://phabricator.wikimedia.org/tag/wikimedia-developer-summit-2016/> is fair
game, but they should fit into the "code quality" topic somehow (other proposals
will be discussed elsewhere).
Context:
The ArchCom is currently in the process of figuring sorting through the list of
proposed sessions, and trying to prioritize and group them. To this end, we have
identified 5 broad topic areas ("Content Format", "Access and APIs",
"Collaboration", "Software Engineering", and "User Interface" - see T119018 for
an overview).
My job is now to figure out which sessions we want in the "Software Engineering"
(aka "code quality") part of the event. I have started to do this at
<https://phabricator.wikimedia.org/T119032>. If you have any thoughts on how
these sessions should be prioritized or grouped, or what is missing, please comment.
Thanks,
Daniel
https://www.mediawiki.org/wiki/Scrum_of_scrums/2015-12-2
= Product =
== Discovery ==
* Language test finished, unfortunately not much user improvement found
* Cirrus logs now available at wmf_raw.CirrusSearchRequestSet on hive
* need ops help to finish https://phabricator.wikimedia.org/T110262
* not blocking anything
=== Maps & Graphs ===
* Vega 2.0 migration is moving forward, service is up, interactive is next
* GeoHack for ruwiki moved to the new maps - 0.5 million tiles a day, no
server effect
* BLOCKER: Services - what's the status of POST storage?
** https://phabricator.wikimedia.org/T105975 (resolved)?
== Reading ==
=== Generally ===
* Q3 planning
* Strategic pillars set (web global south, Android social interactive
reading / customized education, iOS iterative improvements for people who
love to read), testing phases over coming quarter (notable: divert 1/1000
unauthenticated desktop users to mobile web, cookies-based mechanism)
=== Android and Mobile Content Service ===
* Content Service is slated to start rolling out Monday.
* Map functionality to be promoted from beta to prod next release.
* Moving long running tests from Gerrit triggers to periodic.
* Moving more content from WebView to native components.
* Wiktionary service endpoint and client implementations in progress.
=== Web ===
* Cards security review (Gergo will yield GPGMail)
* 5% mobile web beta CTA (call to action?) to be suppressed
* Browser test failures now only go to qa-alerts
* Potential image progressive loading / not srcset-based
=== iOS ===
* Gearing up for Beta w/ non WMF users
* Universal Links work is "blocked" on getting a staging environment for
prototyping Universal Links behavior & user flows
=== Reading Infrastructure ===
* Block: Need replies to the replies on the SessionManager security review.
** Reply to https://phabricator.wikimedia.org/T90895#1802168 would be nice
too.
* Block: ApiSandbox is still blocked on whoever owns oojs-ui for
https://phabricator.wikimedia.org/T91148 - ACTION! let's figure out an
alternative approach (James F suggests separate component in mw.Widgets)
* Two questions for Services or Multimedia or whoever:
https://phabricator.wikimedia.org/T66214#1842439 - ACTION! email thread,
keep discussion on task, though
== Editing ==
=== Collaboration ===
* Cross-wiki Echo notification work is still progressing well.
* Flow external store patch is merged. Collaboration team needs to do
follow-up
* We need to follow up with DBA at some point about the Flow artificial
primary keys.
* Flow dumps code are merged, now we need to work with ops to get them
running and published ( https://phabricator.wikimedia.org/T119511 )
* Following up on LQT->Flow conversion for ptwikibooks.
* Previous approach to cross-DC Flow support was too slow. We're going to
try a different approach.
* Flow anti-spam integration also in-progress, particularly Nuke.
=== Language ===
* cxserver service-runner patch needs review:
https://gerrit.wikimedia.org/r/#/c/244145/(service team)
:* yup, am aware of it, will do (Marko)
:* note: we'll need to coordinate for merging and deploying this, as this
change will need an op/puppet change as well
=== Multimedia ===
* Found an unfortunate i18n OOUI bug for people using mw.Widgets but not
VE. Fixed in master, going to ask for a backport.
https://phabricator.wikimedia.org/T119984
=== Parsing ===
* Added async support to parsoid's html -> wikitext serialization --
required to support serialization scenarios that require template data
lookup or mediawiki API calls
* Work in progress to support native tag extensions in Parsoid (
https://phabricator.wikimedia.org/T55874 )
* Using deployment lull to do some code cleanup and refactoring.
* FY16Q3 draft goals published @ https://phabricator.wikimedia.org/T119088
=== VisualEditor ===
* Big IME support change likely to land in master today. Shouldn't impact
anyone inc. Collaboration's Flow usage. (unless they use IMEs).
== Fundraising Tech ==
* Campaign is up, making big bucks
* Nothing on fire so far
* Minor bugfixes and backup systems buildout
= Technology =
== Services ==
* Blocking: none
* Blocked: none
:* Kafka HW
* High-volume API endpoints
:* /page/summary is out in prod - https://phabricator.wikimedia.org/T117082
::* Reading team - please finalise
https://gerrit.wikimedia.org/r/#/c/252276/
:* other endpoints - https://phabricator.wikimedia.org/T115876
::* tell us your needs!
* EventBus
:* making progress in tandem with Analytics
:* finalising the REST proxy service's API
:* basic event schemas in place
:* MW event production
::* https://gerrit.wikimedia.org/r/#/c/254086/
::* should see the light of day soon: Beta deploy patch -
https://gerrit.wikimedia.org/r/#/c/256418/
* Misc
:* AQS
::* first attempt at deploying via Scap3 in Beta happening today
::* Analytics team: we will need some fake data for BetaCluster domains to
have a fully-working AQS instance in Beta, cf.
https://phabricator.wikimedia.org/T116206
:* RESTBase
::* revamp of config and module specifications in progress -
https://github.com/wikimedia/restbase/pull/425
== Technical Operations ==
* Apologies, will not be making it to the meeting
* Blocking: none
* Blocked: none
* Updates:
* Yubikey for all of ops moving on
* LDAP migration moving on
* Migration of monitoring to shinken ongoing
== Security ==
* Reviews: Session/Auth Manger still in progress, Cards and Thumbor this
week.
== Release Engineering ==
* *Blocking*: (none)
* *Blocked by*: (none)
* *Updates*:
** Deployment tooling implementation continues
*** https://phabricator.wikimedia.org/project/view/1449/
*** AQS deployment to Beta Cluster this morning
** MediaWiki 1.26 released
==Research==
Nothing to report
Hello,
Wikimedia Release Engineering now tracks the time it takes to get a
change merged for MediaWiki core via:
https://grafana.wikimedia.org/dashboard/db/releng-kpis
The idea is to make sure we do not make it worse, but that is also a
good intensive to speed up the merge delay.
Beside CI overhead, most of the delay was from mediawiki-phpunit-zend
which runs the whole PHPUnit tests under Zend PHP 5.3 and takes 10+
minutes. Earlier today, I have split the job in two part and the merges
should happens ~ 3 minutes earlier.
>From now on:
mediawiki-phpunit-zend , excludes any tests flagged with @groups
ParserTests
mediawiki-phpunit-parsertests-zend runs the PHPUnit testsuite
'parsertests' which should load any .txt tests registered.
By running them in parallel, we regain some velocity.
I haven't split the HHVM version seems it looked fast enough compared to
the phpcs job. Will monitor a bit more and adjust as needed.
Reference:
https://phabricator.wikimedia.org/T114314
--
Antoine "hashar" Musso
Also if you are a dumps user or have thoughts about how you would redo
them from scratch, get your ideas in now. We're not waiting for the
Dev Summit to get the work started. See
https://phabricator.wikimedia.org/T114019 for details, especially the
document linked at the end of the task description under 'FOR CURRENT
DISCUSSION'.
We need: comments on the strawman proposed model in the document
mentioned above; proposals for code we can reuse for any of those
pieces, especially the job queue/management piece. (Celery? Something
else?) What should the object store be based on, if we have one? Is
Ceph a dead end? Will Swift be deadly slow or should we use it since we
have it in house already? Etc. All comments on the ticket please so we
have them all in one place.
Expect that most of the code written here at WMF will be in python
unless someone else volunteers to write some. Anyone interested in
doing some of the development can step right up too of course.
Please forward this on to other fora as appropriate.
Ariel