*TL;DR:* Stuff is changing. If you notice problems with session handling in
1.27.0-wmf.11, file a bug in Phabricator and CC me (Anomie) and Gergő (Tgr).
MediaWiki's authentication layer is getting a major overhaul! This will
allow us to do cool stuff like 2-factor authentication without the 2-factor
extension having to be redone for LdapAuth/CentralAuth/every-other-auth,
third-party login (e.g. Google, Facebook) without hacking around everything
with crazy hooks, and supporting more than just one authentication method
at a time.
Broadly speaking, this change involves the addition of two new things:
SessionManager (already merged, being deployed with 1.27.0-wmf.11) and
AuthManager (hopefully coming by the end of February).
- SessionManager replaces the use of PHP's $_SESSION and session_*()
functions and the UserLoadFromSession and UserSetCookies hooks with a more
extensible mechanism.
- AuthManager allows for independent pluggable authentication modules,
multiple authentication methods, and removes the assumption that all logins
can be handled with a one-page form having "username" and "password"
fields. No more needing to adjust every authentication extension to know
about every other authentication extension it might have to cooperate with.
However, the changes we're making do mean that various things are going to
need updating.
== For users ==
You shouldn't notice any changes due to SessionManager. If you do see
problems related to session handling in 1.27.0-wmf.11, file a bug in
Phabricator and CC me (Anomie) and Gergő (Tgr).
Once AuthManager comes around, the most visible change will be that the
login form will probably be a bit different, and if your browser remembers
your login information for you then you might have to re-enter it.
== For bots ==
You shouldn't notice any changes due to SessionManager, but do make sure
your code is handling cookies normally instead of using the deprecated
return values from action=login to construct cookies manually.[1] If you
are handling cookies properly and see problems related to session handling
in 1.27.0-wmf.11, file a bug in Phabricator and CC me (Anomie) and Gergő
(Tgr).
For AuthManager, the new features mean that unattended login might no
longer work since the login flow will now natively support user
interaction: the account might have 2-factor enabled, or might need a
password reset, or some other thing that requires user interaction. We've
created two ways to work around this:
- If possible, switch to OAuth. This week (1.27.0-wmf.10) "owner-only"
consumers are being rolled out to make this easier for bot operators: log
into your bot account, go to
https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration/propose,
and create a consumer with the "This consumer is for use only by MyBotName"
checkbox checked.The consumer will be approved for use immediately, no
waiting or trying to find someone who can approve the consumer for you.
Owner-only consumers also don't tag every edit, since all the edits will be
from the one account anyway.
- If you need to continue using the existing action=login, next week
(1.27.0-wmf.11) we're rolling out Bot Passwords. This is something like
OAuth-lite, or Google's application passwords: go to Special:BotPasswords,
set one up, and then use new bot-password username and password to login as
you've always done (no code changes, just update your bot's configuration).
It's already live in Beta Labs if you want to test it out.
Login with the "main" account password will continue to work until
AuthManager is deployed, and might continue to work after that too (as long
as nothing requires user interaction).
For bots that run on third-party wikis, Bot Passwords are in core and are
enabled by default, but it's possible a wiki could disable them. OAuth is
an extension.
[1]:
https://lists.wikimedia.org/pipermail/mediawiki-api-announce/2015-December/…
== For user scripts and gadgets ==
Since user scripts and gadgets rely on the user already being logged in,
you shouldn't notice any changes.
== For core and extension developers ==
There's some other useful stuff that has already been merged, written as
part of the groundwork for this project:
- User::newSystemUser(), which replaces directly screwing around with
accounts when an extension needs a user for logged actions.
- Deprecation or removal of much of the password handling logic from the
User class. In particular User::getPassword() no longer works.
- CentralIdLookup, a generic mechanism for looking up a "central" ID for
cross-wiki features instead of everything needing that functionality having
to depend on CentralAuth (and on anything else that might implement global
accounts). If you have some sort of cross-wiki login extension that isn't
CentralAuth, you'll want to implement this class.
If you're dealing with session data,
- Don't use $_SESSION. Instead, use $request->getSession() or
SessionManager::getGlobalSession() to fetch a Session object and access the
data through it. Or continue to use the $request->getSessionData() and
$request->setSessionData() methods.
- Don't use session_id() to test if a session is active. Instead, fetch
a Session object as above and call ->isPersistent().
- Don't use wfSetupSession() to make sure a session is active. Instead,
fetch a Session object as above and call ->persist().
- Don't use the other PHP session_*() functions either.
If you're implementing the UserLoadFromSession hook to bypass the usual
cookie-based session handling (e.g. OAuth, SSLClientAuthentication, or
CentralAuth's centralauthtoken), you'll now want to implement a subclass of
the new SessionProvider class and add it to $wgSessionProviders. OTOH, if
you're using UserLoadFromSession because you're trying to add additional
security checks (e.g. SecureSessions), you'll probably want to look into
the new SessionMetadata and SessionCheckInfo hooks. And if you're using
UserLoadFromSession because you're doing some hack around MediaWiki's
existing login form, you'll want to wait for AuthManager. The same goes for
UserSetCookies.
If you maintain an authentication-related extension that does a login of
some sort, you'll eventually want to be implementing one of AuthManager's
AuthenticationProviders. In particular, an extension implementing
AuthPlugin now will want to implement a PrimaryAuthenticationProvider in
the future. If you're using the hooks that mess with the login form, such
as UserCreateForm or UserLoginForm, you'll likely also be wanting to
implement an AuthenticationProvider of some sort. Other hooks called from
LoginForm might also need attention. You can see the work-in-progress patch
at https://gerrit.wikimedia.org/r/#/c/195297/.
A more detailed overview of the new classes is available at
https://www.mediawiki.org/wiki/User:Anomie/SessionManager_and_AuthManager.
If you have more questions, feel free to ask (email works particularly
well).
== For apps that log in ==
Apps shouldn't notice any changes from SessionManager. If you do see
problems related to session handling in 1.27.0-wmf.11, file a bug in
Phabricator and CC me (Anomie) and Gergő (Tgr).
Once AuthManager comes out, though, you'll need to use the new API
action=clientlogin and action=query&meta=authmanagerinfo which will provide
the information necessary to construct a fully-functional login UI. More
details will be provided once I finish writing the new code ;) but be aware
that there's the possibility that you'll have to redirect to a third-party
website in there (e.g. if a "login with your Google account" extension
happens to be installed).
Do not be tempted to have your app somehow use bot passwords, that's not a
supported use case for that feature. A normal (not owner-only) OAuth
consumer would be fine.
== For operators of third-party wikis ==
If you're using $wgSessionsInObjectCache = false (which was the default
before SessionManager), note that that configuration is no longer
supported. Make sure $wgSessionCacheType is set to something that will
function well in your environment.
Other than that, things should mostly just work on upgrade with both
SessionManager and (in the future) AuthManager. If you use authentication-
or session-related extensions that aren't deployed on WMF wikis, however,
you may want to check with the maintainers of those extensions.
If you're using the OAuth extension, you should also be aware that its
$wgMWOAuthGrantPermissions and $wgMWOAuthGrantPermissionGroups settings are
replaced by the new core settings $wgGrantPermissions and
$wgGrantPermissionGroups.
Please report bugs related to these changes in Phabricator, and CC me
(Anomie) and Gergő (Tgr).
--
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-01-13
= 2015-01-13 =
== Reading ==
=== Android ===
*v2.1.137 beta published. Includes a new native article toolbar and lower
memory usage.
=== Reading Infrastructure ===
* Nothing blocked this week.
* SessionManager and bot passwords should be going out next week.
== Discovery ==
* Working on load test for Dallas cluster
* Load spikes on weekend for ES getting worse, cause not clear yet,
investigating ( https://grafana.wikimedia.org/dashboard/db/elasticsearch )
** Did some brainstorming, discovered queries generating a lot of ES calls
to handle variants, with very low success rate. Will investigate removing
them.
* Finalizing data models for TextCat language detection library, security
review next
* Testing for upgrading WDQS to Blazegraph 2.0 once it's released
* No blockers
== Community Tech ==
* Working on assessment report for Community Wishlist Top 10 results:
https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey/Top_10/Statu…
** Would like feedback from Wikidata, Multimedia, Analytics, and
Collaboration before publishing
* Working on dead link rescuing bot - talking with Tool Labs and IA
* Waiting on feedback from Kunal on Gadgets 2.0
* Wrapping up work on PageAssessments extension:
https://www.mediawiki.org/wiki/Extension:PageAssessments
== Infrastructure ==
=== RelEng ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Differential migration discussion at WikiDev16 went really well, very
positive, no roadblocks placed
** Scap3 discussion at WikiDev16 also really positive, looking for more
early adopters
=== Technical Operations ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Started work on the new Quarter goals
=== Services ===
* moved to Node 4.2 - RESTBase, Mobile Content Service, Mathoid
:* in progress: Graphoid and Citoid
* working on enabling pre-generation for Mobile Content Service
=== Security ===
* Should get to Kartographer this week
* Several patches to deploy this week
* Schedule your reviews!
=== Fundraising Tech ===
* cleaning up after December
* updating CiviCRM extensions to use new core features
== Editing ==
=== VisualEditor ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Single Edit Tab work nearing completion. Planned to release to one wiki
next week if all goes well for validation before larger rollout.
** Note the OOUI breaking release from yesterday (in -wmf.11); no known
impacts.
=== Parsing ===
*Blocking: nothing
*Blocked on: nothing
*Updates
** Hoping to finish up (this week) puppetizing testing infrastructure on
ruthenium
** Coordinating with reading and services to reduce size of HTML payload.
=== Multimedia ===
*Blocking: nothing
*Blocked on: nothing
*Updates:
** Christmas A/B test for cross-wiki upload tool inconclusive. Not sure
what's next happening.
=== Language ===
*Blocking: nothing
*Blocked on: nothing
=== Collaboration ===
* Mostly Echo stuff. Cross-wiki notification stuff is getting closer.
* MediaWiki-Vagrant code for cross-wiki notifications is implemented and
almost merged. This will also facilitate other local multi-wiki testing,
e.g. Flow.
* Flow Nuke fix merged.
Hi everyone,
As many of you know (because you were here), we had the Wikimedia
Developer Summit 2016 last week. It was wonderful to see so many of
you! Rachel Farrand and Quim Gil ensured we had an excellent venue
and had a wonderful opportunity to work together, and the people here
had some great conversations. Valerie Aurora taught us a lot about
how to run effective meetings, which greatly increased the quality of
the discussions.
The conversations we started here are just that: a start. We covered
a lot of very important topics. Given the time and expense of
bringing everyone to one place, it would be a shame if the
conversations we started ended up needing to start over. Hence, it
was really important to have good notes, to make it easier to pick up
these conversations where we left off.
For many of the sessions, the notes made their way into the Phab task
associated with the meeting. The main WikiDev '16 landing page makes
it a bit easier to find the notes:
<https://www.mediawiki.org/wiki/WikiDev16>
The session leaders were very motivated to make sure we got something
out of the conversations they led, so they will be clarifying the next
steps on their respective sessions. It's certainly easier than it was
this time last week to find out what happened in each respective
session, but there's a lot of work to do. Please help us out!
Rachel sent out a survey to all of you who attended. Please fill this
out! It will help us figure out if we'd like to have another one like
this next year.
If you're interested in real time conversation about WikiDev '16,
either because you were there and want to follow up or you weren't
there and you want to learn what you missed, please attend the RFC
office hour in a few hours (Wednesday 22:00 UTC, 14:00 PST):
<https://phabricator.wikimedia.org/E134>
Rob
Hi everyone,
As many of you know (because you were here), we had the Wikimedia
Developer Summit 2016 last week. It was wonderful to see so many of
you! Rachel Farrand and Quim Gil ensured we had an excellent venue
and had a wonderful opportunity to work together, and the people here
had some great conversations. Valerie Aurora taught us a lot about
how to run effective meetings, which greatly increased the quality of
the discussions.
The conversations we started here are just that: a start. We covered
a lot of very important topics. Given the time and expense of
bringing everyone to one place, it would be a shame if the
conversations we started ended up needing to start over. Hence, it
was really important to have good notes, to make it easier to pick up
these conversations where we left off.
For many of the sessions, the notes made their way into the Phab task
associated with the meeting. The main WikiDev '16 landing page makes
it a bit easier to find the notes:
<https://www.mediawiki.org/wiki/WikiDev16>
The session leaders were very motivated to make sure we got something
out of the conversations they led, so they will be clarifying the next
steps on their respective sessions. It's certainly easier than it was
this time last week to find out what happened in each respective
session, but there's a lot of work to do. Please help us out!
Rachel sent out a survey to all of you who attended. Please fill this
out! It will help us figure out if we'd like to have another one like
this next year.
If you're interested in real time conversation about WikiDev '16,
either because you were there and want to follow up or you weren't
there and you want to learn what you missed, please attend the RFC
office hour in a few hours (Wednesday 22:00 UTC, 14:00 PST):
<https://phabricator.wikimedia.org/E134>
Rob
Hi,
I have been working on a project to improve the categorization of pictures
in the Upload to Commons Android app <
https://phabricator.wikimedia.org/T115101> as part of the Outreachy Dec '15
program, and I am happy to announce that Phase 1 of the project has been
implemented. :) The app should now suggest nearby categories when a picture
is uploaded.
Feedback on this feature would be greatly appreciated, so please feel free
to download the updated version of the app <
https://play.google.com/store/apps/details?id=fr.free.nrw.commons> and to
post feedback/issues on the GitHub page <
https://github.com/nicolas-raoul/apps-android-commons/issues/new>.
Please note that to use this new feature, you will need to have location
tagging enabled for your camera.
Thanks!
--
Regards,
Josephine
Hey all,
I released OOjs UI 0.15.0 today. It will be in MW from 1.27.0-wmf.11+, which
will be deployed to Wikimedia production in the regular train starting on
Tuesday 19 January. As there are a couple of nominal breaking changes and a
deprecation, please look carefully over them to determine if they affect
your code.
Breaking changes since last release:
- Drop Internet Explorer 8 support from JavaScript code (Ricordisamoa)
We have simplified and streamlined the code to drop support for Internet
Explorer 8 in the JavaScript code. This follows MediaWiki's dropping of
support for executing JavaScript on Internet Explorer 8 clients, as the
only significant user of the library and only cause of IE8 support. This
is thus a nominal breaking change in environments supported which should
not have any effect on developers or end-users in practice.
As part of this, we also deprecated the add/removeCaptureEventListener
methods, which existed only to support Internet Explorer 8. These will be
removed in a later release.
Note that the PHP code and related styling should continue to work
without
issue in Internet Explorer 8; if you have any issues, please raise a task
in Phabricator.
- Delete deprecated aliases 'picture' and 'insert' (Ed Sanders)
These two old icon aliases in the core module were deprecated in v0.13.3
in November. By removing them we save some overhead for all users, as
part
of our work to slim down the library by de-duplicating and later removing
each of the old core icons.
For 'picture', use 'image' from the 'media' icon pack.
For 'insert', use 'add' from core (still to be moved out).
I believe these have had no Wikimedia production usage for months now.
Deprecating change since last release:
- Create single icon for language/translation (Ed Sanders)
As part of our on-going work to slim down the library, we have merged the
old 'textLanguage' icon in the 'editing-styling' icon pack, and the
'translation' icon in the 'editing-advanced' icon pack into a new icon,
'language', also in 'editing-advanced'. We have also changed the icon to
now use the Chinese character '文' which means language, instead of '大'
which means big.
For now, 'textLanguage' and 'translation' will still work, but please
switch
over to 'language' before it is removed in OOjs UI v0.17.0.
- Move 'redirect' icon to 'articleRedirect' and cleanup (Ed Sanders)
For consistency, we have moved 'redirect' in 'editing-advanced' into the
'content' pack as 'articleRedirect', so that it is alongside the other
article type icons. Again, 'redirect' will still work for now, but please
switch over to 'articleRedirect' before it is removed in OOjs UI v0.17.0.
Additional details are in the full change log[0]. If you have any further
questions or need help dealing with deprecations, please let me know. As
always, general library documentation is available on mediawiki.org[1], and
on
doc.wikimedia.org there is generated code-level documentation and
interactive
demos[2].
[0] -
https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md
[1] - https://www.mediawiki.org/wiki/OOjs_UI
[2] - https://doc.wikimedia.org/oojs-ui/master/
Yours,
--
James D. Forrester
Lead Product Manager, Editing
Wikimedia Foundation, Inc.
jforrester(a)wikimedia.org | @jdforrester
(cross-posted to labs-l and wikitech-l)
I've been thinking quite a bit about Tool Labs developers over the
past couple of months. Last week I was in San Francisco for the
Wikimedia Developer Summit and had an opportunity to talk to a number
of people about some of my ideas. One of them that got a pretty good
reception was a few proposed changes to the wikitech.wikimedia.org
wiki.
I've started "Make wikitech more friendly for the multiple audiences
it supports" phab task [0] and started adding blocker tasks for
various ideas that have come up in the discussions. One is to enable a
"Portal" namespace on wikitech [1] to be used to make landing pages
for different audiences. Another is to add a "Tool" namespace [2] that
the various Tool Labs projects could use to document their projects. A
third is to disable searching "Nova resource" pages by default [3].
The wikitech wiki doesn't currently have what anyone would consider an
active on-wiki community to seek consensus from on topics like this so
I am reaching out to what I hope are relevant mailing lists to gather
feedback. Discussion of why any of these ideas is a great or horrible
idea is welcome here on list or possibly even better on the
phabricator tasks themselves. Again since we don't have a strong
social norm for when consensus has been reached for this wiki I'm
going to arbitrarily suggest that discussion remain open until
2016-01-20 after which time I will take further steps to have the new
namespaces enabled (or got back to the drawing board if the concept is
rejected).
[0]: https://phabricator.wikimedia.org/T123425
[1]: https://phabricator.wikimedia.org/T123427
[2]: https://phabricator.wikimedia.org/T123429
[3]: https://phabricator.wikimedia.org/T122993
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA
irc: bd808 v:415.839.6885 x6855
Basically, the xml dumps have 2 IDs: page_id and revision_id.
The page_id points to the article. In this case, 14640471 is the page_id
for Mars (https://en.wikipedia.org/wiki/Mars)
The revision_id points to the latest revision for the article. For Mars,
the latest revision_id is 699008434 which was generated on 2016-01-09 (
https://en.wikipedia.org/w/index.php?title=Mars&oldid=699008434). Note that
a revision_id is generated every time a page is edited.
So, to answer your question, the IDs never change. 14640471 will always
point to Mars, while 699008434 points to the 2016-01-09 revision for Mars.
That said, different dumps will have different revision_ids, because an
article may be updated. If Mars gets updated tomorrow, and the English
Wikipedia dump is generated afterwards, then that dump will list Mars with
a new revision_id (something higher than 6999008434). However, that dump
will still show Mars with a page_id of 1460471. You're probably better off
using the page_id.
Finally, you can see also reference the Wikimedia API to get a similar view
to the dump: For example:
https://en.wikipedia.org/w/api.php?action=query&prop=revisions&titles=Mars&…
Hope this helps.
On Mon, Jan 11, 2016 at 5:09 AM, Luigi Assom <luigi.assom(a)gmail.com> wrote:
> yep, same here!
>
> Also another question about consistency of _IDs in time.
> I was working with an old version of wikipedia dump, and testing some
> data models I built on the dumpusing as pivot a few topics.
> I might have data corrupted on my side, but just to be sure:
> are _IDs of article *persistent* over time, or are they subjected to
> change?
>
> Might happen that due any fallback or merge in an article history, ID
> would change?
> E.g. as test article "Mars" would first point to a version _ID ="4285430"
> and then changed to "14640471"
>
> I need to ensure _IDs will persist.
> thank you!
>
>
> *P.S. sorry for cross posting - I've replied from wrong email - could you
> please delete the other message and keep only this email address? thank
> you! *
>
> On Mon, Jan 11, 2016 at 11:05 AM, XDiscovery Team <info(a)xdiscovery.com>
> wrote:
>
>> yep, same here!
>>
>> Also another question about consistency of _IDs in time.
>> I was working with an old version of wikipedia dump, and testing some
>> data models I built on the dump using as pivot a few topics.
>> I might have data corrupted on my side, but just to be sure:
>> are _IDs of article *persistent* over time, or are they subjected to
>> change?
>>
>> Might happen that due any fallback or merge in an article history, ID
>> would change?
>> E.g. as test article "Mars" would first point to a version _ID ="4285430"
>> and then changed to "14640471"
>>
>> I need to ensure _IDs will persist.
>> thank you!
>>
>>
>> On Mon, Jan 11, 2016 at 6:22 AM, Tilman Bayer <tbayer(a)wikimedia.org>
>> wrote:
>>
>>> On Sun, Jan 10, 2016 at 4:05 PM, Bernardo Sulzbach <
>>> mafagafogigante(a)gmail.com> wrote:
>>>
>>>> On Sun, Jan 10, 2016 at 9:55 PM, Neil Harris <neil(a)tonal.clara.co.uk>
>>>> wrote:
>>>> > Hello! I've noticed that no enwiki dump seems to have been generated
>>>> so far
>>>> > this month. Is this by design, or has there been some sort of dump
>>>> failure?
>>>> > Does anyone know when the next enwiki dump might happen?
>>>> >
>>>>
>>>> I would also be interested.
>>>>
>>>> --
>>>> Bernardo Sulzbach
>>>>
>>>> _______________________________________________
>>>> Wikitech-l mailing list
>>>> Wikitech-l(a)lists.wikimedia.org
>>>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>>>
>>>
>>> CCing the Xmldatadumps mailing list
>>> <https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l>, where
>>> someone has already posted
>>> <https://lists.wikimedia.org/pipermail/xmldatadumps-l/2016-January/001214.ht…> about
>>> what might be the same issue.
>>>
>>> --
>>> Tilman Bayer
>>> Senior Analyst
>>> Wikimedia Foundation
>>> IRC (Freenode): HaeB
>>>
>>> _______________________________________________
>>> Xmldatadumps-l mailing list
>>> Xmldatadumps-l(a)lists.wikimedia.org
>>> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>>>
>>>
>>
>>
>> --
>> *Luigi Assom*
>> Founder & CEO @ XDiscovery - Crazy on Human Knowledge
>> *Corporate*
>> www.xdiscovery.com
>> *Mobile App for knowledge Discovery*
>> APP STORE <http://tiny.cc/LearnDiscoveryApp> | PR
>> <http://tiny.cc/app_Mindmap_Wikipedia> | WEB
>> <http://www.learndiscovery.com/>
>>
>> T +39 349 3033334 | +1 415 707 9684
>>
>
>
>
> --
> *Luigi Assom*
>
> T +39 349 3033334 | +1 415 707 9684
> Skype oggigigi
>
> _______________________________________________
> Xmldatadumps-l mailing list
> Xmldatadumps-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>
>
Bartosz Dziewoński wrote:
>On 2016-01-11 22:06, gnosygnu wrote:
>> So, to answer your question, the IDs never change. 14640471 will always
>> point to Mars, while 699008434 points to the 2016-01-09 revision for
>>Mars.
>
>While it's unlikely/rare, I think the page id can change when a page is
>deleted and re-created, and maybe some other cases. MediaWiki tries to
>keep it constant (for example, I think it's preserved after deletion and
>undeletion), but it's not always possible.
It looks like <https://phabricator.wikimedia.org/T28123> was just fixed,
so page IDs changing should now hopefully be rarer.
MZMcBride