Mediawiki-api-announce

mediawiki-api-announce@lists.wikimedia.org

143 discussions

ApiParse: Deprecation of prop=headitems (and removal of partial ResourceLoader code)
by Brad Jorsch (Anomie) 28 Jul '16

28 Jul '16

The 'headitems' property for action=parse has been deprecated, and using it will return a warning. If constructing a page from scratch, you should use prop=headhtml instead. If updating an existing on-wiki page, use prop=modules|jsconfigvars and load them into the existing ResourceLoader code with mw.config.set() and mw.loader.using(). Also, the 'headitems' property will no longer return code related to ResourceLoader. Loading ResourceLoader would require additional data that is not sent by 'headitems', and given the deprecation it's not worth the trouble to add that data. -- Brad Jorsch (Anomie) Senior Software Engineer Wikimedia Foundation

1 0

Shutting down deprecated rest.wikimedia.org by September 1st
by Gabriel Wicke 22 Jul '16

22 Jul '16

We are going to finally switch off the deprecated domain rest.wikimedia.org by September 1st. This should not affect any REST API users, as this domain has been officially deprecated since January & sunset since April [1]. Since then, requests to that domain have returned an error informing users about the move. Access to the REST API is exclusively through the main project domains, following the following pattern: https://en.wikipedia.org/api/rest_v1/?doc Thank you for your cooperation, -- Gabriel Wicke Principal Engineer, Wikimedia Foundation [1]: https://lists.wikimedia.org/pipermail/wikitech-l/2016-April/085309.html

1 0

[BREAKING CHANGE] meta=notifications outputs array instead of object since 1.27.0-wmf.22
by Roan Kattouw 28 Jun '16

28 Jun '16

As of 1.27.0-wmf.22 (deployed to WMF wikis April 26-28) and MediaWiki release 1.27 (specifically, this change <https://phabricator.wikimedia.org/rECHOb10bd700333d2eef10c7df0a9946b87fd451…> in Echo), meta=notifications will return its result as an array rather than an object. Previously, the meta=notifications output looked like: { "query": { "notifications": { "list": { "12345": { "id": "12345", "type": "edit-user-talk", ... }, ... } Now, it looks like: { "query": { "notifications": { "list": [ { "id": "12345", "type": "edit-user-talk", ... }, ... ] My apologies for the late announcement, and thanks to APerson and Cienca Al Poder for noticing <https://phabricator.wikimedia.org/T138690> and encouraging me to write this announcement.

1 0

Insecure (non-HTTPS) API Requests to become unsupported starting 2016-06-12
by Brandon Black 14 May '16

14 May '16

TL;DR: ---- * All access to Wikimedia production sites/APIs should use https:// URLs, not http:// -- your bot/tool will break in the near future if it does not! * 2016-06-12 - insecure access is unsupported; starting on this date we plan to break (deny with 403) 10% of all insecure requests randomly as a wake-up call. * 2016-07-12 - we plan to break all insecure requests. ---- Hi all, As you may remember, all production Wikimedia wikis switched to HTTPS-only for all canonical domainnames nearly a year ago: https://blog.wikimedia.org/2015/06/12/securing-wikimedia-sites-with-https/ Since way back then, we've been forcing insecure HTTP requests to our canonical domains over to HTTPS by using redirects and Strict-Transport-Security, which is effective for the vast majority of access from humans using browsers and apps. In the time since, we've been chasing down various corner-case issues where loopholes may arise in our HTTPS standards and enforcement. One of the most-difficult loopholes to close has been the "Insecure POST" loophole, which is discussed in our ticket system here: https://phabricator.wikimedia.org/T105794 . To briefly recap the "Insecure POST" issue: * Most of our humans using browser UAs are not affected by it. They start out doing GET traffic to our sites, their GETs get redirected to HTTPS if necessary, and then any POSTs issued by their browser use protocol-relative URIs which are also HTTPS. * However, many automated/code UAs (bots, tools, etc) access the APIs using initial POST requests to hardcoded service URLs using HTTP (rather than HTTPS). * For all of the code/library UAs out there in the world, there is no universally-compatible way to redirect them to HTTPS. There are different ways that work for some UAs, but many UAs used for APIs don't handle redirects at all. * Regardless of the above, even if we could reliably redirect POST requests, that doesn't fix the security problem like it does with GET. The private data has already been leaked in the initial insecure request before we have a chance to redirect it. If we did some kind of redirect first, we'd still just be putting off the inevitable future date where we have to go through a breaking transition to secure the data. Basically, we're left with no good way to upgrade these insecure requests without breaking them. The only way it gets fixed is if all of our API clients in the world use explicit https:// URLs for Wikimedia sites in all of their code and configuration, and the only way we can really force them to do so is to break insecure POST requests by returning a 403 error to tools that don't. Back in July 2015, I began making some efforts to statistically sample the User-Agent fields of clients doing "Insecure POST" and tracking down the most-prominent offenders. We were able to find and fix many clients along the way since. A few months ago Bryan Davis got us further when he committed a MediaWiki core change to let our sites directly warn offending clients. I believe that went live on Jan 29th of this year ( https://gerrit.wikimedia.org/r/#/c/266958 ). It allows insecure POSTs to still succeed, but sends the clients a standard warning that says "HTTP used when HTTPS was expected". This actually broke some older clients that weren't prepared to handle warnings at all, and caused several clients to upgrade. We've been logging offending UAs and accounts which trigger the warning via EventLogging since then, but after the initial impact the rate flattened out again; clients and/or users that didn't notice the warning fairly quickly likely never will. Many of the remaining UAs we see in logs are simply un-updated. For example, https://github.com/mwclient/mwclient switched to HTTPS-by-default in 0.8.0, released in early January, but we're still getting lots of insecure POST from older mwclient versions installed out there in the world. Even in cases where the code is up to date and supports HTTPS properly, bot/tool configurations may still have hardcoded http:// site config URLs. We're basically out of "soft" ways to finish up this part of the HTTPS transition, and we've stalled long enough on this. ** 2016-06-12 is the selected support cutoff date ** After this date, insecure HTTP POST requests to our sites are officially unsupported. This date is: * A year to day after the public announcement that our sites are HTTPS only * ~ 11 months after we began manually tracking down top offenders and getting them fixed * ~ 4 months after we began sending warning messages in the response to all insecure POST requests to the MW APIs * ~ 1 month after this email itself On the support cutoff date, we’ll begin emitting a “403 Insecure POST Forbidden - use HTTPS” failure for 10% of all insecure POST traffic (randomly-selected). Some clients will retry around this, and hopefully the intermittent errors will raise awareness more-strongly than the API warning message and this email did. A month later (two months out from this email) on 2016-07-12 we plan to break insecure access completely (all insecure requests get the 403 response). In the meantime, we'll be trying to track down offending bots/tools from our logs and trying to contact owners who haven't seen these announcements. Our Community team will be helping us communicate this message more-directly to affected Bot accounts as well. Thank you all for your help during this transition! -- Brandon Black Sr Operations Engineer Wikimedia Foundation

1 0

Re: [Mediawiki-api-announce] Deprecating rest.wikimedia.org in favor of /api/rest_v1/
by Gabriel Wicke 15 Apr '16

15 Apr '16

Final reminder on this: We are planning to finally sunset rest.wikimedia.org in the week starting April 25th, 1 1/2 weeks from now. Please move your REST API clients to /api/rest_v1/ at the regular project domains instead! Thanks, Gabriel On Mon, Jan 25, 2016 at 11:00 AM, Gabriel Wicke <gwicke(a)wikimedia.org> wrote: > We have decided to officially retire the rest.wikimedia.org domain in > favor of /api/rest_v1/ at each individual project domain. For example, > > > https://rest.wikimedia.org/en.wikipedia.org/v1/?doc > > becomes > > https://en.wikipedia.org/api/rest_v1/?doc > > Most clients already use the new path, and benefit from better > performance from geo-distributed caching, no additional DNS lookups, > and sharing of TLS / HTTP2 connections. > > We intend to shut down the rest.wikimedia.org entry point around > March, so please adjust your clients to use /api/rest_v1/ soon. > > Thank you for your cooperation, > > Gabriel > > -- > Gabriel Wicke > Principal Engineer, Wikimedia Foundation -- Gabriel Wicke Principal Engineer, Wikimedia Foundation

1 0

REST API to follow page redirects from April 25th
by Gabriel Wicke 15 Apr '16

15 Apr '16

We are planning to enable automatic redirect following in all REST API [1] HTML entry points on April 25th. When responding to a request for a redirected title [2], the response headers will contain: Status: 302 Location: <path-to-redirect-target> For most clients, this means that their HTTP client will automatically follow redirects, simplifying common use cases. The few clients with a need to retrieve the redirect page content itself have two options: 1) Disable following redirects in the client. For HTML and data-parsoid entry points, the response still includes the HTML body & regular response headers like the ETag. 2) Send a `?redirect=false` query string parameter. This option is recommended for browsers, which lack control over redirect behavior for historical security reasons. If you do have a need to avoid following redirects, you can make these changes before the feature is enabled. Internally, we have already done so for VisualEditor and the Mobile Content Service. See also https://phabricator.wikimedia.org/T118548 for background & related discussion. Let us know if you have any concerns or questions about this. Thanks, Gabriel Wicke for the Wikimedia Services Team [1]: https://en.wikipedia.org/api/rest_v1/?doc (using en.wikipedia.org as an example) [2]: https://www.mediawiki.org/wiki/Help:Redirects

1 0

REST API soon redirecting to canonical URLs
by Gabriel Wicke 16 Mar '16

16 Mar '16

Hi, this is to let you know that we will start redirecting GET requests with non-canonical titles to their canonical equivalent next week. This is done to improve caching, and to ensure reliable purging of cached responses. The vast majority of clients handle redirects transparently, so at most this will lead to a small slow-down from the redirect. To avoid being redirected, make sure to use underscores instead of spaces, as in "Main_Page". Thanks, Gabriel Wicke

1 0

Minor REST API cleanup: Remove experimental listings, make timeuuid parameter mandatory for data-parsoid
by Gabriel Wicke 08 Mar '16

08 Mar '16

tl;dr: You are *very* likely not affected. We are planning two changes in the REST API: 1) Remove the experimental /page/html/ and /page/data-parsoid/ listings [1][2]. Our metrics show that these are essentially unused. The same title listing remains available at /page/title/ [3]. 2) Make the `tid` path parameter in the unstable /page/data-parsoid/{title}/{revision}/{tid} [4] end point mandatory. Data-parsoid is tied to a specific HTML render, and only requests with an explicit timeuuid from the corresponding HTML response are guaranteed to get the correct data-parsoid version. If things go to plan, we will deploy these changes sometime next week. Thank you for your understanding, Gabriel Wicke for the Services team [1]: https://en.wikipedia.org/api/rest_v1/?doc#!/Page_content/get_page_html [2]: https://en.wikipedia.org/api/rest_v1/?doc#!/Page_content/get_page_data_pars… [3]: https://en.wikipedia.org/api/rest_v1/?doc#!/Page_content/get_page_title [4]: https://en.wikipedia.org/api/rest_v1/?doc#!/Page_content/get_page_data_pars…

1 0

Re: [Mediawiki-api-announce] [Wikitext-l] [Mediawiki-api] [Engineering] [Wikitech-l] Deprecating rest.wikimedia.org in favor of /api/rest_v1/
by Gabriel Wicke 02 Feb '16

02 Feb '16

Hi Luigi, On Fri, Jan 29, 2016 at 12:31 PM, Luigi Assom <itsawesome.yes(a)gmail.com> wrote: > - how to extract _ID from ETag in headers: > GET /page/title/{title} the page id is indeed not directly exposed in the HTML response. However, the revision number is exposed as part of the ETag. This can then be used to request revision metadata including the page id at https://en.wikipedia.org/api/rest_v1/?doc#!/Page_content/get_page_revision_…. This is admittedly not very convenient, so I created https://phabricator.wikimedia.org/T125453 for generally improved page id support in the REST API. > - how to ensure > GET /page/title/{title with different char encoding or old titles are always > resolved to last canonical version} The storage backing this end point is automatically kept up to date with edits and dependency changes. Edits in particular should be reflected within a few seconds. >> If you refer to >> >> https://en.wikipedia.org/api/rest_v1/?doc#!/Page_content/get_page_graph_png…, >> this is an end point exposing rendered graph images for >> https://www.mediawiki.org/wiki/Extension:Graph (as linked in the end >> point documentation). > > > Oh very interesting! > So basically html markup can be extended ? > Would it be possible to share json objects as html5 markup and embed them in > wiki pages? The graph extension is using the regular MediaWiki tag extension mechanism: https://www.mediawiki.org/wiki/Manual:Tag_extensions Graphs are indeed defined using JSON within this tag. > I want to avoid to update my graph just because titles changes: entities are > always the same. Makes sense. The current API is optimized for the common case of access by title, but we will consider adding access by page ID as well. > I still don't know what parsoid is. Parsoid is the service providing semantic HTML and a bi-directional conversion between that & wikitext: https://www.mediawiki.org/wiki/Parsoid Gabriel

1 0

Deprecating rest.wikimedia.org in favor of /api/rest_v1/
by Gabriel Wicke 30 Jan '16

30 Jan '16

(+mediawiki-api-announce, 2nd try) ---------- Forwarded message ---------- From: Gabriel Wicke <gwicke(a)wikimedia.org> Date: Mon, Jan 25, 2016 at 11:00 AM Subject: Deprecating rest.wikimedia.org in favor of /api/rest_v1/ We have decided to officially retire the rest.wikimedia.org domain in favor of /api/rest_v1/ at each individual project domain. For example, https://rest.wikimedia.org/en.wikipedia.org/v1/?doc becomes https://en.wikipedia.org/api/rest_v1/?doc Most clients already use the new path, and benefit from better performance from geo-distributed caching, no additional DNS lookups, and sharing of TLS / HTTP2 connections. We intend to shut down the rest.wikimedia.org entry point around March, so please adjust your clients to use /api/rest_v1/ soon. Thank you for your cooperation, Gabriel -- Gabriel Wicke Principal Engineer, Wikimedia Foundation -- Gabriel Wicke Principal Engineer, Wikimedia Foundation

1 0

← Newer
1
2
3
4
5
6
7
8
...
15
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

Mediawiki-api-announce