I'm happy to announce a new mirror for datasets other than the XML dumps.
This mirror comes to us courtesy of the Center for Research Computing,
University of Notre Dame, and covers everything "other" [1] which includes
such goodies as Wikidata entity dumps, pageview counts, titles of all files
on each wiki (daily), titles of all articles of each wiki (daily), and the
so-called "adds-changes" dumps, among other things. You can access it at
http://wikimedia.crc.nd.edu/other/ so please do!
Ariel
[1] https://dumps.wikimedia.org/other/
The Wikimedia Language team has been assembling monthly reports about
language support activities for one year. You can read the latest
report at:
https://www.mediawiki.org/wiki/Wikimedia_Language_engineering/Reports/2016-…
Highlights for May include: Special:Translate got an edit summary
field and modernization of web font formats: woff2 is in, eot is out.
Due to the nature of our work, the Language team [1] (Amir, Kartik,
Pau, Runa, Santhosh, and myself) alone cannot adequately support all
the languages of the Wikimedia movement. That is why the report
includes work by volunteers. We have bolded the names who we believe
are contributing as volunteers.
This report focuses on technical activities. You wont find future
plans or high level roadmap items on it. There is currently a major
omission: the i18n work of MediaWiki core itself. That is lacking
because it is more difficult to filter those activities and also
because we have not had much time for MediaWiki core i18n work.
To acknowledge the work of volunteers and to support them better, the
Language team released a statement of intent for code review [2] about
six months ago. To summarize: we attempt to review patches not by us
within a week, and patches stalled due to no updates after review for
three months will be abandoned -- unless we feel they are worth fixing
ourselves.
When we released the statement, we also agreed to reduce the existing
backlog of open patches. The results so far are positive, even though
it is easy to find examples where we have not been able to follow our
intent. Translate extension had 35 open patches when we started in
February -- at end of May it had only 12 open patches [3]. Universal
Language Selector had gone from 10 to 6, and fewer of them unreviewed.
Content Translation had gone from 15 to zero. Our jquery repositories
in GitHub have not fared as well, but we hope to achieve similar
results there in the future.
We excluded many repositories from the statement of intent in the fear
that we would add too much of a burden to ourselves. To our delight,
except MediaWiki core i18n, all those repositories have had swift
reviews and I count only two open patches in them.
- Niklas (on behalf of the Language team)
[1] https://www.mediawiki.org/wiki/Wikimedia_Language_engineering
[2] https://www.mediawiki.org/wiki/Wikimedia_Language_engineering/Code_review_s…
[3] The numbers change constantly. As of 2016-06-17 Translate has 23
open patches, but only 10 of them not from our team. Universal
Language Selector has 13 patches, 5 of them not from our team. Content
Translation currently has 6, one of them not from our team.
Hi folks
I've made some updates to the [ArchCom status][1] page. In
particular, I linked to [the very rough RFC I filed about
Markdown][2], where I'm suggesting we develop a strategy around
Markdown. The plan we agreed to in [2016W24 ArchCom Planning
meeting][3] was to discuss Markdown at [next week's ArchCom-RFC office
hour][4], since we think that'll be a good hallway track conversation
for people at Wikimania, and the IRC conversation will be good fun for
those of us who aren't going to make it out this year but can make it
to the IRC conversation.
Rob
[1]: https://www.mediawiki.org/wiki/Architecture_committee/Status
[2]: https://phabricator.wikimedia.org/T137946
[3]: https://phabricator.wikimedia.org/E212
[4]: https://phabricator.wikimedia.org/E218
Lydia and Amir FTW.
(There seems to be a pattern of good news from the Wikidata and ORES teams.
I wonder if Team Practices could facilitate the replication of some of
their methods!)
Onward ho,
Pine
On Jun 16, 2016 10:14, "Lydia Pintscher" <Lydia.Pintscher(a)wikimedia.de>
wrote:
Hey folks :)
Amir and other have worked hard over the past months to bring ORES to
Wikidata. The goal is to use machine learning to make it easier to spot
potentially bad edits. ORES is now available as a beta feature on Wikidata.
Once you have enabled it you can see some edits in recent changes and
watchlist will show up in a different color or have a little r in front of
them. These edits are judged as potentially bad and should probably get
more review. In your preferences you can adjust how harsh ORES should
judge. You can also filter your watchlist/recent changes to only show
potentially bad edits. Patrolled edits won't be shown as potentially bad.
This should be a huge step towards making it easier to find and fight
vandalism on Wikidata.
Cheers
Lydia
--
Lydia Pintscher - http://about.me/lydia.pintscher
Product Manager for Wikidata
Wikimedia Deutschland e.V.
Tempelhofer Ufer 23-24
10963 Berlin
www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
_______________________________________________
Wikidata mailing list
Wikidata(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata
The Community Tech team just posted a new job opening for a developer to
join the team (http://grnh.se/xp3wfk). We are especially interested in
hiring someone with knowledge of MediaWiki and the community.
As you may know, the Community Tech team works on requests from the
community, typically what you might call "power-user" tools and features.
These might be MediaWiki extensions, Tool Labs tools, or even just fixing
broken gadgets. One of the bigger projects that we're going to start
working on next quarter is cross-wiki watchlists.
If you think this job might be a good fit for you, please fill out the
application at http://grnh.se/xp3wfk. It's especially helpful if you
include a cover letter explaining why you would like to work for the team
and what your experience with MediaWiki and/or the community is.
https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-06-15
= 2016-06-15 =
== Product ==
=== Reading ===
==== Android ====
* Still working on the content feed.
* Need from SERVICES: regeneration of certain mobile-section endpoints to
fix a couple of outstanding bugs (T136964)
** Is anything needed from us before this can be kicked off?
==== Mobile Content Service ====
* Need from SERVICES: Hook up public facing feed endpoints (T136960)
==== Reading Web ====
* Fixing things to prepare wikidata descriptions on mobile web to stable in
following sprints
* Continue work on lazy loading images and references on mobile web
==== iOS ====
* It's WWDC week
==== Reading Infrastructure ====
* AuthManager has shipped, working on follow-up tasks
* blocked on T111486 Update Translate to use AuthManager (Language?)
=== Community Tech ===
* Still working on CopyPatrol tool ( https://tools.wmflabs.org/copypatrol )
* Beta testing RevisionSlider Extension on Labs with WMDE TCB team (
https://www.mediawiki.org/wiki/Extension:RevisionSlider )
* Beta testing PageAssessments Extension on Labs (
https://www.mediawiki.org/wiki/Extension:PageAssessments )
* Blockers: Beta Labs wikis locked
=== Editing ===
==== Collaboration ====
* Blocked - None
* Blocking - External Store on Beta: Waiting to hear back at
https://phabricator.wikimedia.org/T119567
* Updates
** Working on new Special:Notifications sidebar to show pages with unread
notifications
** Finishing expandable Echo bundles
** Working on Flow deletion-related issues
==== Parsing ====
* Second RFC to replace Tidy with HTML5Depurate on the WMF cluster last
Wednesday
** Things moving ahead with setting up some tools to help editors figure
out how this will affect them and make wikitext changes where required
(since there is broken wikitext out there that will render differently with
a HTML5 parser vs. Tidy).
** Tim in conversation with Ops about how to set up this HTML5Depurate
service on the app cluster.
* Work ongoing to migrate Parsoid to use service-runner. Marko and Services
are now in the loop and handling this.
* Scott & Kunal at Wikimania next week; Scott @ Kiwix hackathon this week
and will focus on OCG (PDF generation) there.
==== Language ====
* Blocking: None?
* Blocked: None
* Updates:
** Compact Language Links deployment starting next week.
** Jessie migration for Apertium in progress.
** Lots of template related fixes done/coming up in Content Translation.
==== Multimedia ====
* No blocking, no blocked
* Gallery work progressing, nearing working prototype
* FileAnnotations repository exists, wikitext storage, editing interface on
the way
* Performance/Gilles: Thumbor status?
== Technology ==
=== RelEng ===
* '''Blocking''': T123718 - Phase out antimony
** this should be done
* '''Blocked''': None
* '''Updates''':
** REMINDER! Migrate your things to scap3:
https://phabricator.wikimedia.org/project/view/1824/
*** What have you touched? Please migrate! :)
** Proposal for the #scrum-of-scrums workboard:
https://phabricator.wikimedia.org/T137904
=== Services ===
* started work on the auth and session storage service
** will write up an RfC describing it to get wider input
* Change-propagation
** struggling with transclusion rules
* Parsoid move to Jessie
** started working on porting it to service::node
** next: move to Jessie, then Scap3
* better MW deploys
** https://phabricator.wikimedia.org/T110068
** want to add a Swagger spec to MW for automatic monitoring
* Services team out 21/06 - 02/07 - wikimania + offsite in Lisbon
=== Security ===
* One usability bugs for Ex::OATHAuth in progress (T136988)
* Two-factor usability survey work resuming
* Fix for T137264 will be deployed today
* Reviews: Android Services for Reading
=== Technical operations ===
* '''Blocked''':
** None
* '''Blocking''':
** Work on parallel corpora dumps resumed (was blocking Language
Engineering)
* Updates:
** Got a kubernetes backend for webservice in tool labs
*** migrating webservices to it
** Jessie appservers work ongoing
** 10% block for insecure POST (not HTTPS that is) went live on Monday
13:00 UTC
** various performance optimizations on cache_upload varnish cluster
=== Discovery ===
* '''Blocking''': none
* '''Blocked''': https://phabricator.wikimedia.org/T128546 need help with
scripts
* Portal launched new A/B test for collapsing long list of languages
* TextCat A/B test running on English wiki
=== Interactive ===
* Maps launched on English Wikivoyage
* Switched to production servers
* Soon removing referrer check, and allowing maps client code on all wikis
(but not yet the wiki markup just yet)
Hi everyone,
We're holding another ArchCom-RFC meeting this week to follow up on
the thread Yuri started about T120452.
The ArchCom status page is finally up-to-date with last week's info:
<https://mediawiki.org/wiki/Architecture_committee/Status>
(and it's quoted below)
Links!:
This week's meeting: <https://phabricator.wikimedia.org/E213>
RFC for this week's meeting: <https://phabricator.wikimedia.org/T120452>
Subject of the RFC: Allow tabular datasets on Commons (or some similar
central repository) (CSV, TSV, JSON, XML)
Location: #wikimedia-office IRC channel
Time: 2016-06-15 Wednesday 21:00 UTC (2pm PDT, 23:00 CEST)
Meetbot looks forward to scribing your presence!
Rob
----
Source of [[mw:Architecture_committee/Status]], where "phab:" links
are pointers to https://phabricator.wikimedia.org
This page provides status update for [[Requests for
comment|ArchCom-RFCs]], with an emphasis on ArchCom team member. As
of this writing on 2016-04-29, this update is an experiment discussed
[[Topic:T2zctt083izvx07l|weekly ArchCom update discussion on the
"ArchCom/Team practices" talk page]].
===== Recent RFC meetings =====
*ArchCom Planning meeting 2016W23: 2016-06-08: [[Phab:E202]] (E156/10)
**Notes: [[Architecture committee/2016-06-08]]
*ArchCom-RFC office hour 2016W23: 2016-06-08: [[Phab:E203]] (E66/38)
** [[Phab:T89331|T89331 Replace Tidy in MW parser with HTML 5
parse/reserialize]]
===== Upcoming RFC meetings =====
*ArchCom Planning meeting 2016W24: 2016-06-15: [[Phab:E212]] (E156/11)
**Notes: [[Architecture committee/2016-06-15]]
*ArchCom-RFC office hour 2016W24: 2016-06-15: [[Phab:E213]] (E66/39)
** [[Phab:T120452|T120452]]: Allow tabular datasets on Commons (or
some similar central repository) (CSV, TSV, JSON, XML)
*** See also: comments in T124569 and T134426
===== Entering Final Comment Period =====
* None.
===== Recently Approved =====
* none
==== RFC inbox ====
* [[phab:tag/archcom-rfc/|ArchCom RFC board]]:
** [[Phab:T124569|T124569 RFC: Data namespace blob storage on wikidata.org]]
===== Shepherd status =====
* Brion
** [[Phab:T107595|T107595]] Multi-content revisions is interesting,
needed for various things in multimedia land
*** Meeting happened earlier this week; notes are on the ticket
** T66214 - predictable thumb URLs
*** Break this out into:
**** Define set of core & extensible media file options for Handler extensions
**** Predictable thumb URLs
**** Improve InstantCommons perf by reducing need to run thumbnail URL lookups
**** Iframe-based rich media embedding for InstantCommons
** plan to write up new RfCs for:
*** In-browser SVG rendering (pick up existing bug & mailing list notes)
*** iframe+CSP-isolated JS widgets for rich content
**** & extend that to InstantCommons via embedding
*** iframe+CSP-isolated JS gadgets for UI plugins
**** Build these out from ideas from hackathon project T131436
* Daniel
** Software Quality working group - will follow up on earlier
proposal. Will talk to people at Wikimania
** Working on Multi Content Rev Spec with Brion
** T113034 [[phab:T113034|RFC: Overhaul Interwiki map, unify with
Sites and WikiMap]]: checking in with Adam
** T89733 (approved, with Stas driving implementation)
* Gabriel
** Looking into content composition working group, possibly kick-off
at Wikimania
** Discussing Multi Content Rev / RB interaction with Daniel;
follow-up at Wikimania
* Roan
** T108655 [[phab:T108655|RFC: Standardise JavaScript interfaces]]: I
need to start the second part, but the recent comments have me
confused. I'll need to talk to Timo and figure out what the subject of
part two should be.
* RobLa
** Working with [[User:DPatrick (WMF)|DPatrick]] on [[Wikimedia
Security Team]] issues in an attempt to be useful there.
** T123753 [[phab:T123753|Establish retrospective reports for Security
and Performance incidents]]
*** In scope for this group?
** Forming ArchCom-affiliated working groups
*** RFCs
**** T124504 [[phab:T124504|Transition WikiDev '16 working areas into
working groups]] and
**** T123606 [[Phab:T123606|RFC: Implement ArchCom-affiliated working
groups (process inspired by Rust's "subteams")]]
*** Testing Conpherence use for means of piloting ArchCom working
groups. Still using [[Phab:Z425]] as asynchronous ArchCom-RFC triage
channel. A Security/ArchCom quasi-working group discusses some issues
in [[Phab:Z411]]. I started renaming "subteams" to "working groups"
on [[Requests for comment/Governance]] (with Nemo's help).
* Tim
** [[Phab:T89331|T89331 (Replace Tidy in MW parser with HTML 5
parse/reserialize)]] - should meet to discuss migration rather than
implementation plan
** T114444 [[phab:T114444|RFC: Introduce notion of DOM scopes in
wikitext]]: (Update?)
*** Scott has implementation work in progress
* Timo
** [[phab:T18691|T18691 RFC: Section headings should have a clickable
anchor]]: Reading team expressed interest to help shape the solution.
Still open for more use cases and different ideas for how to solve it.
Possible scope creep.
** [[Phab:T135963|T135963 RFC: Implement CSP in MediaWiki]]: Was
discussed in the IRC office hours last week
([[phab:E198|https://phabricator.wikimedia.org/E198]]).
===== No activity in the last two weeks =====
* T122942 [[phab:T122942|RFC: Support language variants in the
RESTBase]] (Gabriel)
* T39902 [[phab:T39902|RFC: Implement rendering of redlinks in
Parsoid]] (no shepherd)
* T18691 [[phab:T18691|RFC: Section headings should have a clickable
anchor]] (Timo)
* T111588 [[phab:T111588|RFC: API-driven web front-end]] (Timo)
* T122825 [[phab:T122825|Service ownership and minimum maintenance
requirements]] (Gabriel)
* T105766 [[phab:T105766|RFC: Dependency graph storage]] (Gabriel)
* T66214 [[phab:T66214|Use content hash based image / thumb URLs &
define an official thumb API]] (Brion)
* T91162 [[phab:T91162|RFC: Shadow namespaces]] (Brion)
* T128351 [[phab:T128351|RFC: Notifications in core]] (Brion)
* T122825 [[phab:T122825|Service ownership and minimum maintenance
requirements]] (Gabriel)
* T54807 [[phab:T54807|Identify and remove legacy preferences from
MediaWiki core]] (no shepherd)
* T88596 [[phab:T88596|Improving extension management]] (Daniel)
===== Useful Phab links =====
* [[phab:maniphest/query/xc.j4DEYcjwu/#R|Query for shepherd assignments]]
* [[phab:search/query/lgPo47yENoTl/#R|Query for all ArchCom RFCs]]
* [[phab:project/board/52/|ArchCom-RfCs board]]
{{ArchComMinutes}}
I spent some time between projects today exploring the idea of progressive
image decoding using the VP9 video codec -- sort of a mashup of progressive
JPEG and WebP.
Like a progressive JPEG, each resolution step (a separate frame of the
"video") encodes only the differences from the previous resolution step.
Like WebP, it's more space-efficient than the ancient JPEG codec.
This sort of technique might be useful for lazy-loading images in our
modern internet, where screen densities keep going up and network speeds
can vary by a factor of thousands. On a slow network the user sees
immediate feedback during load, and on a fast network they can reach full
resolution quickly, still in less bandwidth than a JPEG. And since JS would
have control of loading, we can halt cleanly if the image scrolls
offscreen, or pick a maximum resolution based on actual network speed.
Detail notes on my blog:
https://brionv.com/log/2016/06/14/exploring-vp9-as-a-progressive-still-imag…
Sample page if you just want to look at some decoded images at various
resolutions (loading not optimized for slow networks yet!):
https://media-streaming.wmflabs.org/pic9/
It looks plausible, and should be able to use native VP9 decoding in
Firefox, Chrome, and eventually MS Edge in some configurations with a
JavaScript fallback for Safari/etc. Currently my demo just plops all the
frames into a single .webm, but to avoid loading unneeded high-resolution
frames they should eventually be in separate files.
-- brion
People are now working to conclude the "Code of Conduct/Cases" part of
the draft Code of Conduct:
* Section:
https://www.mediawiki.org/wiki/Code_of_Conduct/Draft#Page:_Code_of_Conduct.…
* Talk:
https://www.mediawiki.org/wiki/Talk:Code_of_Conduct/Draft
* Alternatively, you can provide anonymous feedback to
conduct-discussion at wikimedia.org .
This is the best time to make any final necessary changes to this
section (and explain why, in edit summaries and/or talk) and discuss it
on the talk page.
After this last call, I will send out another email seeking approval,
like before.
Thanks,
Matt Flaschen
Hello,
The Discovery Portal team would like to add Wikipedia app
<https://www.mediawiki.org/wiki/Wikimedia_Apps> download badges near the
bottom of the wikipedia.org <https://www.wikipedia.org> portal page and
we're looking for your feedback.
These new app badges will link to a free download of the Wikipedia app for
Android and iOS mobile platforms and will be displayed just to those
visitors to the wikipedia.org portal page that are using a mobile device.
We hope that the app badges will provide an introduction to our visitors
who are interested in, or might prefer to use, the platform-native mobile
apps. It's an update that makes sense to the Portal team, but we wanted to
ask the community for feedback and suggestions.
If you're interested in finding out more, please visit this page
<https://www.mediawiki.org/wiki/Wikipedia.org_add_mobile_app_badges> to
read more details and view the draft mocks. Constructive feedback, thoughts
and ideas can be added and discussed on the talk page
<https://www.mediawiki.org/wiki/Talk:Wikipedia.org_add_mobile_app_badges>;
and additional information about the portal can be found here
<https://www.mediawiki.org/wiki/Wikipedia.org_Portal>.
Cheers,
Deb
--
Deb Tankersley
Product Manager, Discovery
IRC: debt
Wikimedia Foundation