Hello all,
over the past 1,5 years WMDE’s Technical Wishes team has been working on a
new functionality that allows to transfer files from a local wiki to
Wikimedia Commons, while keeping all original data intact and documenting
the move in the version history. [1]
The new functionality has been implemented via two extensions: FileImporter
and FileExporter. [2] [3]
We’re happy to announce that the new feature set can be tried out as a beta
feature on first wikis as of today! [4]
Why this feature
Many wikis have locally stored files, but most of them (if the license
allows it) should be stored on Wikimedia Commons instead, so that all wikis
can use them. Out of that reason, contributors have been moving locally
stored files to Wikimedia Commons with the help of tools for years now.
However, the current transfer process has limits: If you move a file, its
version history page isn’t kept on Commons, so e.g. it doesn’t show who
originally uploaded the file. Besides, previous revisions of the file have
to be added separately to the file page. Keeping all file information
intact currently can’t be done by tools, as this has to be programmed in
the backend. [5]
This is why we developed the FileImporter/FileExporter extensions.
On a side note: The way the features are built, they theoretically could be
used for transferring files from any mediawiki wiki to another mediawiki
wiki in the future (the different wikis don't even have to be part of the
same wikifarm).
However, our focus for now is:
Transfer of files from local wikis to Wikimedia Commons
The FileExporter provides a link to Wikimedia Commons on the local wiki and
hands over a URL to the extension FileImporter. The FileImporter is a
Wikimedia Commons extension which imports the file including all data when
all prerequisites are met. Technically, the files won't be "moved", but
"copied". The local communities can decide which templates to use to
indicate that a file should or shouldn’t be moved to Commons. [6] Further,
the decision what to do with the files after they got transferred to
Commons stays with the local communities (e.g. admins can delete the files).
To make use of the FileImporter, you have to activate the FileExporter in
your local wiki, provided as a beta feature on Arabic Wikipedia, German
Wikipedia, Persian Wikipedia and Mediawiki.org for now. The FileImporter on
Wikimedia Commons is not a beta feature, so there is no need to activate
it. [7]
If your local wiki community is interested to test the new functionality in
this first round as well, please discuss this with your community and let
us know by filing a request ticket on Phabricator :-) (example ticket:
https://phabricator.wikimedia.org/T196908)
Last but not least, we’d like to thank all the people who have tested the
feature earlier, and who have provided valuable feedback. This is much
appreciated!
Further feedback is very welcome on the central feedback page:
https://www.mediawiki.org/wiki/Help_talk:Extension:FileImporter
Thank you,
Birgit (for the Technical Wishes Team)
[1]
https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Move_files_to_Commons
(main project page on Meta)
[2] https://www.mediawiki.org/wiki/Extension:FileImporter
[3] https://www.mediawiki.org/wiki/Extension:FileExporter
[4] https://phabricator.wikimedia.org/T196969 (Deployment ticket for
FileExporter/beta feature and FileImporter)
[5]
https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Move_files_to_Commons
(main project page on Meta)
[6] https://www.mediawiki.org/wiki/Extension:FileImporter/Data, please note
that any updates to these files will be used by FileImporter only by next
week!
[7] https://www.mediawiki.org/wiki/Help:Extension:FileImporter (central
help page)
--
Birgit Müller
Community Communications Manager
Software Development and Engineering
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Hi All,
Here are the minutes from this week's TechCom meeting:
* Call for nominations of new TechCom members was closed on June 18 with 15
nominations received. We will be reviewing the candidates over the coming weeks.
* New RFC: “Implement a standardized way to track the usage of parser functions
and tags” <https://phabricator.wikimedia.org/T197493>
* Moving towards implementation: multi-file packages for ResourceLoader
<https://phabricator.wikimedia.org/T133462>.
* Moving towards implementation: “Tracking dependencies for multiple Content
objects per page (MCR)” <https://phabricator.wikimedia.org/T190063>
* Implementation in progress: “Allow users to be blocked from editing a specific
article or all articles inside a namespace”
<https://phabricator.wikimedia.org/T2674>
* Interested to not record IPs for anon edit, see
<https://phabricator.wikimedia.org/T133452>
* Pending alternative proposal to be written: “Use ar_page_id to determine the
parent IDs for undeleted revisions” <https://phabricator.wikimedia.org/T193690>
* Public RFC discussion on Wednesday: “Factoring page update logic out of
WikiPage”. This is about the overall architecture of the code we use to process
page updates. It’s a major part of a comprehensive refactoring of the MediaWiki
storage layer. <https://phabricator.wikimedia.org/T198075>
This discussion will take place in the #wikimedia-office channel at 21:00 UTC
(23:00 CEST, 2pm PDT ).
-- Daniel
(I was trying to take this entire weekend offline, but having failed at that for a variety of reasons, I am now giving up completely.)
MZMcBride, while I appreciate your energetic engagement, I think that it's reasonable to ask that you slow down a bit and consider alternative ways to express your disagreement. Unless I am overlooking something in that thread, I don't think that Andre was trolling.
Regarding how the Technical Code of Conduct Committee works, I agree that there should be some changes, but I suggest that proposals for change should be discussed in a separate thread and that they should be proposed in a civil way.
I suggest that you try to think about topics other than Wikimedia for 24 hours, and that you return when you feel more relaxed. Phabricator will probably still be there after a 24 hour break, and hopefully after your break you will feel better.
Thanks,
Pine
( https://meta.wikimedia.org/wiki/User:Pine )
null
Howdy,
There is an IRC office hour scheduled next week to discuss
improvements to the mobile MediaWiki web site for contribution tools
and workflows. It will be from 16:00-17:00 UTC Monday 25 June 2018.
Links to join are on Meta. [0]
The Readers web team would like to make more contribution tools (like
Special: pages) available on mobile. Come talk to us about what you
want to see the most. More information on what we'd like to discuss
can be found on MediaWiki.org. [1]
I hope you can make it. If not, please take a look at the project page
and leave any comments on the talk page.
[0] https://meta.wikimedia.org/wiki/IRC_office_hours#Upcoming_office_hours
[1] https://www.mediawiki.org/wiki/Reading/Web/Advanced_mobile_contributions
Yours,
Chris Koerner
Community Liaison
Wikimedia Foundation
https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-06-20
== Callouts ==
* Security: Security review for Wikidata queries data release proposal
https://phabricator.wikimedia.org/T190875
* MCR: a couple huge patches related to page editing and revision
read/write have been merged and are on group0 for testing, keep an eye out
for strange behavior (
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/405015https://gerrit.wikimedia.org/r/c/mediawiki/core/+/406595https://gerrit.wikimedia.org/r/c/mediawiki/core/+/416465)
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
** 5.8.2 w/ event logging analytics and bug fixes released (
https://phabricator.wikimedia.org/project/view/3358/ )
** Continuing work on next major release, 6.0 - Feed customization and
design updates ( https://phabricator.wikimedia.org/project/view/3238/ )
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
**
==== Readers Web ====
* Blocked by:
** Ruby to JS Cucumber refactor needs help from the RelEng team to fix our
flaky Ruby tests: https://phabricator.wikimedia.org/T190710
* Blocking:
* Updates:
** Mobile web (MobileFrontend) page issues
*** A/B testing framework merged T193584
*** UI in progress T191303
** Mobile web (MobileFrontend / MinervaNeue) bug fixes and improvements
T189225 T194544 T196758 T156186 T196802 T190549 T161399 T193517 T197627
T192725 T195371
** Add WikidataPageBanner per skin config flag to allow subtitles to render
above the banner T113642
** Page previews (Popups) post-en/dewiki launch clean up T191888 T192627
T191646 T192928 T193522 T196952 T193519 T193792
** PDF rendering service (Proton) T178665 T186748 T189307
** Mobile web in-article navigation user testing and design T193772
** Wrapping up annual planning
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
==== Readers Infrastructure ====
* Blocked by:
* Blocking:
* Updates:
** Safari Reading List extension: still waiting on QA to finish
** MCS: Working on using clients' Accept-Language header value for MCS's MW
API calls, too.
** PCS: Working on content-html endpoint.
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
**[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of
client apps by coalescing and moving more logic to the server]]
***Reading Infra depends on Parsing, Services
===== Maps =====
* Blocked by:
* Blocking:
* Updates:
** New style and borders work deploy cancelled indefinately
==== Multimedia ====
* Work on search prototypes for depicts statements is underway - getting a
head start on next quarter
* Further search work continues on various indexing patches
* OOUI migration for Wikibase is stalled mostly due to Mark being
interrupted every five minutes or so
* Quarterly goal dependency update
** Prepare for launch of the first Structured Data on Commons feature
(multilingual file captions)
*** SDC depends on Multimedia,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
*** Work continues here, MCR schedule is known, OOUI migration is underway
(but slow) - upload integration mostly complete
*** Goal will bleed slightly into next quarter especially with regard to
OOUI migration for Wikibase
** Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
*** Mostly complete
** Develop a better understanding of existing needs for Structured Commons-
T171252
*** Research depends on Multimedia
*** AFAIK still complete/underway and no engineering resources needed
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
** CodeMirror is out of beta
** Final fixes for GlobalPreferences
** Still working on adding Draft: namespace to PageTriage
==== Anti-Harassment Tools ====
* Blocked by:
* Blocking:
* Updates:
**
==== Editing ====
* Blocked by:
* Blocking:
** Updates:
**
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
** Language variants endpoint now live in Beta Cluster. Plan to deploy
to production on Monday pending all tests okay.
** Tidy to Remex switch over nearly complete
** We are planning to do the final switch from Tidy -> RemexHtml for
all wikis on the wikimedia cluster on July 5th
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]] Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
*** Parsing depends on MediaWiki Platform, Services
**[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of
client apps by coalescing and moving more logic to the server]]
Increase code sharing of client apps by coalescing and moving more logic to
the server.
*** Reading Infra depends on Parsing, Services
**
[[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better
data|Outcome 2: Objective 1]]: Revision storage scaling
*** Services depends on SRE, Parsing
==== Collaboration ====
* Blocked by:
* Blocking:
* Updates:
**
==== Language ====
* Blocked by:
* Blocking:
* Updates:
** ContentTranslation 2 work continue: MT Options, Title error fixes.
** Purge script and Apertium (apy, fra-cat) updates likely to be unblocked
next week.
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.1]] Improve and consolidate our unified editing platform so that it's
great on all devices
***Language depends on Editing
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
**
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
**
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
**
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
**
=== Fundraising Tech ===
* Blocked by: -
* Blocking: -
* Updates:
** Normal maintenance on payment processor integrations, especially
Ingenico.
** Moving pages in the Fundraising workflow currently on Foundation Wiki to
Donate Wiki.
** Export and deletion features in CiviCRM (system for donor data).
** Continuing work on data pipeline for sampled banner and landing page
data.
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]] Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
** [[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 1.1]] Assist
with deploying MultiContent Revisions on Commons
[[phab:T174022|T174022]] Implement multi-content revisions,
[[phab:T174023|T174023]] Implement MCR storage layer,
[[phab:T174045|T174045]] DB schema migration for MCR,
[[phab:T174044|T174044]] Deploy MCR storage layer,
[[phab:T174043|T174043]]Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
=== Performance ===
* Blocked by:
**
* Blocking:
**
* Updates:
**
=== Release Engineering ===
* Blocked by:
** None
* Blocking:
** Working on https://phabricator.wikimedia.org/T190710 for Readers
* Updates:
** wmf.999 is running on group0 wikis for testing MCR related changes, see
https://phabricator.wikimedia.org/T196585
** thanks to those that participated in the Beta Cluster survey!
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
=== Research ===
* Blocked by: None
* Blocking: None
* Updates:
** Finished adding UI translations (all 5 languages) to the Gapfinder tools
project.: http://gapfinder-tools.wmflabs.org/
** Script to train models based on Wikidata sitelink count is done. The
next step is to add one more feature:
https://github.com/wikimedia/research-translation-recommendation-models/
** We'll be modifying the Unsourced Statements Wikilabels campaign and
re-run it soon.
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Structured_Data#Segment_4:_Programs|Segment
4, Outcome 2]]: Develop a better understanding of existing needs for
Structured Commons- [[phab:T171252|T171252]]
***Research depends on Multimedia
=== Scoring Platform ===
* Blocked by: None
* Blocking: None
* Updates:
**wp10 model deployed - also available in MediaWiki database
***Call out Community Tech & Growth
** JADE scalability discussion continues
https://phabricator.wikimedia.org/T196547
=== Search Platform ===
* Blocked by:
** Security: Security review for Wikidata queries data release proposal
https://phabricator.wikimedia.org/T190875
* Blocking:
* Updates:
** Reindexed Serbian, Croatian, Serbo-Croatian and Bosnian Wikis to make
use of new analyzers
** Fixed regex timeout handling in Elasticsearch:
https://phabricator.wikimedia.org/T195491
** Ideas on using NLP is search:
https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Potential_Applicatio…
** Reviewed applying Indonesian analysis chain to Malay:
https://phabricator.wikimedia.org/T196780
** Reviewing Esperanto morphological libraries:
https://phabricator.wikimedia.org/T197240
** Working on Lexeme fulltext search:
https://phabricator.wikimedia.org/T196188
** Working on fixing Geodata issues:
https://phabricator.wikimedia.org/T196526
** Working on query parsing refactoring:
https://phabricator.wikimedia.org/T185108
**
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|*Objective 2.1]]
Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
=== Security ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]]: Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
=== Services ===
* Blocked by: none
* Blocking: none?
* Updates:
** REST API Language variants support for HTML is live on beta for some
wikis
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of
client apps by coalescing and moving more logic to the server.]]
***Reading Infra/Parsing, Services
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]] Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
[[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better
data|Outcome 2: Objective 1]]: Revision storage scaling
*** Services depends on SRE, Parsing
=== Site Reliability Engineering ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
[[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better
data|Outcome 2: Objective 1]]: Revision storage scaling
*** Services depends on SRE, Parsing
== Wikidata ==
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 1.1]] Assist
with deploying MultiContent Revisions on Commons
[[phab:T174022|T174022]] Implement multi-content revisions,
[[phab:T174023|T174023]] Implement MCR storage layer,
[[phab:T174045|T174045]] DB schema migration for MCR,
[[phab:T174044|T174044]] Deploy MCR storage layer,
[[phab:T174043|T174043]]Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
**
== SoS Meeting Bookkeeping ==
* Updates:
**
Sorry for cross-posting!
Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 3-4 pm
UTC** on #wikimedia-tech.
The Technical Advice IRC meeting is open for all volunteer developers,
topics and questions. This can be anything from "how to get started" over
"who would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting: https://www.mediawiki.org/
wiki/Technical_Advice_IRC_Meeting
Hope to see you there!
Michi (for WMDE’s tech team)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
A couple of issues have been affecting phabricator.wikimedia.org since last
week and are only partially mitigated. I'd like to provide a brief summary
of the current status for those who haven't been following along closely:
1. There were several incidents of vandalism by apparently manual or
semi-automated means. In response, phabricator admins took steps to limit
further damage. The most visible consequence of this is that we have
temporarily enabled manual approval of new accounts. This means that all
newly-created accounts must be approved by an admin before they can use
phabricator. We will approve any and all accounts which appear to be
legitimate users but there might be some delay as we have a limited amount
of time to devote to reviewing new accounts.
2. The recent gerrit upgrade had unexpected consequences for Phabricator's
git repository mirroring functionality. Because gerrit was migrated to
"notedb," a lot of new meta-data was exposed via git references.
Phabricator was unable to mirror the new metadata without hitting some
timeouts and as a result, many git repositories were failing to import into
Phabricator. This has been mostly resolved now, however, there may still be
some repositories which are failing to update. Please report any issues
that you encounter on the associated task:
https://phabricator.wikimedia.org/T196855
Mukunda Modell
Wikimedia Release Engineering