We have a very old wiki which has basically never been updated for the past
decade and which was proving stubbornly resistant to updating several years
ago. And now the owner of the server has drifted away, but we do still
have control over the domain name itself. The best way that we can think
of to update everything is to scrape all of the pages/file, add them to a
brand new updated wiki on a new server, then point the domain to that new
server. Yes, user accounts will be broken, but we feel that this is the
most feasible solution unless someone else has another idea.
However, there's a lot of pages on meritbadge.org -- which is the wiki I'm
talking about. Any suggestions for how to automate this scraping process?
I can scrape the HTML off every page, but what I really want is to get the
wikitext off of every page.
Bart Humphries
bart.humphries(a)gmail.com
(909)529-BART(2278)
Hey,
I updated one of my extensions to use PHPUnit 6+ and am having troubles
getting the tests to run with older versions of MediaWiki. In particular,
I'd like my tests to run with MW 1.27. I suspect this is somehow possible
and has already been done in other extensions.
Cheers
--
Jeroen De Dauw | https://entropywins.wtf | https://keybase.io/jeroendedauw
Software Crafter | Speaker | Student | Strategist | Contributor to Wikimedia
and Open Source
~=[,,_,,]:3
Hi everyone,
As part of the MediaWiki Platform Team's goal to develop a roadmap for
MediaWiki, we want to solicit input on the future direction of
MediaWiki's extension management system. One of MediaWiki's greatest
strengths is the diverse extension ecosystem. But navigating that
ecosystem and managing those extensions can be difficult and cumbersome.
Please provide input on what features you'd like to see at:
<https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_fe…>.
You can suggest a specific feature, or endorse and comment on other
people's proposals. I've put down a few proposal to help start the
discussion, but I'm sure that everyone has plenty more :-).
The goal will be to collate the input provided there, and use it to
plan future development work on improving extension management.
If you know other people who use MediaWiki and have valuable opinions,
please let them know about the RfC.
Thanks,
Kunal / Legoktm
https://www.mediawiki.org/wiki/Scrum_of_scrums/2018-05-16
=2018-05-16=
== Callouts ==
* Fundraising campaigns
https://meta.wikimedia.org/wiki/CentralNotice/Calendar
* Readers Web, Performance: Help reviewing the CitationUsage schema
instrumentation: https://gerrit.wikimedia.org/r/#/c/432534/.
* Modern Event Platform -- Analytics starting interviewing user, anyone
interested should reach Andrew Otto (
https://phabricator.wikimedia.org/T185233)
== Audiences ==
=== Readers ===
==== iOS native app ====
* Blocked by:
* Blocking:
* Updates:
**5.8.1 released; small bug fixes + Wikidata magic word override (
https://phabricator.wikimedia.org/project/view/3339/ )
**Continuing work on tech debt release, 5.8.2 (
https://phabricator.wikimedia.org/project/view/3358/ )
==== Android native app ====
* Blocked by:
* Blocking:
* Updates:
**
==== Readers Web ====
* Blocked by:
* Blocking:
* Updates:
**Working with Services on Proton (Chromium PDF rendering backend service)
instrumentation https://phabricator.wikimedia.org/T189307
**Page issues A/B testing framework in progress
https://phabricator.wikimedia.org/T193584
**Team is at offsite + hackathon so not much else going on
**Full stack engineer backfill hiring continues
**Minor Popups refactoring
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
==== Readers Infrastructure ====
* Blocked by:
** RelEng on https://gerrit.wikimedia.org/r/#/c/432310/
* Blocking:
* Updates:
** Closed out most follow-up work for Short description magic word.
** Reading list browser extensions (Chrome/FF/Safari) coming soon.
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
**[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of
client apps by coalescing and moving more logic to the server]]
***Reading Infra depends on Parsing, Services
===== Maps =====
* Blocked by:
* Blocking:
* Updates:
**
==== Multimedia ====
* Updates
** Hackathon prep impacting some throughput
** OOUI work on WikibaseMediaInfo elements to start soon; UploadWizard
license selection getting OOUI makeover as well
** Search indexing hitting some snags - discussion ongoing
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 3.1]] Prepare
for launch of the first Structured Data on Commons feature (multilingual
file captions)
***SDC depends on Multimedia,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
** [[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 2.1]]
Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Structured_Data#Segment_4:_Programs|Segment
4, Outcome 2]]: Develop a better understanding of existing needs for
Structured Commons- T171252
***Research depends on Multimedia
=== Contributors ===
==== Community Tech ====
* Blocked by:
* Blocking:
* Updates:
** GlobalPreferences live on test wikis, will be slowly rolling it out
further
** Investigating how to integrate PageTriage with ORES and some external
copyvio detection service
==== Anti-Harassment Tools ====
* Blocked by:
* Blocking:
* Updates:
**
==== Editing ====
* Blocked by:
* Blocking:
** Updates:
**
==== Parsing ====
* Blocked by:
* Blocking:
* Updates:
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]] Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
*** Parsing depends on MediaWiki Platform, Services
**[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of
client apps by coalescing and moving more logic to the server]]
Increase code sharing of client apps by coalescing and moving more logic to
the server.
*** Reading Infra depends on Parsing, Services
**
[[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better
data|Outcome 2: Objective 1]]: Revision storage scaling
*** Services depends on SRE, Parsing
==== Collaboration ====
* Blocked by:
* Blocking:
* Updates:
**
==== Language ====
* Blocked by: Core Platform to take a look at:
https://phabricator.wikimedia.org/T151116 - Abuse filters set to warn users
require two captchas
* Blocking: None
* Updates:
** ContentTranslation Version 2 work continue.
** Most of team members will be at Hackathon, so slow week.
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.1]] Improve and consolidate our unified editing platform so that it's
great on all devices
***Language depends on Editing
=== Audiences Design ===
* Blocked by:
* Blocking:
* Updates:
**
==== UI Standardization ====
* Blocked by:
* Blocking:
* Updates:
**
== Technology ==
=== Analytics ===
* Blocked by:
* Blocking:
* Updates:
** Modern Event Platform -- We're starting interviewing user, anyone
interested should reach Andrew Otto (
https://phabricator.wikimedia.org/T185233)
** We have a replacement for pivot called Turnilo (exact same but
opensource) - https://turnilo.wikimedia.org - We'll soon redirect pivot
URLs to turnilo
** Geoeditors available in Hive and Druid -
https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Geoeditors -
Dashboard in Superset (wikitech account + ping to analytics needed)
** virtualpageview_hourly (aggregation of pageviews and page-previews) are
available in hive and druid
** Testing of Druid 0.11 is done, some changes in how we handle streaming
data to be done before rolling out
** Burrow (Kafka monitoring tool) has been upgrade to version 1.1
=== Cloud Services ===
* Blocked by:
* Blocking:
* Updates:
**
=== Fundraising Tech ===
* Blocked by:
* Blocking:
* Updates:
**
=== MediaWiki Platform ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]] Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
** [[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 1.1]] Assist
with deploying MultiContent Revisions on Commons
[[phab:T174022|T174022]] Implement multi-content revisions,
[[phab:T174023|T174023]] Implement MCR storage layer,
[[phab:T174045|T174045]] DB schema migration for MCR,
[[phab:T174044|T174044]] Deploy MCR storage layer,
[[phab:T174043|T174043]]Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
=== Performance ===
* Blocked by:
**
* Blocking:
**
* Updates:
**
=== Release Engineering ===
* Blocked by:
* Blocking:
* Updates:
** Most of the team are in Barcelona this week for the Wikimedia hackathon
as well as a team offsite. Ping twentyafterfour in #wikimedia-releng if you
need assistance.
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption
*** Reading Web depends on SRE, RelEng, Reading Infra
=== Research ===
* Blocked by: None (just waiting for some reviews on schema instrumentation
code)
* Blocking: None
* Updates:
** Finished up the first version of the CitationUsage schema
instrumentation. https://gerrit.wikimedia.org/r/#/c/432534/
** Working on improving the recommendation API algorithm.
https://phabricator.wikimedia.org/T190774
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Structured_Data#Segment_4:_Programs|Segment
4, Outcome 2]]: Develop a better understanding of existing needs for
Structured Commons- [[phab:T171252|T171252]]
***Research depends on Multimedia
=== Scoring Platform ===
* Blocked by:
* Blocking:
* Updates:
=== Search Platform ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|*Objective 2.1]]
Integrate structured file captions into search
*** SDC depends on Search Platform, Multimedia
=== Security ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]]: Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
=== Services ===
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Readers|Increase code sharing of
client apps by coalescing and moving more logic to the server.]]
***Reading Infra/Parsing, Services
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Product#Program_3:_Increase_device_support_for_editing|Goal
3.6]] Support work towards unifying MediaWiki's parser implementations, in
liaison with Technology's MediaWiki team
***Parsing depends on MediaWiki Platform, Services
[[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better
data|Outcome 2: Objective 1]]: Revision storage scaling
*** Services depends on SRE, Parsing
=== Site Reliability Engineering ===
* Blocked by: Collaboration on Flow dumps, T172025
* Blocking: None
* Updates:
** Various SREs will be at the hackathon
** Addressing a mailing list subscription spam attack. For now it's a whack
a mole game, will probably implement some captcha on registration
** Moving forward with mcrouter deployment for multi-dc
https://phabricator.wikimedia.org/T192370
** Debmonitor passed reviews, pending deployment
https://phabricator.wikimedia.org/T191298
** Looking at AES128-SHA usage from toolforge bots
** Proton deployment passed beta by services, proceeding with production
deployment
**
*Quarterly goal dependency update:
**[[metawiki:Wikimedia_Foundation_Annual_Plan/2017-2018/Draft/Programs/Product#Program_2:_Better_Encyclopedia|Outcome
1, Objective 4]]: Continue improving the ways that users can download
articles of interest for later consumption. Proton deployment passed beta
by services, proceeding with production deployment
*** Reading Web depends on SRE, RelEng, Reading Infra
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
[[Wikimedia Technology/Goals/2017-18 Q4#Program 7. Smart tools for better
data|Outcome 2: Objective 1]]: Revision storage scaling
*** Services depends on SRE, Parsing
== Wikidata ==
* Blocked by:
* Blocking:
* Updates:
**
*Quarterly goal dependency update:
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Prepare for launch of the
first Structured Data on Commons feature]] (multilingual file captions)
***SDC depends on Multimedia/,SRE, WMDE, Search Platform, MediaWiki
Platform, Research
**[[Wikimedia Audiences/2017-18 Q4 Goals#Programs|Objective 1.1]] Assist
with deploying MultiContent Revisions on Commons
[[phab:T174022|T174022]] Implement multi-content revisions,
[[phab:T174023|T174023]] Implement MCR storage layer,
[[phab:T174045|T174045]] DB schema migration for MCR,
[[phab:T174044|T174044]] Deploy MCR storage layer,
[[phab:T174043|T174043]]Deploy Multi-Content Revisions
***SDC depends on MediaWiki Platform, WMDE
== German Technical Wishlist ==
* Blocked by:
* Blocking:
* Updates:
**
== SoS Meeting Bookkeeping ==
* Updates:
**
*On 8 May 2018, account hijacking activities were discovered on Wikiviajes
- Spanish Wikivoyage (es.wikivoyage.org <http://es.wikivoyage.org>). It was
identified by community stewards and communicated to the Trust and Safety,
Legal, and Security teams who responded to the event. At this time the
event is still under investigation and we are unable to share more about
what is being done without risking additional hijacking of accounts.
However, we feel it is important to share what details we can and inform
the community of what happened. Similar to past security incidents, we
continue to encourage everyone to take some routine steps to maintain a
secure computer and account - including regularly changing your passwords,
actively running antivirus software on your systems, and keeping your
system software up to date. The Wikimedia Foundation's Security team and
others are investigating this incident as well as potential improvements to
prevent future incidents. We are also working with our colleagues in other
departments to develop plans for how to best share future status updates on
each of these incidents. However, we are currently focused on resolving the
issues identified. If you have any questions, please contact the Trust and
Safety team (ca{{(a)}}wikimedia.org <http://wikimedia.org>). John
BennettDirector of Security, Wikimedia Foundation*
As part of some long-deferred routine maintenance, we need to update
(and, in one case, physically move) the network servers that handle all
traffic between WMCS instances. During each change, all WMCS network
traffic (including network access to all tools and VMs) will be
interrupted for several minutes.
The first outage will be:
Tuesday, May 15 at 13:00 UTC
The second outage will be three hours later:
Tuesday, May 15 16:00 UTC
In each case outages should last no more than ten to fifteen minutes.
More details about this move can be found at
https://phabricator.wikimedia.org/T193579 .
-Andrew
Sorry for cross-posting!
Reminder: Technical Advice IRC meeting again **tomorrow, Wednesday 3-4 pm
UTC** on #wikimedia-tech.
The Technical Advice IRC meeting is open for all volunteer developers,
topics and questions. This can be anything from "how to get started" over
"who would be the best contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting:
https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
Hope to see you there!
Michi (for WMDE’s tech team)
--
Michael F. Schönitzer
Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
http://wikimedia.de
Stellen Sie sich eine Welt vor, in der jeder Mensch an der Menge allen
Wissens frei teilhaben kann. Helfen Sie uns dabei!
http://spenden.wikimedia.de/
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.