I came here to post about the same problem. It appears to have broken
this morning. Did the API change again?
On 4 February 2016 at 07:33, Dr. Michael Bonert
<michael(a)librepathology.org> wrote:
> In September 2014, I described an intermittent recurrent problem associated
> with the InstantCommons images.
>
> I hadn't seen the problem seen since April or May of 2015.
>
> It is now back -- after I upgraded to MediaWiki 1.26.2.
> Unlike in the past, it hasn't resolved with a server re-boot.
>
> I did have record traffic earlier in the week... but the problem --like in
> the past-- doesn't seem to be traffic related.
>
> I cannot find a relation to memory. I'm not running out of memory.
> The underlying LAMP (Debian Linux Apache MySQL PHP) stack hasn't changed.
>
> Product Versions
> Operating System: Debian (stable)
> MediaWiki 1.26.2 (f465524)
> PHP 5.6.13-0+deb8u1 (apache2handler)
> MySQL 5.5.44-0+deb8u1
>
> The details of what is installed is here:
> http://librepathology.org/wiki/index.php/Special:Version
>
> I do have git installed -- and am using it to upgrade.
>
> Like in the past-- the image that is local to the site, i.e.
> non-InstantCommons,
> (
> http://librepathology.org/wiki/index.php/File:Atypical_ductal_hyperplasia_-…
> )
> is still there-- and displays properly. At the same time, all the images
> from the InstantCommons are gone.
>
>
> A list of previous posts I did can be found here:
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043340.html
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043345.html
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043348.html
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043349.html
>
> Help on this would be much appreciated...
>
> I have a testing site (pathologyprotocols.org) that is running the exact
> same set-up (behind a login).
> It is also failing suddenly in the same way; the InstantCommons images are
> gone.
>
> I wonder whether...
> - It is the InstantCommons server
> - There is "talk" between the InstantCommons server and a MediaWiki install
> (with the InstantCommons activated).
> I know this as images that are removed from the InstantCommons... are
> then removed on then
> MediaWiki install with InstantCommons activated.
>
> I think the communication is governed by 'descriptionCacheExpiry' and
> 'apiThumbCacheExpiry'
> https://www.mediawiki.org/wiki/Manual:$wgUseInstantCommons
> ? I wonder whether setting those to infinity would solve anything.
> - I suspect the thumb cache is purged... and then there is a bug preventing
> the image from being
> confirmed as being the same as on the InstantCommons... or there a
> processing bottle neck as the thumbnails have to be re-created.
>
> Thanks in Advance,
> Michael
>
>
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Hi,
This is the monthly report from the Wikimedia Performance Team for January
2016.
## Our progress ##
### Multi-datacenter
* The central login system started to use DB slaves for some actions
instead of master DB.
* MediaWiki Special pages and Action classes now support defining DB query
and write expectations (with logging for violations thereof).
* Clean up of haphazard database transaction methods largely finished in
MediaWiki core and extensions.
* Lock acquisition time reduced. Logic was inefficient and resulted in
wasted time. This reduced time spent in backend when saving edits.
* "Rebound purges" enabled in production. To compensate for DB lag, a
secondary purge for articles avoids stale content in Varnish. –
https://phabricator.wikimedia.org/T113192
### Navigation Timing
* We experimenting with creating a new metric "Time to first image" (how
long for the principle image to show). Based on video capture, we were
unable to correlate User Timing API measurements with when an image
actually becomes visible. To be revisited at a later time. –
https://phabricator.wikimedia.org/T115600.
### Media handling (Thumbor)
* Added cgroup support for controlling resource consumption.
* Implemented ability to pre-render multiple thumbnail sizes with a single
request.
* Video thumbnails render faster by loading only the relevant frame from
Swift (not the whole video).
* Working on a strategy for Thumbor deployment in Wikimedia production.
Thumbor is stateless and acts as drop-in replacement for current MediaWiki
PHP image scalers. – https://phabricator.wikimedia.org/T121388
* Our Thumbor plugins have been consolidated into a single Git repo and
moved Gerrit to Phabricator Diffusion.
### ResourceLoader:
* Work has started on the solution for the cache performance problem with
static MediaWiki resources. – https://phabricator.wikimedia.org/T99096
### Metric dashboards
* Earlier this month, Timo Tijhof gave a tech talk on "Creating Useful
Dashboards with Grafana" – https://www.youtube.com/watch?v=UlL6UoRUQAM
* New dashboard: https://grafana.wikimedia.org/dashboard/db/edit-count (Global
edit rate of Wikimedia wikis)
* New dashboard:
https://grafana.wikimedia.org/dashboard/db/time-to-first-byte (Navigation
Timing "responseStart" metric)
## How are we doing? ##
### Metrics
Client-side performance has remained stable over the past month. Save
Timing has also remained stable, around the 1s median mark.
Backend Save Processing Timing has improved slightly and was consistently
45ms (median) and 95ms (p75) lower in January compared to December.
https://performance.wikimedia.org/#!/monthhttps://grafana.wikimedia.org/dashboard/db/save-timing?from=now-50dhttps://grafana.wikimedia.org/dashboard/db/navigation-timing?from=now-50dhttps://grafana.wikimedia.org/dashboard/db/time-to-first-byte?from=now-50d
### Job Queue
On January 1st, the Job Queue started growing rapidly with htmlCacheUpdate
jobs. This was mitigated on 21 January by adding a dedicated runner for
that job type. Total queue size reached over 7 million before the dedicated
runner went live (typical size is under 300K). –
https://phabricator.wikimedia.org/T124194. There is an ongoing
investigation at https://phabricator.wikimedia.org/T124418 about the
increase of those jobs.
https://grafana.wikimedia.org/dashboard/db/job-queue-health?from=now-50d
Until the next time,
Gilles, Peter, Aaron, Ori, and Timo.
https://www.mediawiki.org/wiki/Wikimedia_Performance_Team
Hello,
I am writing to let you know that Pygments, the library we use for
syntax-highlighting, has been updated to the latest release (version 2.1)
in the SyntaxHighlight_GeSHi extension and across Wikimedia wikis. This
release includes support for 53 new languages and language variants.
They are listed below, with the lexer name (the string to use as a value
for the 'lang' attribute of <syntaxhighlight lang=...> tags) in parenthesis:
- ABNF ("abnf")
- ADL ("adl")
- Arduino ("arduino")
- BC ("bc")
- BNF ("bnf")
- Boogie ("boogie")
- CAmkES ("camkes", "idl4")
- CPSA ("cpsa")
- CSS+Mako ("css+mako")
- Component Pascal ("cp", "componentpascal")
- Crmsh ("pcmk", "crmsh")
- Csound Document ("csound-csd", "csound-document")
- Csound Orchestra ("csound", "csound-orc")
- Csound Score ("csound-score", "csound-sco")
- Earl Grey ("earlgrey", "earl-grey", "eg")
- Easytrieve ("easytrieve")
- Elm ("elm")
- Ezhil ("ezhil")
- Fish ("fishshell", "fish")
- FortranFixed ("fortranfixed")
- HTML+Mako ("html+mako")
- Hexdump ("hexdump")
- J ("j")
- JCL ("jcl")
- JavaScript+Mako ("js+mako", "javascript+mako")
- LessCss ("less")
- MSDOS Session ("doscon")
- Mako ("mako")
- ODIN ("odin")
- PacmanConf ("pacmanconf")
- ParaSail ("parasail")
- PkgConfig ("pkgconfig")
- PowerShell Session ("ps1con")
- Praat ("praat")
- QML ("qbs")
- QVTO ("qvt", "qvto")
- Roboconf Graph ("roboconf-graph")
- Roboconf Instances ("roboconf-instances")
- Shen ("shen")
- SuperCollider ("sc", "supercollider")
- TAP ("tap")
- Tcsh Session ("tcshcon")
- Termcap ("termcap")
- Terminfo ("terminfo")
- Terraform ("terraform", "tf")
- Thrift ("thrift")
- TrafficScript ("trafficscript", "rts")
- Turtle ("turtle")
- TypeScript ("typescript")
- X10 ("xten", "x10")
- XML+Mako ("xml+mako")
- cADL ("cadl")
J and BNF were previously supported by mapping them to other languages with
close-enough syntax: Objective-J and Extended BNF, respectively. Each of
those now has a dedicated lexer, so the highlighted output should hew more
closely to the language definition.
For more information on SyntaxHighlight_GeSHi and Pygments, see:
- https://www.mediawiki.org/wiki/Extension:SyntaxHighlight_GeSHi
- http://pygments.org/
Ori Livneh
Perf Team
wikitech.wikimedia.org will be unavailable for a short time on
Friday, beginning around 7AM San Francisco time while we apply kernel
updates and reboot[1]. Ideally the downtime will only last 5-10 minutes
but I've scheduled an hour window on the deploy calendar in case any of
the boxes have trouble reviving.
Labs instances and Toollabs services should not be affected. Some
alerts may fire during the window as we're planning to reboot one of the
monitoring hosts and the effects of that are unpredictable.
In case of emergency, the content of wikitech (minus some images)
is always available on the external backup:
https://wikitech-static.wikimedia.org/wiki/Main_Page
-Andrew
[1] For the curious (and my future reference), we're rebooting silver,
holmium, labcontrol1002, labnet1001, labmon1001
= 2015-02-03 =
== Product ==
=== Reading ===
==== Web ====
* New related pages desktop design pushed, will measure engagement
* Instrumentation in prep for language switcher change [analytics]
* new user page designs on mobile web beta going live
* Extension:Gather - PageImages showing non-free images - meeting
* scheduled to iron out API request/response format to first try in
* Related Articles on web to adjust output
==== Android ====
* New beta coming soon with improved memory usage for images and support
* for animations. Also includes A/B testing for CirrusSearch (T125393).
==== iOS ====
* Will be integrating "top articles" feature using pageviews API
* Looking forward to mobileview API change from web team to get article
* namespaces!
==== Reading Infrastructure ====
* Nothing much this week. SessionManager should be coming back into
* master once wmf.12 is verified not to break stuff.
=== Community Tech ===
* No update.
=== Editing ===
==== Collaboration ====
* '''Blocking''':
** Dry run patch for external store migration is merged. Now we need to
set External Store up on Beta, then test the dry run patch there:
https://phabricator.wikimedia.org/T119567
* '''Blocked''':
** Flow dump generation on dumps.wikimedia.org:
https://phabricator.wikimedia.org/T119511
* '''Updates''':
** We're still working on human-readable names for cross-wiki
notifications: https://phabricator.wikimedia.org/T121936
** MediaWiki presence at FOSDEM went well.
==== Language ====
* No update.
==== Multimedia ====
* '''Blocking''': none
* '''Blocked''': none
* '''Updates''':
** Work on image tweaks extension continues; may need input later.
==== Parsing ====
* '''Blocking''': none
* '''Blocked''': Need input from Collaboration, see below.
* '''Updates''':
** ruthenium updated to jessie and node 4.2 with almost everything
puppetized (thanks to ops, Marko, Ori). Tests look good. Need to run
separate memory load tests before making a decision to move production
parsoid to node 4.2
** Need input from collaboration team about
https://phabricator.wikimedia.org/T124837 (migrating Flow to talk with
RESTBase) since it will simplify support when we remove inlined data-mw
from Parsoid HTML (I consider this ticket a weak blocker)
*** Matt: Should we schedule a meeting about this? Sure .. let us do it
this week.
** Will work with services team to finalize REST API versioning policy
this week -- last chance to provide input on
https://phabricator.wikimedia.org/T124365 ... Policy will be applied
when Parsoid HTML version is changed after inlined data-mw is moved out
of Parsoid HTML
** Heads up (VE, Language, Flow): We might be able to deploy
templatedata-based serialization of transclusions next week (depends on
reviews this week).
*** https://gerrit.wikimedia.org/r/#/c/264043/ if you want to test your
respective clients against it (
https://gerrit.wikimedia.org/r/#/c/264043/16/tests/mocha/templatedata.js
and https://gerrit.wikimedia.org/r/#/c/264043/16/tests/mockAPI.js has
tests that spec behavior if you want to take a closer look)
==== VisualEditor ====
* '''Blocking''': none known
* '''Blocked''':
** https://phabricator.wikimedia.org/T58337 being worked on in review
from Krinkle for https://gerrit.wikimedia.org/r/#/c/259771/ and
https://gerrit.wikimedia.org/r/#/c/265878/ and so
https://gerrit.wikimedia.org/r/#/c/265879/
* '''Updates''':
** Released yesterday (wmf.12), editing via jQuery.IME (thanks to
Language for their support); table editing improvements (move
columns/rows; copy-paste multiple cells; make/unmake tables
sortable/wikitable; cell and table contexts)
** wmf.13 will contain a split up version of OOUI; see
https://phabricator.wikimedia.org/T113677 for work on this and some
numbers. This is not a breaking change except for non-MediaWiki users of
OOUI like VE, for whom we'll flag this.
=== Discovery ===
* Data import from analytics to ES started
* Working on integrating completion suggester for all prefix searches
* (will involve some small API changes in SearchEngine)
* TextCat is ready for inclusion in mediawiki/vendors, waiting for final
* security signoff
* Preparing for A/B test to use opening_text instead of text in morelike
* query, to improve performance
* Upgrading Wikidata Query Service to Blazegraph 2.0, so far working ok
* but some weird exceptions, investigating
* '''Blocking''': none
* '''Blocked''': security final signoff for textcat
== Technology ==
=== Analytics ===
* Dashiki: implemented limn-like layout, will end-of-life most limn
* dashboards soon
* Event Logging: problems were largely due to large tables, getting
* better as we're trimming some of those
* Wikimetrics: finished program metrics feature, deploying soon
* Jobs to count Uniques based on the Last-Access cookie are being
* productionized, will be available soon
* Bot convention thread on analytics-l concluded with us asking
* non-human user agents to include the word "Bot", for analytics
* purposes. We'll communicate that here soon:
* https://meta.wikimedia.org/wiki/User-Agent_policy
=== Performance ===
* No update.
=== Release Engineering ===
* Blocking:
** Phase out antimony.wikimedia.org,
https://phabricator.wikimedia.org/T123718
** /var/log/phd/damons.log growing on iridium,
https://phabricator.wikimedia.org/T124651
* Blocked: none
* Updates:
** 1.27.0-wmf.11 was skipped (and burned in effigy)
** 1.27.0-wmf.12 delayed by staging issues yesterday but train is
starting today
*** fun firefighting due to /srv/mediawiki-staging being wiped out
*** wmf.12 does not contain SessionManager due to an outstanding bug
that remains unreproducible
** scap 3.0 tagged and packaged! thanks ops!
** Finishing up work on puppet scap provider
=== Research ===
* No update.
=== Security ===
* No update.
=== Services ===
* Decommissioning parsoid-lb.wikimedia.org around Feb 22 --
* https://phabricator.wikimedia.org/T110474
:* use RESTBase instead
* RESTBase
:* added streaming support
:* refactor: separating out the framework part of RESTBase
:* minor tweaks and bug fixes
* EventBus
:* enabled on all wikis (module private ones)
:* final tweaks to the schemas -
https://phabricator.wikimedia.org/T124741
=== Technical Operations ===
* No update.
== Advancement ==
=== Fundraising Tech ===
* Adam Wight and Andrew Green taking the month off fr-tech to work on
* Education Program extension
* Got CI jobs running against paymentswiki branch of mediawiki (voting)
* with 1.27 non-voting (thank you releng)
* more CiviCRM enhancements
* fixes and enhancements for backup credit card processor
* prep for Latin America fundraising expansion
* investigating banner impression data outage that started yesterday
* (https://phabricator.wikimedia.org/T125676)
--
| Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E |
| identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Google Code-in 2015 has come to an end.
Thanks to our students for resolving 461 Wikimedia tasks. Thanks to our
35 mentors for being available, also on weekends & holidays. Thanks to
everybody on IRC for your friendliness, patience, and help provided to
new contributors.
Some more achievements, apart from those already mentioned in
https://lists.wikimedia.org/pipermail/wikitech-l/2015-December/084421.html :
* The CommonsMetadata extension parses vcards in the src field
* The MediaWiki core API exposes "actual watchers" as in "action=info"
* MediaWiki image thumbnails are interlaced whenever possible
* Kiwix is installable/moveable to the SD card, automatically opens
the virtual keyboard for "find in page", (re)starts with the last
open article
* imageinfo queries in MultimediaViewer are cached
* Twinkle's set of article maintenance tags was audited and its XFD
module has preview functionality
* The RandomRootPage extension got merged into MediaWiki core
* One can remove items from Gather collections
* A new MediaWiki maintenance script imports content from text files
* Pywikibot has action=mergehistory support implemented
* Huggle makes a tone when someone writes something
* Many i18n issues fixed and strings improved
* Namespace aliases added to MediaWiki's export dumps
* The Translate extension is compatible with PHP 7
The Grand Prize winners & finalists will be announced on February 8th.
Again congratulations everybody, and thanks for the hard work.
See you around on IRC, mailing lists, Gerrit, and Phabricator!
Cheers,
andre
--
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/
---------- Forwarded message ----------
From: Kevin Leduc <kevin(a)wikimedia.org>
Date: Tue, Feb 2, 2016 at 4:22 PM
Subject: February 2016 Lightning Talks
To: "Staff (All)" <wmfall(a)lists.wikimedia.org>
Hi All,
The next Lightning Talks are scheduled for February 16th (two weeks from
today). We hope at least 4 people will sign up for the talks by Friday
February 12th otherwise we will postpone them another month. Lightning
Talks are an opportunity for teams @ WMF & in the Community to showcase
something they have achieved: a quarterly goal, milestone, release, or
anything of significance to the rest of the foundation and the movement as
a whole.
Each presentation will be 10 minutes or less including time for questions.
Sign up here: https://www.mediawiki.org/wiki/Lightning_Talks#February_2016
Next round of Lightning Talks:
When: Tuesday February 16, 1900 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?msg=Lightning+Talks&is…>,
11am PST (We have added this Lightning Talk to the WMF Engineering, Fun &
Learning, and Staff calendars)
Where: 5th Floor
Remotees: On-Air google hangout will be provided just before the meeting
IRC: #wikimedia-tech
YouTube stream: http://www.youtube.com/watch?v=D3fyCgBWvFc
Thanks!
Kevin Leduc, Megan Neisler, Brendan Campbell
The Winter break took longer than expected. Busy times!
You can help develop the next summary.
*Developer Relations focus*
* Wrapping up Google Code-in 2015
* Wikimedia Hackathon 2016 travel sponsorship requests
* Wikimedia Developer Summit 2016/Lessons Learned
* Developer Relations strategy and annual plan
There is a lot more at
https://www.mediawiki.org/wiki/Developer_Relations/Weekly_summary#2016-01-26
--
Quim Gil
Engineering Community Manager @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
Cool. Forwarding this email.
Pine
On Tue, Feb 2, 2016 at 10:30 AM, Jonathan Morgan <jmorgan(a)wikimedia.org>
wrote:
> Yesterday, 2013 IEG grantee Jeph Paul started seeing 1000s of hits on his
> (grant-funded, volunteer-maintained) ReplayEdits tool which visually
> replays edit histories of Wikipedia articles.
>
> Here's why:
>
> http://www.nytimes.com/2016/02/02/us/politics/wikipedia-donald-trump-2016-e…
> (the tool <https://cosmiclattes.github.io/wikireplay/player.html> is
> linked from one of the last few paragraphs in the article)
>
> The article itself is not exceptional. It's just cool to see one of our
> locally grown research prototypes get good press.
>
> Jonathan
>
> --
> Jonathan T. Morgan
> Senior Design Researcher
> Wikimedia Foundation
> User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>
>
>
> _______________________________________________
> Wiki-research-l mailing list
> Wiki-research-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wiki-research-l
>
>
I've been working on a little redesign project for the Main Page on
wikitech [0] and three key sub pages it points to since 2016-01-01 in
my User space. Tonight I decided that although it is far from perfect
it is better enough. I hope that some of you like it better than the
old page and that none of you hate it with a fiery passion that
compels you to revert it rather than helping me make it better.
[0]: https://wikitech.wikimedia.org/wiki/Main_Page
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA
irc: bd808 v:415.839.6885 x6855
