For some time now I've thought that possibilities for data visualization in
Wikipedia (or in MediaWiki) are pretty bad. So I'll describe one idea below
and would like to get some feedback. Reason is simple: I'd like to give
this task to one university student so that he could solve that issue.
What I'd like to see is some development, that would make it possible for
user to create visualizations inside MediaWiki. Something so easy that a
child could do that. Like this <https://infogr.am/>. Workflow example: 1)
user selects sth like Create Data Visualization, 2) has some selections
about cart type, colors, etc, 3) place to write down text (title, axes,
description) and 4) a table to fill in with data (values + their text
labels). That could then be saved as one revision. After that every other
user could edit this graph with the same selections and data tables just
like users edit articles and edit history is saved and easy to compare.
Image files like thi
<https://et.wikipedia.org/wiki/Pilt:VikiArtiklitearv.jpg>s or that
<https://commons.wikimedia.org/wiki/File:Tree_map_exports_2010_Estonia.svg> are
ridiculous and fixes like that
<https://en.wikipedia.org/wiki/Template:Pie_chart> are not that flexible,
pretty and easy to use as what we need. So lets move forward. There are
plenty of GPL licensed solutions that could be integrated with MediaWiki.
But I can't be only one thinking about this. So what should I know about
that topic so that this work could really be useful? I.e. how to avoid
reinventing the wheel (like building something already in development) and
how to be sure that it could be easily incorporated later? Who would be the
perfect people to talk about this topic?
Also: are there some very specific tasks within this data visualization
topic that suite well as a research project(s) for an IT student(s)?
Regards,
Ivo Kruusamägi
P.S: I also have some development plans for a web platform that will help
to gamify organizing media files in Wikimedia Commons (coordinates,
categories, descriptions, quality assessment, etc). Sort like adding an
additional data layer and when everything works fine then migrating that
information into Commons. Any great ideas there as well? (not so great
ideas could be sent to list :P )
P.S 2: There is this feedback platform named WikiComment
<http://wikicomment.ut.ee/>. Some testing is need by wikipedians :)
One of the conclusions from the recent SessionManager rollout failure [0] was:
"we should have recruited and coordinated testing by developers and
users inside and outside of the WMF while the code was only on the
beta testing cluster"
SessionManager is back on the WMF beta cluster [1] now after being
briefly removed for the 1.27.0-wmf.12 release cycle, so an
announcement seems in order. The beta cluster implements a SUL
authenticated wiki farm that is completely separate from the Wikimedia
production SUL system. Helping test SessionManager there would involve
logging in, logging out, creating new user accounts and generally
wandering around the wikis doing things you would normally do in
production while keeping an eye out for session related issues.
If you spot something (or just think you spotted something) file a
Phabricator task with as many details as you can provide and tag it
with the #reading-infrastructure-team project. For session related
issues getting traces of the headers and cookies used in the requests
that are failing is most helpful. You can also poke around in the
logging interface [2] to try and find associated error messages.
If you find other bugs, report them in Phabricator too. :)
Also please remember NOT TO USE passwords in the beta cluster that
match the passwords you use anywhere else on the planet!
[0]: https://wikitech.wikimedia.org/wiki/Incident_documentation/20160123-Session…
[1]: http://deployment.wikimedia.beta.wmflabs.org/wiki/Main_Page
[2]: https://logstash-beta.wmflabs.org/#/dashboard/elasticsearch/default
Bryan
--
Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org>
[[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA
irc: bd808 v:415.839.6885 x6855
Hello,
to anyone who is a client of stream.wikimedia.org
(https://wikitech.wikimedia.org/wiki/RCStream), so people who run tools
relying on the RC stream.
In about 48 hours, on February 3rd at 20:00 UTC, we will have to reboot
the backend servers of the stream.wm.org service, rcs1001 and 1002.
This service is loadbalanced and we will only reboot the 2 servers one at a
time,
so there should be no service downtime.
But nevertheless your clients will get disconnected and may need your
intervention to reconnect.
Please be prepared to do so if your client will not automatically reconnect.
I will send another mail once this has happened.
Best regards,
Daniel
--
Daniel Zahn <dzahn(a)wikimedia.org>
Operations Engineer
Hi folks,
In the ArchCom meeting earlier today, Daniel, Timo, Tim and I discussed the
way we handle RFC assignments in Phabricator. Previously, the RFC would
frequently be assigned to person writing the RFC. As we try out the Rust
model (per T123606 <https://phabricator.wikimedia.org/T123606>), and as we
try to increase the speed by which RFCs move though the process, we thought
it would make sense to also assign RFCs to shepherds on the ArchCom.
We didn't discuss all of the implications of this in the meeting today, but
we think this might help us scale our RFC triage process. What do you all
think?
Rob
A security vulnerability has been discovered in MediaWiki setups which
use both Wikibase and MobileFrontend.
All projects in the Wikimedia cluster have been since patched but if
you use these two extensions please be sure to apply the fix.
Patch file and issue are documented on https://phabricator.wikimedia.org/T125684
I came here to post about the same problem. It appears to have broken
this morning. Did the API change again?
On 4 February 2016 at 07:33, Dr. Michael Bonert
<michael(a)librepathology.org> wrote:
> In September 2014, I described an intermittent recurrent problem associated
> with the InstantCommons images.
>
> I hadn't seen the problem seen since April or May of 2015.
>
> It is now back -- after I upgraded to MediaWiki 1.26.2.
> Unlike in the past, it hasn't resolved with a server re-boot.
>
> I did have record traffic earlier in the week... but the problem --like in
> the past-- doesn't seem to be traffic related.
>
> I cannot find a relation to memory. I'm not running out of memory.
> The underlying LAMP (Debian Linux Apache MySQL PHP) stack hasn't changed.
>
> Product Versions
> Operating System: Debian (stable)
> MediaWiki 1.26.2 (f465524)
> PHP 5.6.13-0+deb8u1 (apache2handler)
> MySQL 5.5.44-0+deb8u1
>
> The details of what is installed is here:
> http://librepathology.org/wiki/index.php/Special:Version
>
> I do have git installed -- and am using it to upgrade.
>
> Like in the past-- the image that is local to the site, i.e.
> non-InstantCommons,
> (
> http://librepathology.org/wiki/index.php/File:Atypical_ductal_hyperplasia_-…
> )
> is still there-- and displays properly. At the same time, all the images
> from the InstantCommons are gone.
>
>
> A list of previous posts I did can be found here:
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043340.html
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043345.html
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043348.html
> https://lists.wikimedia.org/pipermail/mediawiki-l/2014-September/043349.html
>
> Help on this would be much appreciated...
>
> I have a testing site (pathologyprotocols.org) that is running the exact
> same set-up (behind a login).
> It is also failing suddenly in the same way; the InstantCommons images are
> gone.
>
> I wonder whether...
> - It is the InstantCommons server
> - There is "talk" between the InstantCommons server and a MediaWiki install
> (with the InstantCommons activated).
> I know this as images that are removed from the InstantCommons... are
> then removed on then
> MediaWiki install with InstantCommons activated.
>
> I think the communication is governed by 'descriptionCacheExpiry' and
> 'apiThumbCacheExpiry'
> https://www.mediawiki.org/wiki/Manual:$wgUseInstantCommons
> ? I wonder whether setting those to infinity would solve anything.
> - I suspect the thumb cache is purged... and then there is a bug preventing
> the image from being
> confirmed as being the same as on the InstantCommons... or there a
> processing bottle neck as the thumbnails have to be re-created.
>
> Thanks in Advance,
> Michael
>
>
> _______________________________________________
> MediaWiki-l mailing list
> To unsubscribe, go to:
> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
Hi,
This is the monthly report from the Wikimedia Performance Team for January
2016.
## Our progress ##
### Multi-datacenter
* The central login system started to use DB slaves for some actions
instead of master DB.
* MediaWiki Special pages and Action classes now support defining DB query
and write expectations (with logging for violations thereof).
* Clean up of haphazard database transaction methods largely finished in
MediaWiki core and extensions.
* Lock acquisition time reduced. Logic was inefficient and resulted in
wasted time. This reduced time spent in backend when saving edits.
* "Rebound purges" enabled in production. To compensate for DB lag, a
secondary purge for articles avoids stale content in Varnish. –
https://phabricator.wikimedia.org/T113192
### Navigation Timing
* We experimenting with creating a new metric "Time to first image" (how
long for the principle image to show). Based on video capture, we were
unable to correlate User Timing API measurements with when an image
actually becomes visible. To be revisited at a later time. –
https://phabricator.wikimedia.org/T115600.
### Media handling (Thumbor)
* Added cgroup support for controlling resource consumption.
* Implemented ability to pre-render multiple thumbnail sizes with a single
request.
* Video thumbnails render faster by loading only the relevant frame from
Swift (not the whole video).
* Working on a strategy for Thumbor deployment in Wikimedia production.
Thumbor is stateless and acts as drop-in replacement for current MediaWiki
PHP image scalers. – https://phabricator.wikimedia.org/T121388
* Our Thumbor plugins have been consolidated into a single Git repo and
moved Gerrit to Phabricator Diffusion.
### ResourceLoader:
* Work has started on the solution for the cache performance problem with
static MediaWiki resources. – https://phabricator.wikimedia.org/T99096
### Metric dashboards
* Earlier this month, Timo Tijhof gave a tech talk on "Creating Useful
Dashboards with Grafana" – https://www.youtube.com/watch?v=UlL6UoRUQAM
* New dashboard: https://grafana.wikimedia.org/dashboard/db/edit-count (Global
edit rate of Wikimedia wikis)
* New dashboard:
https://grafana.wikimedia.org/dashboard/db/time-to-first-byte (Navigation
Timing "responseStart" metric)
## How are we doing? ##
### Metrics
Client-side performance has remained stable over the past month. Save
Timing has also remained stable, around the 1s median mark.
Backend Save Processing Timing has improved slightly and was consistently
45ms (median) and 95ms (p75) lower in January compared to December.
https://performance.wikimedia.org/#!/monthhttps://grafana.wikimedia.org/dashboard/db/save-timing?from=now-50dhttps://grafana.wikimedia.org/dashboard/db/navigation-timing?from=now-50dhttps://grafana.wikimedia.org/dashboard/db/time-to-first-byte?from=now-50d
### Job Queue
On January 1st, the Job Queue started growing rapidly with htmlCacheUpdate
jobs. This was mitigated on 21 January by adding a dedicated runner for
that job type. Total queue size reached over 7 million before the dedicated
runner went live (typical size is under 300K). –
https://phabricator.wikimedia.org/T124194. There is an ongoing
investigation at https://phabricator.wikimedia.org/T124418 about the
increase of those jobs.
https://grafana.wikimedia.org/dashboard/db/job-queue-health?from=now-50d
Until the next time,
Gilles, Peter, Aaron, Ori, and Timo.
https://www.mediawiki.org/wiki/Wikimedia_Performance_Team
Hello,
I am writing to let you know that Pygments, the library we use for
syntax-highlighting, has been updated to the latest release (version 2.1)
in the SyntaxHighlight_GeSHi extension and across Wikimedia wikis. This
release includes support for 53 new languages and language variants.
They are listed below, with the lexer name (the string to use as a value
for the 'lang' attribute of <syntaxhighlight lang=...> tags) in parenthesis:
- ABNF ("abnf")
- ADL ("adl")
- Arduino ("arduino")
- BC ("bc")
- BNF ("bnf")
- Boogie ("boogie")
- CAmkES ("camkes", "idl4")
- CPSA ("cpsa")
- CSS+Mako ("css+mako")
- Component Pascal ("cp", "componentpascal")
- Crmsh ("pcmk", "crmsh")
- Csound Document ("csound-csd", "csound-document")
- Csound Orchestra ("csound", "csound-orc")
- Csound Score ("csound-score", "csound-sco")
- Earl Grey ("earlgrey", "earl-grey", "eg")
- Easytrieve ("easytrieve")
- Elm ("elm")
- Ezhil ("ezhil")
- Fish ("fishshell", "fish")
- FortranFixed ("fortranfixed")
- HTML+Mako ("html+mako")
- Hexdump ("hexdump")
- J ("j")
- JCL ("jcl")
- JavaScript+Mako ("js+mako", "javascript+mako")
- LessCss ("less")
- MSDOS Session ("doscon")
- Mako ("mako")
- ODIN ("odin")
- PacmanConf ("pacmanconf")
- ParaSail ("parasail")
- PkgConfig ("pkgconfig")
- PowerShell Session ("ps1con")
- Praat ("praat")
- QML ("qbs")
- QVTO ("qvt", "qvto")
- Roboconf Graph ("roboconf-graph")
- Roboconf Instances ("roboconf-instances")
- Shen ("shen")
- SuperCollider ("sc", "supercollider")
- TAP ("tap")
- Tcsh Session ("tcshcon")
- Termcap ("termcap")
- Terminfo ("terminfo")
- Terraform ("terraform", "tf")
- Thrift ("thrift")
- TrafficScript ("trafficscript", "rts")
- Turtle ("turtle")
- TypeScript ("typescript")
- X10 ("xten", "x10")
- XML+Mako ("xml+mako")
- cADL ("cadl")
J and BNF were previously supported by mapping them to other languages with
close-enough syntax: Objective-J and Extended BNF, respectively. Each of
those now has a dedicated lexer, so the highlighted output should hew more
closely to the language definition.
For more information on SyntaxHighlight_GeSHi and Pygments, see:
- https://www.mediawiki.org/wiki/Extension:SyntaxHighlight_GeSHi
- http://pygments.org/
Ori Livneh
Perf Team
wikitech.wikimedia.org will be unavailable for a short time on
Friday, beginning around 7AM San Francisco time while we apply kernel
updates and reboot[1]. Ideally the downtime will only last 5-10 minutes
but I've scheduled an hour window on the deploy calendar in case any of
the boxes have trouble reviving.
Labs instances and Toollabs services should not be affected. Some
alerts may fire during the window as we're planning to reboot one of the
monitoring hosts and the effects of that are unpredictable.
In case of emergency, the content of wikitech (minus some images)
is always available on the external backup:
https://wikitech-static.wikimedia.org/wiki/Main_Page
-Andrew
[1] For the curious (and my future reference), we're rebooting silver,
holmium, labcontrol1002, labnet1001, labmon1001
