Hey,
I'm happy to announce the first release of a new little extension I wrote
called Diff.
https://www.mediawiki.org/wiki/Extension:Diff
It's a small utility library which might be of use to anyone creating a new
extension :)
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
So it's time to have this discussion again. At least, I think we're
having it again, though I could not find previous threads on this list
about the subject.
In short, scaled media is currently generated on the fly for any size
and for any user. The resulting files are kept around forever or until
we run perilously short of space, at which point we make some guesses
about what we can toss and then do a mass purge. Last time we did so, we
had the rotation bug going at the same time, which made for a real fine
mess.
A little bit of crunching shows me that we have about 6 million images
in use on the projects, and yet we manage to have around 130 million
thumbnails. Just for fun I checked to see how many thumbs each image
has, what sizes we are looking at, etc. Here's the results.
Some "standard" sizes are most popular, with between 200K and 640K media
files having thumbs scaled to each of these widths:
75, 120, 150, 180, 200, 220, 320, 640, 800, 1024, and 1280 pixels
But there's plenty of "odd" sizes with lots of thumbs too. For example,
over 65K files with width 181px, 20K with width 138px.
As an experiment and before having this data, I purged from ms5 (no
longer in use for thumbs) 1/16 of the thumbs that were greater than
100px wide but not one of these widths:
120px, 200px, 220px, 250px, 320px, 640px, 800px
We got back over 300GB of space.
The other thing about delivering any scaled version on demand is that we
have some media files with several hundred different thumb sizes in
there. Here's a few of the top offenders for your entertainment:
2514 wikipedia/commons/thumb/f/f9/Orange_and_cross_section.jpg
2285 wikipedia/commons/thumb/f/fb/Thrermal_grease.jpg
2218 wikipedia/commons/thumb/f/fc/Blue_sport.jpg
2071 wikipedia/commons/thumb/f/f3/Flag_of_Switzerland.svg
2062 wikipedia/commons/thumb/f/f2/Flag_of_Costa_Rica.svg
2034 wikipedia/commons/thumb/f/f8/Wiktionary-logo-en.svg
1915 wikipedia/commons/thumb/f/f6/VeulesLesRoses.JPG
1689 wikipedia/commons/thumb/f/fa/Wikibooks-logo.svg
1447 wikipedia/commons/thumb/f/fa/Wikiquote-logo.svg
1371 wikipedia/commons/thumb/f/f0/Mori_Uncanny_Valley.svg
1249 wikipedia/commons/thumb/f/f5/Grand_prismatic_spring.jpg
1246 wikipedia/commons/thumb/f/f3/Mature.jpg
1191 wikipedia/commons/thumb/f/f7/Kirchdorf_in_Tirol.JPG
1187 wikipedia/commons/thumb/f/f8/Camille_Cabral_pour_les_Trans.JPG
1143 wikipedia/commons/thumb/f/f7/Profanity.svg
1079 wikipedia/commons/thumb/f/f2/HSV_color_solid_cone.png
1040 wikipedia/commons/thumb/f/f2/Carmen_Electra.jpg
1032 wikipedia/commons/thumb/f/f1/Pink_eye.jpg
1001 wikipedia/commons/thumb/f/f6/USNS_Medgar_Evers_announcement.jpg
I'd comment on some of those but I'd be too snarky.
So there are some things we could change:
1. We could generate and keep only certain sizes, tossing the rest.
2. We could keep *nothing*, scaling all media as required.
3. We could have a cron job that was clever about tossing thumbs every
day (not sure how easy it would be to be clever).
4. ??
In any of these cases, the squids will have copies of recently requested
scaled media, so we won't be scaling the same file to the same size over
and over in a short time frame.
What do folks think about how to proceed?
Ariel
Hey,
Is it possible to push to a branch other then the default one on gerrit
using git review?
This is needed when you want to have more then one branch on which you have
reviewed code, or if you want different levels of review. For example if
you want a novice committer play around with an extension a bit and push
new functionality that gets reviewed but is not ready to go onto master
until it really has stabilized and finalized.
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--
While going through the bugs that were marked as 1.20 tarball blockers
today, most seemed to have been left to rot -- that is, they didn't
really deserve the tag since people were not motivated to fix them and
the people who marked them 1.20 tarball blockers didn't put in the
effort to find someone to fix them.
Sometimes, that person was me, so this shouldn't be seen as overly harsh
criticism.
I did see a couple of exceptions:
* https://bugzilla.wikimedia.org/38273 -- Tidy occasionally isn't
executed
Seems to be happening somewhat regularly since July on different WMF
sites. Some effort to track it down has been made, but no solution has
been found yet. I've a feeling this bug would bite third-party users of
MW if we released a tarball with it.
* https://bugzilla.wikimedia.org/35894 -- Reports of secret key
generation "hanging" on windows`
This is a bug that will really affect only new tarball users on Windows.
While this probably isn't a lot of people, I would still like to see it
fixed before a tarball is released. MaxSem had some ideas and I'll talk
to him about how feasibly he can implement them.
Otherwise, there are several patches mostly by Krinkle that need to be
merged for a 1.20 tarball:
https://bugzilla.wikimedia.org/31676 - ResourceLoader should work
around IE stylesheet limit
https://bugzilla.wikimedia.org/38158 - jquery.byteLimit sometimes
causes an unexpected 0 maxLength being enforced
https://bugzilla.wikimedia.org/40448 - Replace
mediawiki.legacy.mwsuggest with SimpleSearch.
https://bugzilla.wikimedia.org/40500 - ResourceLoader should not ignore
media-type for urls in debug mode
https://bugzilla.wikimedia.org/35923 - Followup to bug 11374 - tweaks
to mediawiki.action.history.diff.css
https://bugzilla.wikimedia.org/40498 - ResourceLoader should not output
an empty "@media print { }" block.
I still plan on being in #wikimedia-dev at UTC1100 on Tuesday, October 2
(See [0] for timezone conversion) for the triage, but I'll probably be
spending that time putting in merge requests for the above issues
(unless someone beats me to it).
Also Tuesday, I'll make a new tarball with the code merged as my final
release candidate for 1.20. Sam has said that because we're doing a
review-first model with Gerrit people should feel more liberty to put
merge requests in, so have at it! [1]
Mark.
[0-short] <http://hexm.de/lr>
[0-long]
<http://www.timeanddate.com/worldclock/meetingdetails.html?year=2012&month=1…>
[1-short] <http://hexm.de/lw>
[1-long]
<https://gerrit.wikimedia.org/r/#/q/status:open+project:mediawiki/core+branc…>
--
http://hexmode.com/
Necessitous men are not, truly speaking, free men.
-- Vernon v Bethell (1762), quoted by FDR's Second Bill of Rights
Hi there,
Asking for a task to volunteer, Sumana encouraged me to look at the
topic of community metrics. She pointed to
https://wikitech.wikimedia.org/view/Pentaho as a starting point.
After a first look at Pentaho and what some colleagues at the MeeGo
project did with it [1], I searched (a bit) for any wiki pages or
discussions about community metrics here, but couldn't find any.
http://www.mediawiki.org/wiki/User:Qgil/MediaWiki_Community_Metrics
Edits welcome!
I'm looking for feedback, help, and a first prototype of an
automatically refreshed report hopefully sooner than later. Something
simple to build upon.
Even if it's too tempting to define the first prototype thinking first
on tools or data available, you are encouraged to start by proposing
what questions do you want actually answered. What community trends do
you want to know?
See
http://www.mediawiki.org/wiki/User:Qgil/MediaWiki_Community_Metrics#Trends_…
- in few days we should have agreed on the first and most important
trends we want to visualize.
[1] http://wiki.meego.com/Metrics/Dashboard
--
Quim
When I was hired as QA Lead almost seven months ago, WMF lacked a test
environment where
* code was routinely deployed ahead of production
* the test environment emulated the production environment closely
* aspects of the test environment (config, permissions, etc.) could be
easily and reliably manipulated for testing purposes
Today I am happy to announce that beta labs fulfills those needs.
Beta labs is intended to host the upcoming release of Mediawiki, plus those
extensions scheduled for deployment to production, for the purpose of
testing and investigation.
As of a little while ago, Mediawiki, AFTv5, New Pages Feed/Page Curation,
and UploadWizard are being deployed to beta labs from git automatically and
reliably. The configurations for those extensions are also being managed
in git. The environment itself is managed via puppet, and emulates
production to the greatest extent possible. Many many thanks to Antoine
Musso for making this possible.
As of this week, all these extensions are up, running, and configured to be
useful. Note that they are not perfect, just useful. For example, right
now on beta enwiki both AFTv4 and AFTv5 input forms appear on the same page
in many cases, because I was experimenting with what happens when these
extensions are not configured correctly. Some actions from the Page
Curation toolbar never complete. As these glitches become important to
testing, we will get them working correctly, and likely will find out some
interesting things about the software along the way.
The timing for this announcement is excellent, because new QA Engineers
will be joining WMF soon (more on that next week), and beta labs will be a
prime target for the browser-level end-to-end automated tests we will
shortly be creating. Also, we have been wanting to retire the 'prototype'
host for some time, and having AFTv5 etc. on beta labs should make that
possible.
In summary, beta labs is up and running with current code for Mediawiki and
critical extensions, and at this point the best way to improve beta labs is
to use it.
http://en.wikipedia.beta.wmflabs.org/wiki/Special:ArticleFeedbackv5http://en.wikipedia.beta.wmflabs.org/wiki/Special:NewPagesFeedhttp://commons.wikimedia.org/wiki/Special:UploadWizard
Hi everyone!
I have found myself in the following situation several times: I
created a wiki for some event or small project, everything works fine
and after the event or project was done - nobody have seen this wiki
for several months and does nothing on it. After several months
somebody needs the wiki once again and realizes that the wiki database
now have 3 Gb of text spam. Suppose that there is no back-up or
rollback option in a wiki hosting. So here is the question: how to
1) remove all the spam
2) delete all the spam accounts
3) reduce the database size from 3Gb to the original size
Cheers,
Yury Katkov, WikiVote
Dear all,
Starting October 1, 2012, translatewiki.net will drop support for all
MediaWiki extensions it currently supports that still remain in the
Subversion repository svn.wikimedia.org.
Many, if not all of the extensions that by that time have not been
migrated to git/Gerrit, are clearly not that well maintained and we want
to prevent our translators from spending their time on it. At the moment,
a maximum of 273 extensions are affected.
In case this message makes you wonder what git and Gerrit is all about,
and how you can revive your precious extension, please see the following
links:
* https://www.mediawiki.org/wiki/Git/Tutorial
* https://www.mediawiki.org/wiki/Git/Conversion/Extensions_queue
Cheers!
Siebrand