There've been some issues reported lately with image scaling, where
resource usage on very large images has been huge (problematic for batch
uploads from a high-resolution source). Even scaling time for typical
several-megapixel JPEG photos can be slower than desired when loading up
into something like the MMV extension.
I've previously proposed limiting the generatable thumb sizes and
pre-generating those fixed sizes at upload time, but this hasn't been a
popular idea because of the lack of flexibility and potentially poor
client-side scaling or inefficient network use sending larger-than-needed
fixed image sizes.
Here's an idea that blends the performance benefits of pre-scaling with the
flexibility of our current model...
A classic technique in 3d graphics is
mip-mapping<https://en.wikipedia.org/wiki/Mip-mapping>,
where an image is pre-scaled to multiple resolutions, usually each 1/2 the
width and height of the next level up.
When drawing a textured polygon on screen, the system picks the most
closely-sized level of the mipmap to draw, reducing the resources needed
and avoiding some classes of aliasing/moiré patterns when scaling down. If
you want to get fancy you can also use trilinear
filtering<https://en.wikipedia.org/wiki/Trilinear_filtering>,
where the next-size-up and next-size-down mip-map levels are combined --
this further reduces artifacting.
I'm wondering if we can use this technique to help with scaling of very
large images:
* at upload time, perform a series of scales to produce the mipmap levels
* _don't consider the upload complete_ until those are done! a web uploader
or API-using bot should probably wait until it's done before uploading the
next file, for instance...
* once upload is complete, keep on making user-facing thumbnails as
before... but make them from the smaller mipmap levels instead of the
full-scale original
This would avoid changing our external model -- where server-side scaling
can be used to produce arbitrary-size images that are well-optimized for
their target size -- while reducing resource usage for thumbs of huge
source images. We can also still do things like applying a sharpening
effect on photos, which people sorely miss when it's missing.
If there's interest in investigating this scenario I can write up an RfC
with some more details.
(Properly handling multi-page files like PDFs, DjVu, or paged TIFFs could
complicate this by making the initial rendering extraction pretty slow,
though, so that needs consideration.)
-- brion
Hi folks,
We just enabled Media Viewer by default on the Dutch and French Wikipedias, besides our 14 other pilot sites.
The images look beautiful on both sites, where we’re getting positive comments from many users :)
A big round of thanks to our community partners on both sites: Romaine and Mathonius on the Dutch Wikipedia, as well as Binabik, Jean-Fred and Trizek on the French Wikipedia :)
They kindly announced this to their communities and helped created special discussion pages where we can respond to local user feedback:
https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan#Large_W…
Overall user response so far has been generally favorable across all pilot sites, as shown in our first survey results:
https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Survey#Results
It’s worth noting that approval ratings are increasing on all surveys, after people have a chance to try the tool (e.g.: Hungarian approval was at 42% a week ago, now up to 55%). We will update results again tomorrow, to add more results and comments from Dutch and French users.
It also appears that reduce image load times has been declining, partly due to more people clicking on thumbnails, which puts them into a ‘warm’ cache that loads faster than a ‘cold’ cache, as shown in this graph:
http://multimedia-metrics.wmflabs.org/graphs/mmv_performance_image_global
(Please note that our metrics dashboard are still being updated, so they don’t yet include all the data we’ve collected in recent days, due to a temporary data migration issue, which should be solved shortly.)
For now, we just checked the global Ganglia and Gdash stats, which suggest that everything looks good from an ops and performance standpoint: https://www.mediawiki.org/wiki/Multimedia/Metrics#Ops
Next week, we plan to release Media Viewer on Japanese, Portuguese, Spanish, Swedish and Telugu Wikipedias, as well as Wikimedia Commons. If you are active on any of these sites, we would appreciate your help in introducing this new feature to your community. Please contact Keegan or I so we can coordinate announcements, as well as local discussion pages.
As always, please let us know if you encounter any serious technical issues, or share your views on this discussion page:
https://www.mediawiki.org/wiki/Talk:Multimedia/About_Media_Viewer
Thanks again to everyone for your gracious help on this project!
Enjoy,
Fabrice — for the Multimedia Team
P.S.: New improvements take about 2 weeks to get deployed to all wikis. If you would like to test the latest version of Media Viewer, follow the test tips on this demo page on MediaWiki.org:
https://www.mediawiki.org/wiki/Lightbox_demo
_______________________________
Fabrice Florin
Product Manager, Multimedia
Wikimedia Foundation
https://www.mediawiki.org/wiki/User:Fabrice_Florin_(WMF)
Update from Ori about replication. db1047 is not caught up but data should
be consistent on the one big box. If it's not, please file bugs.
---------- Forwarded message ----------
From: Ori Livneh <ori(a)wikimedia.org>
Date: Thu, May 1, 2014 at 2:29 PM
Subject: Re: [Engineering] [Analytics] EventLogging migration
To: "A mailing list for the Analytics Team at WMF and everybody who has an
interest in Wikipedia and analytics." <analytics(a)lists.wikimedia.org>
Cc: Development and Operations Engineers <engineering(a)lists.wikimedia.org>,
EventLogging-Alerts(a)lists.wikimedia.org
On Wed, Apr 30, 2014 at 7:24 PM, Sean Pringle <springle(a)wikimedia.org>wrote:
> On Thu, May 1, 2014 at 7:54 AM, Dan Andreescu <dandreescu(a)wikimedia.org>wrote:
>
>> And, of course, thanks very much to Sean who is working crazy hours to
>> get this working. Sean - beverage of choice sir, anytime.
>>
>
> Not so much "working", just short bursts or work followed by a lot of
> waiting :-) Databases. Bah!
>
> Note that eventlogging replication has already caught on the One Box,
> analytics-store.eqiad.wmnet; any tools that use the 'research' user and
> read-only queries could consider switching to analytics-store and give
> db1047 time to recover.
>
I finished loading data logged during the migration process into the
database. There should be no gaps at all. db1047 has still not fully caught
up yet, but it is getting closer -- I see it picking up events from the
29th.
_______________________________________________
Engineering mailing list
Engineering(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/engineering
https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/495
There are a few links at the bottom of the current Media Viewer interface -
they say "About Media Viewer | Discuss this feature | Help | Preferences".
There was a request to change "Discuss this feature" to "Discussion" and
I wanted to ask for advice here before we did it.
I'm afraid that "Discussion" will be misinterpreted as being a link to the
file's talk page, and that we'll get feedback about the files (or worse,
the subjects of the files a la AFTv5) on the Media Viewer talk pages.
Does anyone else share that concern? Am I being unnecessarily paranoid?
Justifiably paranoid?
Cheers,
--
Mark Holmquist
Software Engineer, Multimedia
Wikimedia Foundation
mtraceur(a)member.fsf.org
https://wikimediafoundation.org/wiki/User:MHolmquist
Thanks, Pine!
This is beautiful, nice of you to share this link. :)
I’ve looped in the Multimedia mailing list to get comments from the team and community.
Our plate is pretty full in coming months with core projects like Technical Debt, Upload Wizard and Structured Data.
But if there is something simple we can do to help make this type of visualization available to editors, that would be very cool.
Thanks for passing this on ….
Fabrice
On Apr 30, 2014, at 11:46 PM, ENWP Pine <deyntestiss(a)hotmail.com> wrote:
> http://earth.nullschool.net has open-source visualizations of Earth atmospheric and
> oceanic conditions based on periodically updated public data. Using a
> mouse allows the user to zoom and pan the visualizations. Clicking the
> "earth" button shows the options panel. Could the Multimedia team
> integrate this tool into Wikimedia? Imagine how engaging it would be for readers to open articles and find interactive and informative visualizations.
>
> I can imagine similar visualizations in articles for countless other topics on Wikipedia.
>
> Pine
> _______________________________________________
> EE mailing list
> EE(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/ee
_______________________________
Fabrice Florin
Product Manager
Wikimedia Foundation
http://en.wikipedia.org/wiki/User:Fabrice_Florin_(WMF)
On Tue, Apr 29, 2014 at 12:30:57PM -0700, Oliver Keyes wrote:
> Geneally speaking my advice to the multimedia team would be "don't go near
> surveys". I've done a lot of them in the last 3 years, and the one thing
> I've learned is that surveys are very, very difficult to get right. Another
> thing I've learned is that if you don't get them right, the results are
> meaningless and it's hard to tell when that happens.
>
> As I understand it, Jared's team is hiring a qualitatively-focused UX
> researcher or two in the upcoming budget to do research around design and
> feature usage; we should hold off until they come in, first because they're
> simply going to be better at it than we are, and second because it's
> probably going to be frustrating for them if they come in and find a tool
> locked in as How We Do Things (and frustrating for us if they want to
> change that tool):
Hi, Multimedia list!
Just thought I'd cross-post this reply to my call for help with surveys
[0] on the analytics list.
I'm sort of of the mind that skipping the survey for UploadWizard is a
good idea, especially now that I've thought about it more - using a survey
from a third-party site is silly for logged-in users, because logged-in
users will know how to use the talk pages and/or bugzilla.
Thoughts?
[0] http://lists.wikimedia.org/pipermail/analytics/2014-April/001911.html
--
Mark Holmquist
Software Engineer, Multimedia
Wikimedia Foundation
mtraceur(a)member.fsf.org
https://wikimediafoundation.org/wiki/User:MHolmquist
Hi folks,
Here are our plans for the multimedia team’s next development cycle.
This 6-week cycle starts tomorrow, May 1, and ends on June 11 (second half of fiscal Q4), and will include 6 weekly sprints. In this next cycle, we plan to switch our focus to Upload Wizard and Technical Debt, while addressing Media Viewer's most critical issues, as needed.
For this cycle, we propose to divide our time evenly between these three main projects:
* Media Viewer - address remaining critical tasks (e.g. serious bugs, global load estimates, basic zoom feature)
* Technical Debt - fix serious issues that need quick solutions (e.g. image scalers, timed media handler update)
* Upload Wizard - start planning for this project (metrics, feedback, UX designs, code review, unit tests)
In subsequent cycles, we expect work on Upload Wizard and Technical Debt to continue to be a primary focus for us. We also hope to start work on Structured Data and File Notifications this summer, as time allows.
Here are our meeting notes, for your review:
https://www.mediawiki.org/wiki/Multimedia/Meetings/Next_Cycle_2_Q4_2013-14
And here is our updated wall for what is now our current cycle:
http://ur1.ca/h7w5s
This cycle wall now includes separate columns for Media Viewer (~28 points), Tech Debt (~29 points) and Upload Wizard (~21 points). We moved all leftover Media Viewer cards into either this cycle wall -- or in one of the Media Viewer Release walls in Team Favorites (see upper right corner of Mingle site). We could easily spend another quarter or two working on Media Viewer, but our team is needed on other projects that were neglected during this big product push. For the next few weeks, we will continue to front-load Media Viewer tasks to address critical issues from our worldwide deployment, but then start shifting more and more to Technical Debt and Upload Wizard projects.
The meeting notes also include results of the previous cycle and a team retrospective, which will give you a sense of the agile development process we have been using in recent months. We have found this agile process invaluable, and we note that it has helped improve our overall performance, so we are now getting more work done, more efficiently. Music to our ears … :)
Please let us know if you have any questions about these plans. We will be sure to consult you as we start new projects like Upload Wizard -- and Gilles will update you shortly on our current sprint, which we just planned today.
Onward!
Fabrice
for the Multimedia Team
_______________________________
Fabrice Florin
Product Manager, Multimedia
Wikimedia Foundation
https://www.mediawiki.org/wiki/User:Fabrice_Florin_(WMF)