FYI!
---------- Forwarded message ----------
From: Yuvi Panda <yuvipanda(a)wikimedia.org>
Date: Tue, Aug 25, 2015 at 12:59 AM
Subject: PuppetSWAT, coming to a patch near you!
To: Development and Operations Engineers <engineering(a)lists.wikimedia.org>
Hello!
MediaWiki SWAT deploys(
https://wikitech.wikimedia.org/wiki/SWAT_deploys) are a very popular
way to get config changes deployed. In an effort to trim down the ops
review queue and encourage more non-opsen to write puppet patches,
we're experimenting with a similar setup for Puppet patches,
creatively called - PuppetSWAT!
https://wikitech.wikimedia.org/wiki/PuppetSWAT has more preliminary
details.
The tl;dr is that if you have patches to operations/puppet.git, you
can now put them in the Deployment Calendar
(https://wikitech.wikimedia.org/wiki/Deployments) and get them looked
at / merged. Just like SWAT, not all patches are suitable for Puppet
SWAT - see https://wikitech.wikimedia.org/wiki/PuppetSWAT#What_kind_of_patches_can_go_…
for guidelines and examples.
This is an experiment that'll go on for a few weeks, in an attempt to
encourage non opsen to write operations/puppet patches. There will be
two Puppet SWATs every week - one on Tuesday and one on Thursday,
right after Morning SWAT - so 16:00 - 17:00 UTC. See the Deployments
page for time in your local timezone! After a month of this (all of
September), we'll re-evaluate and commit to doing this as a team if
there's enough gains from this.
So write your simple patches and put them up for SWAT!
Thanks.
--
Yuvi Panda T
http://yuvi.in/blog
As part of Brion's and mine struggle for better A/V support on Wikipedia, I
have concluded that our current support for subtitles is rather...
improvised.
Currently all our SRT files are referenced from HTML using action=raw. But
then not actually used from action=raw, but instead served up as semi html
using api.php. Which is ridiculous...
If we want to move to more HTML5 compliancy, we also will want to switch
from the SRT format to the VTT format.
Ideally, I want to host multiple subtitle formats, and dynamically
serve/convert them as either SRT or VTT. These can be directly referenced
from a <track> element so that we are fully compatible.
The question is now, how to best do this. The endpoint needs to be dynamic,
cacheable, allow multiple content types etc.
Ideas suggested have been:
* Api.php
* Restbase
* New endpoint
* ResourceLoader modules
I'm listing the current problems, future requirements and discussing
several ideas at:
https://www.mediawiki.org/wiki/Extension:TimedMediaHandler/TimedTextRework?…
If you have any ideas or remarks, please contribute them !
DJ
Please join for the following tech talk:
*Tech Talk**:* Let's talk about web performance
*Presenter:* Peter Hedenskog
*Date:* August 18, 2015
*Time: *19:30 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?msg=Tech+Talk%3A+Let%2…>
Link to live YouTube stream <http://www.youtube.com/watch?v=acZ3SwbhsaM>
*IRC channel for questions/discussion:* #wikimedia-office
Google+ page
<https://plus.google.com/u/0/b/103470172168784626509/events/c8dmvqt3n4mjn3f7…>,
another
place for questions
*Summary: *
We want our pages to be fast but how do we know they are? Today I’ll go
through the latest trends about what makes pages really super fast for the
user and ways of measuring and really knowing they are. I'll talk about
these things:
- How the browser render and show the page for the user
- What makes a web page fast and what tricks do people use?
- Measuring: why, when and how?
TL;DR: Getting started with RESTBase is now easier thanks to a new SQLite3
backend
Hello,
The WMF Services team~[0] is proud to announce the initial release of the
SQLite storage back-end~[1] for RESTBase~[2], the service that powers
Wikimedia projects’ public REST API~[3]. Traditionally, RESTBase was
limited to Cassandra persistence, which, while scalable, presented a
weighty and difficult dependency for smaller installations and test
environments. This new module allows RESTBase administrators to replace
Cassandra with a lightweight, file-based persistence based on SQLite.
This is an exciting milestone in RESTBase’s development. We expect the
option to run RESTBase atop an SQLite database to significantly widen the
target audience, allowing users with small installations and limited
resources to take advantage of this valuable service. We are also hoping to
attract more contributors by having a simpler, less resource-intensive
alternative to Cassandra; SQLite is now the storage module of choice in
MediaWiki-Vagrant~[6] installations with the restbase role enabled.
Contributing is just a vagrant up away!
The abstract storage interface~[7] we developed for RESTBase allows users
to seamlessly switch between storage back-ends (Cassandra or SQLite). Due
to data volumes we are facing in WMF, we are not using the SQLite back-end
in production. However, we are testing each RESTBase changeset against both
Cassandra and SQLite to ensure they are behaving as expected. Additionally,
a 93% code coverage makes us confident to recommend it for production use.
Bringing the SQLite storage module to a usable state represents a
significant amount of team effort, but nevertheless we’d like to extend a
special thank you to Petr Pchelko for his tireless efforts in driving this
project home. Thanks Petr!
This is only the first step in providing better third-party and developer
support. One of the next challenges will be basing Parsoid~[8] on
service-runner~[9,10], a general-purpose library for running and managing
Node.JS services. Amongst other things, it enables bundling and running
multiple services together. Packaging and distributing Parsoid and RESTBase
together will simplify their installation, configuration and administration
in small-setup and development environments.
Best,
Marko Obrovac, PhD
Senior Services Engineer
Wikimedia Foundation
[0] https://www.mediawiki.org/wiki/Wikimedia_Services
[1] https://github.com/wikimedia/restbase-mod-table-sqlite
[2] https://www.mediawiki.org/wiki/RESTBase
[3] https://en.wikipedia.org/api/rest_v1/?doc (for a complete list of
supported domains go to https://rest.wikimedia.org/)
[4] https://phabricator.wikimedia.org/tag/restbase/
[5] https://phabricator.wikimedia.org/tag/restbase-api/
[6] https://www.mediawiki.org/wiki/MediaWiki-Vagrant
[7] https://github.com/wikimedia/restbase-mod-table-spec
[8] https://www.mediawiki.org/wiki/Parsoid
[9] https://github.com/wikimedia/service-runner
[10] https://phabricator.wikimedia.org/T90668
== Apologies for Crossposting ==
Below you will find useful information and links about the program, our
satellite events and registration opportunities for
SEMANTiCS 2015 -- 11th International Conference on Semantic Systems
15-17 September 2015, Vienna / Austria
http://www.semantics.cc/ // #semantics2015 // #semanticsconf
*— Conference Scope ---*
The annual SEMANTiCS conference is the meeting place for professionals
who make semantic computing work, and understand its benefits and know
its limitations. Every year, SEMANTiCS attracts information managers,
IT-architects, software engineers, and researchers, from organisations
ranging from NPOs, universities, public administrations to the largest
companies in the world.
*— Conference Program ---*
The 2015 edition offers a rich program consisting of 5 keynotes, 24
scientific presentations, 30 industry talks, 38 posters and various
workshops and social events. For details please visit our program page:
http://www.semantics.cc/programme
*— Keynote Speakers --- *
·Jeanne Holms -- Chief Knowledge Architect at NASA
·Peter Mika -- Director Semantic Lab at Yahoo
·Oscar Orcho -- Associate Professor for Artifical Intelligence,
Universidad de Madrid
·Klaus Tochtermann -- Leibniz Information Center for Economics
·Sam Rehman -- CTO at EPAM Systems
*— Workshops & Satellite Events ---*
*MeetUp: SMART DATA SOLUTIONS*
An outlook into the world of data centric business, technologies and
innovations: Data is everywhere these days and efficient data management
is THE key factor for success in nearly all industries in the meantime.
McKinsey lists data as a key factor for production alongside with labor
and capital in one of their recent reports. Furthermore, Data is
produced in huge amounts by sensors, social networks or mobile devices
and the amounts of data available worldwide grow exponentially…
Place: Haus der Ingenieure, Eschenbachgasse 9, 1010 Wien
Date: 15.09.2015, Entrance: 18:30pm CET ; Start event: 19:30 - 22:30pm CET
*2nd International Workshop on Geospatial Linked Data*
In recent years, Semantic Web technologies have strengthened their
position in the areas of data and knowledge management. Standards for
organizing and querying semantic information, such as RDF(S) and SPARQL
are adopted by large academic communities, while corporate vendors adopt
semantic technologies to organize, expose, exchange and retrieve their
datasets as Linked Data.
Chairs: Alejandra Garcia-Rojas M. (Ontos AG), Robert Isele or Rene
Pietzsch, Jens Lehmann (AKSW, University of Leipzig)
Date: 15th of September 2015, 09.00 to 13.00 CEST
*The SEMANTIC EXPERIENCE Coffee & Cocktail LOUNGE*
Sponsored Event: Enjoy semantic technology inspiration in a relaxed
atmosphere. Refresh yourself with Viennese coffee or a cocktail and get
in touch with semantic industry experts.
Date: 15th of September 2015, 15.45 to 18.00 CEST
Room: LC Club Room
*Linked Data in Industry 4.0*
The overall goal of the workshop is to identify challenges and
limitations from the manufacturing engineering industry in the scope of
the mentioned design principles, and bring them together with experts
and solution approaches from the linked data community in the scope of
Industry 4.0.
Chairs: Thomas Moser (FH St. Pölten), Stefan Hupe (IoT Austria)
Date: 15th of September 2015, 14.00 to 17.00 CEST
*European Data Economy Workshop - Focus Data Value Chain & Big and Open
Data*
This workshop is to overview the state of the art in Europe regarding
Big and Open Data initiatives and its impact in the Europan economy and
benefits for theEuropean society. Representatives from the Big Data
Value Association, the annual European Data Forum and data related
projects will participate during the first session of the workshop.
Furthermore it gives information about the Austrian Big Data Study
carried out in 2014 by AIT and IDC
Chairs: Nelia Lasierra (STI Innsbruck), Martin Kaltenböck (Semantic Web
Company)
Date: 15th of September 2015, 09.00 to 13.00 CEST
*1st Workshop on Data Science: Methods, Technology and Applications
(DSci15)*
This workshop is meant as an opportunity to bring together researchers
and practitioners interested in data science to present their ideas and
discuss the most important scientific, technical and socio-economical
challenges of this emerging field.
Chairs: Bernhard Haslhofer (AIT - Austrian Institute of Technology),
Elena Simperl (Univ. Southampton), Rainer Stütz (AIT - Austrian
Institute of Technology), Ingo Feinerer (FH Wiener Neustadt)
Date: 15th of September 2015, 09.00 to 17.00 CEST
*Workshop on Linked Data Strategies - Commercialisation of Interlinked Data*
In this workshop, we will give several demos and concrete examples of
how Linked Data can be used by enterprises in various industries. The
workshop aims to give users and providers of Linked Data valuable
methods and best practices at hand, which help them to make profound
decisions in their Linked Data projects.
Chairs: Christian Dirschl (Wolters Kluwer), Andreas Blumauer (Semantic
Web Company), Tassilo Pellegrini (FH St. Pölten)
Date: 15th of September 2015, 14.00 to 15.30 CEST
*Hackathon on "The power of Linked Data in Agriculture and Food Safety"*
“Data+Need=Hack”, this is the idea of a hackathon that brings together
like-minded people to develop, in a short time frame, novel solutions to
problems around the theme “Agriculture and Food Safety”.
Chairs: Christian Blaschke (Semantic Web Company, Vienna), Stasinos
Konstantopoulos (Institute of Informatics & Telecommunications of the
NCSR Demokritos, Athens)
Date: 18th of September 2015, 10.00 to 16.00 CEST
*— Registration ---*
To register, please go to:
http://www.semantics.cc/registration
We are looking forward to meet you at SEMANTiCS 2015!
In the version history of an image (or any attached file in MediaWiki), the
page displays "Date/Time" with a link to that version. The timestamp
displayed is the upload timestamp of that version. If you look closely, you
can see that the real filename includes a different timestamp. This turns
out to be the timestamp of when that file was superseded by a subsequent
version.
I have looked in the database tables and can see that in the oldimage
table, each row has an "oi_archive_name" with the timestamp of when that
version was superseded and an "oi_timestamp" of when that version was
actually uploaded.
Is there a reason to name the old versions of the files with the
superseding timestamp instead of the upload timestamp? It seems to me that
the timestamp of when that version was uploaded is more relevant.
Daren
--
__________________
http://enterprisemediawiki.orghttp://mixcloud.com/darenwelshhttp://www.beatportfolio.com
FYI
---------- Forwarded message ----------
From: Yuvi Panda <yuvipanda(a)gmail.com>
Date: Thu, Aug 20, 2015 at 4:45 PM
Subject: Evaluation of opt-in alternatives to Grid Engine on Tool Labs
('clustering solution')
To: Wikimedia Labs <labs-l(a)lists.wikimedia.org>
Hello!
One of the experimental goals for this quarter for labs' team is to
make available an new, more modern gridengine alternative just for
webservices on Tool Labs. We are starting to evaluate which systems we
should use - this is tracked at
https://phabricator.wikimedia.org/T106475. The (still incomplete)
evaluation spreadsheet is at
https://docs.google.com/spreadsheets/d/1YkVsd8Y5wBn9fvwVQmp9Sf8K9DZCqmyJ-ew…
We are evaluationg Kubernetes and Mesos/Marathon as alternatives.
GridEngine is also being scored along with them, so if we find that it
wins we'll abandon the experiment and continue using GridEngine only.
Do provide comments on the phab ticket and follow along :)
== WHY? ==
Because our current webservices setup is a pile of hacks on top of
GridEngine, causing... interesting problems due to the complexity
involved.
GridEngine doesn't support a lot of features that people using more
modern systems take for granted - like containerization + isolation, a
nice API, continuous deploy, autoscaling.... Having an alternative to
play with allows us to build newer, better featured and more robust
systems.
== OMG, WILL I HAVE TO CHANGE THE WAY MY CODE WORKS NOW?! ==
Nope. For now this is just an alternative - when completed, you will
be able to run your webservice on this cluster by something like:
webservice --provider=<something> start
And nothing else will change - everything else should still be
compatible. We'll eventually provide more features on the new setup,
but there will be no forced migration of any sort. If the alternative
becomes the default at any point, we'll ensure that things that worked
before continue working without any extra effort from the Tool
Author's part.
--
Yuvi Panda T
http://yuvi.in/blog
--
Yuvi Panda T
http://yuvi.in/blog