Can Parsoid be used to convert arbitrary HTML to wikitext? It's not clear
to me whether it will only work with Parsoid's HTML+RDFa. I'm wondering if
I could take snippets of HTML from non-MediaWiki webpages and convert them
into wikitext.
Thanks,
James
Hey,
I just wanted to check in about the status of enabling JavaScript package
management usage in MediaWiki. I am basically talking about an equivalent
for JS to what we have with Composer for PHP.
Real-world example:
The "data-values/value-view" package[0] is defining
"jquery.event.special.eachchange.js":
ValueView/lib/jquery.event/jquery.event.special.eachchange.js
Now, recently I needed the same functionality in one of my extensions, so
I just copied it over. [1]
I know that this is the worst way one could do this, but as far as I can
see we don't have that much of a choice right now. Here are the alternative
options I can see:
Moving "jquery.event.special.eachchange.js" out of the
"data-values/value-view" package into its own "WMDE/jquery-eachchange"
package...
1. ... and using it in my extension via composer.
+ pro: two or more extensions or other packages requiring this package
will still result in having only one MW-wide installation..
- con: requires MW specific code which is actually not related to the
MW-independent package to feed the resource loader.
- con: using Composer to manage pure JavaScript packages! Uuuh, ugly!
2. ... and having a build step in other packages using the package, pulling
the "WMDE/jquery-eachchange" somewhere into the file structure of the
packages/extensions using it.
+ pro: don't need to abuse composer, we can use "npm", "Bower" or any
other arbitrary JS package manager here.
- con: got to tell resource loader somehow... (didn't think so much about
that yet)
- con: if more than one extensions or other packages require this package
we still end up with the same code twice or more often in one MW
installation.
3. Combining 1 and 2: Start with 2, using a JS package manager. Then going
to 1, creating a composer package and pulling the "WMDE/jquery-eachchange"
package in via some build script.
+ pro: The two pros from 1 + 2
+ pro: ^^
- con: still got to tell resource loader somehow...
- con: Overhead; We now create two packages where the Composer one is
just a bridge to the MW-world, still polluting packagist.org. Still kind of
ugly and more effort for publishing a package and therefore potentially
scaring programmers away from doing so since they've got better things to
do than doing work that could be automated.
I have not seen Approach 2 and 3 yet. Though I could imagine that the
VisualEditor team has used something like that.
Approach 1 is the way the "data-values/value-view" package itself is being
handled. And that package should actually be a MW independent pure JS
package but right now it contains MW specific code and uses composer for
distribution!
There is still another option but that had to be properly implemented:
4. Choose one native JS package manager for now and go with it (add support
for others later perhaps). Integrate it properly with MW (resource loader
to begin with), document how to use it and finally distribute JS code
coming from the MW world but useful for other projects in a way where it
can actually be used in a non-MW context.
This has already been bugging me when working on Wikidata. Now I'd like to
reuse some of the code I have written there without spending hours and
hours with option 3 because there should be support for option 4 rather
sooner or later.
So I am wondering; Does anyone have any thoughts, any alternatives perhaps
or is there any roadmap on anything like the option 4 that I have shown?
Cheers,
Daniel
[0]: https://packagist.org/packages/data-values/value-view
[1]:
https://github.com/DanweDE/mediawiki-ext-UserBitcoinAddresses/blob/master/r…
Superprotect [1] was introduced by the Wikimedia Foundation to resolve a
product development disagreement. We have not used it for resolving a
dispute since. Consequently, today we are removing Superprotect from
Wikimedia servers.
Without Superprotect, a symbolic point of tension is resolved. However, we
still have the underlying problem of disagreement and consequent delays at
the product deployment phase. We need to become better software partners,
work together towards better products, and ship better features faster. The
collaboration between the WMF and the communities depends on mutual trust
and constructive criticism. We need to improve Wikimedia mechanisms to
build consensus, include more voices, and resolve disputes.
There is a first draft of an updated Product Development Process [2] that
will guide the work of the WMF Engineering and Product teams.[3] It
stresses the need for community feedback throughout the process, but
particularly in the early phases of development. More feedback earlier on
will allow us to incorporate community-driven improvements and address
potential controversy while plans and software are most flexible.
We welcome the feedback of technical and non-technical contributors. Check
the Q&A for details.[4]
[1] https://meta.wikimedia.org/wiki/Superprotect
[2] https://www.mediawiki.org/wiki/WMF_Product_Development_Process
[3] https://www.mediawiki.org/wiki/Wikimedia_Engineering
[4]
https://www.mediawiki.org/wiki/WMF_Product_Development_Process/2015-11-05#Q…
--
Quim Gil
Engineering Community Manager @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
Hi,
I'm behind on my summit planning, but here's what I'd like to discuss at
the developer summit for the Shadow namespaces RfC[1].
1. At what layer should shadow namespaces be integrated into the code?
Should we continue to subclass Article? How do we efficiently implement
things like batch lookups? (T88644)
2. How will we integrate remote content with search and other discovery
mechanisms? We currently have Commons file results integrated with
normal search results, but it's usefulness is questionable (T96535).
Will it be possible to implement this for people using remote content
via the API?
3. How will we keep track of usage and invalidate caches? Should we just
have a central link table like GlobalUsage? What about API users?
4. Should we allow people to chain "repositories" like we currently
allow with file repos?
The outcome I'd like to see is having a plan ready on how to migrate the
foreign file system to a generic arbitrary namespace one, which should
completely obsolete the GlobalUserPage extension.
The Phabricator task for this session at the summit is T115762[2].
[1] https://www.mediawiki.org/wiki/Requests_for_comment/Shadow_namespaces
[2] https://phabricator.wikimedia.org/T115762
-- Legoktm
As part of a side project in my job at Redwerks[1], I've released an
early version of a mediawiki-extension cli tool.
https://github.com/redwerks/mediawiki-extension-command#readme
The tool requires Node.js (in addition to git and php to run clones and
composer).
It can be installed with `sudo npm install -g mediawiki-extension` and
then `mediawiki-extension setup`.
The command can download and upgrade any extension we have in Gerrit.
Extensions using composer will automatically be installed via composer
otherwise it'll be cloned from git. If available, release tags (like
"1.0.0" or "v3.0.0") will be used, otherwise master will be used.
You'll still need to require and configure the extension yourself. But
this is supposed to greatly simplify acquiring and bulk-updating
extensions via the cli.
Some examples of use.
Clone ParserFunctions from git.
$ mediawiki-extension download ParserFunctions
Install SemanticMediaWiki and SemanticForms with composer.
$ mediawiki-extension download SemanticMediaWiki SemanticForms
Clone Widgets with from git and checkout the most recent version tag.
$ mediawiki-extension download Widgets
Update all your MediaWiki extensions:
$ mediawiki-extension update --all
Switch an extension cloned from git master to the REL branch for the
installed version of MediaWiki.
$ mediawiki-extension switch ParserFunctions git-rel
[1] http://redwerks.org/
--
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
Hello,
This is the first monthly report from the Wikimedia Performance Team.
The purpose
of these reports is to showcase our work, track our progress, and spark
conversation.
## Who we are ##
As the Wikimedia Foundation’s Performance Team, we want to create
value for readers
and editors by making it possible to retrieve and render content at the
speed of thought, from anywhere in the world, on the broadest range of
devices and connection profiles.
We are five engineers: Aaron Schulz, Gilles Dubuc, Peter Hedenskog,
Timo Tijhof,
and me. Each of us was born in a different country and has a different mother
tongue. We work out of San Francisco, London, Stockholm, and Montpellier.
## What we are working on ##
* Availability. Although Wikimedia Foundation currently operates six data
centers, MediaWiki is only running on one (in Ashburn, Virginia). If you
are an editor in Jakarta, Indonesia, content has to travel over 15,000
kilometers to get from our servers to you (or vice versa). To run MediaWiki
concurrently from multiple places across the globe, our code needs to be
more resilient to failure modes that can occur when different subsystems
are geographically remote from one another.
https://phabricator.wikimedia.org/T88445
* Performance testing infrastructure. WebPageTest provides a stable
reference for a set of browsers, devices, and connection types from
different points in the world. It collects very detailed telemetry that we
use to find regressions and pinpoint where problems are coming from. This
is addition to the more basic Navigation Timing metrics we gather from real
users in production.
https://phabricator.wikimedia.org/T109666
* Media stack. We're currently working on overhauling our thumbnail
infrastructure to achieve multiple goals. Improving future-proof
maintainability by taking thumbnail code out of MediaWiki-Core and using a
service instead to perform thumbnail operations. Saving on expensive
storage by no longer storing multiple copies of all thumbnails on disk
forever. Enabling far-future expires for images, to greatly improve their
client cacheability. And finally switching to the best performing
underlying software to generate thumbnails faster.
https://phabricator.wikimedia.org/T111718
* ResourceLoader. ResourceLoader is the MediaWiki subsystem that is
responsible for loading JavaScript and CSS. Whereas much of MediaWiki's
code executes only sparingly (in reaction to editors modifying content)
ResourceLoader code runs over half a billion times a day on hundreds of
millions of devices. Its contribution to how users experience our sites is
very large. Our current focus is on improving ResourceLoader's cache
efficiency by packaging and delivering JavaScript and CSS code in a way
that allows it to be reused across page views without needing to be
repeatedly downloaded.
https://phabricator.wikimedia.org/T102578
## How are we doing? ##
In future reports, we will spend less time on introductions and more time
reporting on how particular performance metrics have trended since the
previous report and why. In the meantime, we invite you to check out our
dashboards:
* https://performance.wikimedia.org/
* https://grafana.wikimedia.org/dashboard/db/navigation-timing
* https://grafana.wikimedia.org/dashboard/db/time-to-first-byte
* https://grafana.wikimedia.org/dashboard/db/job-queue-health
## Feedback? ##
Please let us know what you think about these reports and whether we have
picked the right lists to send it to. (We're going to make sure this
information is available on mediawiki.org too.)
Until next time,
Aaron, Gilles, Peter, Timo & Ori
Hi Everyone --
We wanted to give you an update on where we were with the strategy process
and provide some more avenues for feedback and participation.
When we last spoke, we had presented an initial draft of our strategy along
with some details of the process we are using. This did not get the
feedback that we had hoped for[1]. We considered this feedback and came up
with some conclusions that we've used to move forward:
1. The strategic problem itself was too broad and unclear to be a
suitable foundation for discussion or specifics.
2. The process is difficult to understand if you haven't been immersed
in it.
3. Strategy is hard and collaborative strategy, particularly with a
large group of distributed stakeholders, is harder.
In response to these conclusions, we've done the following:
1. Simplified and pared down our scope and made our strategic problem
more accessible and actionable.
2. Changed our communications plan to focus more on the outcomes of the
strategic process (what we're proposed we do v. how we came to these ideas)
with the belief that this will be easier to discuss.
3. Moved forward with the process to get to more specific plans and most
importantly start the process to reverse engineer them to see if they will
be effective. We also made a video to explain where we are and have started
a process to gather input on design from the community.
We'd like to ask for your help in the following way:
1. Please review this page:
https://www.mediawiki.org/wiki/Reading/Strategy/Strategy_Process/Testing for
details on how you can provide feedback become more involved in the
process. We're also very happy to recieve feedback on the process itself
and on the strategic options we are reverse-engineering.
- The Reading Team
[1] Our favorite was something along the lines of "It sounds like you put
all of the buzzwords into a bag, shook it up, and poured it out on a table"
-- this was good.