Reminder: Technical Advice IRC meeting **today (Wednesday) 4-5 pm UTC** on
#wikimedia-tech.
Question can be asked in English, Spanish & German.
The Technical Advice IRC Meeting is a weekly support event for volunteer
developers. Every Wednesday, two full-time developers are available to help
you with all your questions about Mediawiki, gadgets, tools and more! This
can be anything from "how to get started" over "who would be the best
contact for X" to specific questions on your project.
If you know already what you would like to discuss or ask, please add your
topic to the next meeting:
https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
Hope to see you there!
Michi (for the Technical Advice IRC Meeting crew)
--
Michael F. Schönitzer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Tel. (030) 219 158 26-0
https://wikimedia.de
Unsere Vision ist eine Welt, in der alle Menschen am Wissens der Menschheit
teilhaben, es nutzen und mehren können. Helfen Sie uns dabei!
https://spenden.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/029/42207.
Hello,
This is the weekly update from the Search Platform team for the week
starting 2019-01-14.
As always, feedback and questions welcome.
== Discussions ==
=== Search ===
* Trey updated TextCat with models for detecting Russian typed on an
English keyboard and vice-versa, and UTF-8 Russian text improperly
encoded as Windows-1251, [0] as a precursor to providing
wrong-keyboard/encoding detection and suggestion. [1]
* Erik and the team did a lot of work on an epic ticket (with several
sub tasks) to explore and figure out next steps in using user click
data to tune Wikidata search parameters [2] and [3]. The team will
ship the newly tuned wbsearchentities profile for en soon with de, fr,
es afterward.
* The team also had lots of discussions and exploration on how to
transform Wikidata autocomplete click logs into a useful dataset. They
are now transformed: Relevance Forge now has a utility for taking in
the Wikidata completion search logs and tuning the parameters of
search based on those logs. [4]
* David fixed a minor regression where search request failures when
offset+limit is out of bounds (cirrussearch-backend-error) [5]
* Mathew discovered that the required metrics have been exposed by the
prometheus exporter but they are displaying and fixed the issue with
help from David and Gehel [6]
* David reconfigured the ElasticSearch crosscluster on production
search servers to have persistent configs [7]
=== WDQS ===
* Stas & Guillaume finished moving categories namespace into a
separate Blazegraph instance [8]
== Did you know? ==
English text, like many others, is written left-to-right (LTR), but
some languages—most notably Arabic, Hebrew, Persian, and Urdu, but
also many others [9]—are written right-to-left (RTL). To handle
different writing directions—especially in mixed LTR and RTL
texts—Unicode classifies characters as having "strong", "weak", or
"neutral" directionality. Strong characters definitely go in a
particular direction, like ABC or אבג. Weak characters have a "vague"
directionality, but can be changed in context, mostly numbers. Neutral
characters pick up their directionality from context, like punctuation
and whitespace characters used across scripts.
Mirrored characters change the way they display based on context. For
example "A>B>C" and "א>ב>ג" both only have the greater than character
(>) in them, but, if you are reading this somewhere that follows the
Unicode bidirectional algorithm, the ones between Latin letters point
to the right and those between Hebrew letters point to the left.
The algorithms are complicated [10], and when they don't work, there
are explicit characters that indicate things like "text should flow
left to right from here". The explicit formatting characters have the
most potential to cause trouble for search because they are usually
invisible, and you can pick one up without realizing it. For example,
when copying an Arabic word from a page in English, or a French word
from a page in Hebrew, the word that is "the other way around" from
the main text might have one of these marks at the beginning or end of
it. Fortunately, we can usually identify them and strip them out.
Finally, there are some scripts that have been written in other
interesting directions. Vertical text includes Chinese, Japanese, and
Korean, [11] and Mongolian. [12]. Hanunó'o [13] and Ogham [14] were
written bottom-to-top! My [Trey's] favorite "direction" is
"boustrophedon," [15] which means "like an ox ploughs" and alternates
left-to-right and right-to-left, and was used particularly in old
manuscripts and inscriptions in may writing systems. Why jump from one
side of the page to the other when you can just curve around where you
are or flip to mirrored letters and keep going?!
[0] https://phabricator.wikimedia.org/T213931
[1] https://phabricator.wikimedia.org/T138958
[2] https://phabricator.wikimedia.org/T193701
[3] https://phabricator.wikimedia.org/T213105
[4] https://phabricator.wikimedia.org/T205111
[5] https://phabricator.wikimedia.org/T213745
[6] https://phabricator.wikimedia.org/T210592
[7] https://phabricator.wikimedia.org/T213150
[8] https://phabricator.wikimedia.org/T213212
[9] https://en.wikipedia.org/wiki/Right-to-left#List_of_RTL_scripts
[10] https://www.w3.org/International/articles/inline-bidi-markup/uba-basics
[11] https://en.wikipedia.org/wiki/Horizontal_and_vertical_writing_in_East_Asian…
[12] https://en.wikipedia.org/wiki/Mongolian_script
[13] https://en.wikipedia.org/wiki/Hanun%C3%B3%27o_alphabet
[14] https://en.wikipedia.org/wiki/Ogham
[15] https://en.wikipedia.org/wiki/Boustrophedon
----
Subscribe to receive on-wiki (or opt-in email) notifications of the
Discovery weekly update.
https://www.mediawiki.org/wiki/Newsletter:Discovery_Weekly
The archive of all past updates can be found on MediaWiki.org:
https://www.mediawiki.org/wiki/Discovery/Status_updates
Interested in getting involved? See tasks marked as "Easy" or
"Volunteer needed" in Phabricator.
[1] https://phabricator.wikimedia.org/maniphest/query/qW51XhCCd8.7/#R
[2] https://phabricator.wikimedia.org/maniphest/query/5KEPuEJh9TPS/#R
Yours,
Chris Koerner (he/him)
Community Relations Specialist
Wikimedia Foundation
Hi,
Is the UserMerge extension actually used on Wikimedia sites?
As far as I can see, it is only installed on Wikivoyage, and the last log
entries of its usage are from 2015. However, it's possible that I'm missing
something.
The reason I'm asking is that I'd love to know how to configure it best in
translatewiki. If it's not actually used on Wikimedia sites or developed
significantly, then it should be filed under "Legacy".
Thanks! :)
--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
“We're living in pieces,
I want to live in peace.” – T. Moore
📘 Read this post on Phabricator at
https://phabricator.wikimedia.org/phame/post/view/130/
<https://phabricator.wikimedia.org/phame/post/view/130/production_excellence…>
-------
How’d we do in our strive for operational excellence last month? Read on to
find out!
- Month in numbers.
- Lighting round.
- Current problems.
## 📊 *Month in numbers*
* 4 documented incidents. [1]
* 20 Wikimedia-prod-error tasks closed. [2]
* 18 Wikimedia-prod-error tasks created. [3]
* 172 currently open Wikimedia-prod-error tasks (as of 16 January 2019).
Terminology:
* An *Exception* (or fatal) prevents a user action. For example, a page
would display “Exception: Unable to render page”, instead the article
content.
* An *Error* (or non-fatal, warning) can produce pages that are technically
unaware of a problem, but may show corrupt, incorrect, or incomplete
information. For example — a user may receive a notification that says “You
have (null) new messages”.
For December, I haven’t prepared any stories or taken interviews. Instead,
I’ve got a lightning round of errors in various areas that were found and
fixed this past month.
## ⚡️ *Contributions view fixed*
MarcoAurelio reported that Special:Contributions failed to load for certain
user names on meta.wikimedia.org (PHP Fatal error, due to a faulty database
record). Brad Jorsch investigated and found a relation to database
maintenance from March 2018. He corrected the faulty records, which
resolved the problem. Thanks! — https://phabricator.wikimedia.org/T210985
## ⚡️ *Undefined talk space now defined*
The newly created Cantonese Wiktionary (yue.wiktionary.org) was
encountering errors from the Siteinfo API. We found this was due to invalid
site configuration. Urbanecm patched the issue, and also created a new unit
test for wmf-config that will prevent this issue from happening on other
wikis in the future. Thanks! — https://phabricator.wikimedia.org/T211529
## ⚡️ *The undefined error status... error*
After deploying the 1.33.0-wmf.8 train to all wikis, we found a regression
in the HTTP library for MediaWiki. When MediaWiki requested an HTTP
resource from another service, and this resource was unavailable, then
MediaWiki failed to correctly determine the HTTP status code of that error.
Which then caused another error! This happened, for example, when
Special:Collection was unable to reach the PediaPress.com backend in some
cases. Fixed by Bill Pirkle. Thanks! —
https://phabricator.wikimedia.org/T212005
## ⚡️ *Fatal error: Call to undefined function in Kartographer API*
When the 1.33.0-wmf-9 train reached the canary phase on Tue 18 December
(aka, group0 [1]), Željko spotted a new fatal error in the logs. The fatal
originated in the Kartographer extension and would have affected various
users of the MediaWiki API. Patched the same day by Michael Holloway,
reviewed by James Forrester, and deployed by Željko. Thanks! —
https://phabricator.wikimedia.org/T212218
## 📉 *Current problems*
Take a look at the workboard and look for tasks that might need your help.
The workboard lists known issues, grouped by the week in which they were
first observed.
→ https://phabricator.wikimedia.org/tag/wikimedia-production-error/
December’s theme will continue for now, as I imagine lots of you were on
vacation during that time! I’d like to draw attention to a subset of PHP
fatal errors. Specifically, those that are publicly exposed (e.g. don’t
need elevated user rights) and emit an HTTP 500 error code.
* Wikibase: Clicking “undo” for certain revisions fatals with a
PatcherException. — https://phabricator.wikimedia.org/T97146
* Flow: Unable to view certain talk pages due to workflow
InvalidDataException. — https://phabricator.wikimedia.org/T70526
* Translate: Certain Special:Translate urls fatal. —
https://phabricator.wikimedia.org/T204833
* MediaWiki (Special-pages): SpecialDoubleRedirects unavailable on
tt.wikipedia.org. — https://phabricator.wikimedia.org/T204800
* MediaWiki (Parser): Parse API exposes fatal content model error. —
https://phabricator.wikimedia.org/T206253
* CentralNotice: Certain SpecialCentralNoticeBanners urls fatal. —
https://phabricator.wikimedia.org/T149240
* PageViewInfo: Certain “mostviewed” API queries fail. —
https://phabricator.wikimedia.org/T208691
Public user requests resulting in fatals can (and have) caused alerts to
fire that notify SRE of wikis potentially being less available or down.
💡*ProTip*: Use “Report Error” on
https://phabricator.wikimedia.org/tag/wikimedia-production-error/ to create
a task with a helpful template. This template is also available as “Report
Application Error”, from the “Create Task” dropdown menu, on any task
creation form.
## 🎉 *Thanks!*
Thank you to everyone who has helped by reporting, investigating, or
resolving problems in Wikimedia production. Including MarcoAurelio, Anomie,
Urbanecm, BPirkle, zeljkofilipin, Mholloway, Esanders, Jdforrester-WMF, and
hashar.
Until next time,
— Timo Tijhof
-------
Footnotes:
[1] Incidents. —
https://wikitech.wikimedia.org/wiki/Special:AllPages?from=Incident+document…
[2] Tasks closed. —
https://phabricator.wikimedia.org/maniphest/query/Pe2KaRZhJJ.H/#R
[3] Tasks opened. —
https://phabricator.wikimedia.org/maniphest/query/aqbDey80TU02/#R
[4] What is group0? —
https://wikitech.wikimedia.org/wiki/Deployments/One_week#Three_groups
Hey all!
I'm reviving an old project to embed sandboxed HTML/JavaScript "widgets"
into wiki pages as a click-to-play media type, using modern browsers'
<iframe> sandbox and Content-Security-Policy restrictions.
Intro and detail notes which I'll keep updating:
https://www.mediawiki.org/wiki/User:Brion_VIBBER/EmbedScript_2019
I hope to extend it with a headless "plugin" mode which allows less-trusted
user-written code to interact safely with fully-trusted host APIs, and a
dependency system to let common library modules, string localizations,
image files from Commons, and data from Wikidata be bundled up and used
safely, without cross-site data exposure.
I'm hoping to solicit some more feedback while I'm in the prototyping
stage, with an eye towards issues we'll need to resolve before it reaches a
productizable stage we could seriously deploy.
Open questions include:
* Can we really replace some user scripts and gadgets with a split-trust
model, and which ones are good ones to start experimenting with?
* What should a user-permissions UX look like for plugins? What threat
models are not examined yet?
* What kind of comment / code review system is needed?
* What about patches, and forks, and copies and centralization? what's the
best Commons-centric or alternate model that will prevent fragmentation of
code?
* How should libraries / dependencies work?
* How should localization work?
* How much coupling to MediaWiki is desired/required?
* How to implement mobile app and offline support?
Feel free to poke me directly or on the wiki talk page with
questions/comments/ideas. Love it? Hate it? Great! Let me know. :)
-- brion
Using an abstract language as an basis for translations have been
tried before, and is almost as hard as translating between two common
languages.
There are two really hard problems, it is the implied references and
the cultural context. An artificial language can get rid of the
implied references, but it tend to create very weird and unnatural
expressions. If the cultural context is removed, then it can be
extremely hard to put it back in, and without any cultural context it
can be hard to explain anything.
But yes, you can make an abstract language, but it won't give you any
high quality prose.
On Mon, Jan 14, 2019 at 8:09 AM Felipe Schenone <schenonef(a)gmail.com> wrote:
>
> This is quite an awesome idea. But thinking about it, wouldn't it be possible to use structured data in wikidata to generate articles? Can't we skip the need of learning an abstract language by using wikidata?
>
> Also, is there discussion about this idea anywhere in the Wikimedia wikis? I haven't found any...
>
> On Sat, Sep 29, 2018 at 3:44 PM Pine W <wiki.pine(a)gmail.com> wrote:
>>
>> Forwarding because this (ambitious!) proposal may be of interest to people
>> on other lists. I'm not endorsing the proposal at this time, but I'm
>> curious about it.
>>
>> Pine
>> ( https://meta.wikimedia.org/wiki/User:Pine )
>>
>>
>> ---------- Forwarded message ---------
>> From: Denny Vrandečić <vrandecic(a)gmail.com>
>> Date: Sat, Sep 29, 2018 at 6:32 PM
>> Subject: [Wikimedia-l] Wikipedia in an abstract language
>> To: Wikimedia Mailing List <wikimedia-l(a)lists.wikimedia.org>
>>
>>
>> Semantic Web languages allow to express ontologies and knowledge bases in a
>> way meant to be particularly amenable to the Web. Ontologies formalize the
>> shared understanding of a domain. But the most expressive and widespread
>> languages that we know of are human natural languages, and the largest
>> knowledge base we have is the wealth of text written in human languages.
>>
>> We looks for a path to bridge the gap between knowledge representation
>> languages such as OWL and human natural languages such as English. We
>> propose a project to simultaneously expose that gap, allow to collaborate
>> on closing it, make progress widely visible, and is highly attractive and
>> valuable in its own right: a Wikipedia written in an abstract language to
>> be rendered into any natural language on request. This would make current
>> Wikipedia editors about 100x more productive, and increase the content of
>> Wikipedia by 10x. For billions of users this will unlock knowledge they
>> currently do not have access to.
>>
>> My first talk on this topic will be on October 10, 2018, 16:45-17:00, at
>> the Asilomar in Monterey, CA during the Blue Sky track of ISWC. My second,
>> longer talk on the topic will be at the DL workshop in Tempe, AZ, October
>> 27-29. Comments are very welcome as I prepare the slides and the talk.
>>
>> Link to the paper: http://simia.net/download/abstractwikipedia.pdf
>>
>> Cheers,
>> Denny
>> _______________________________________________
>> Wikimedia-l mailing list, guidelines at:
>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
>> https://meta.wikimedia.org/wiki/Wikimedia-l
>> New messages to: Wikimedia-l(a)lists.wikimedia.org
>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
>> <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>
>> _______________________________________________
>> Wikipedia-l mailing list
>> Wikipedia-l(a)lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikipedia-l
>
> _______________________________________________
> Wikidata mailing list
> Wikidata(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikidata
Official site: https://ocs.letras.up.pt/index.php/IWSC2019/IWSC2019
The* International Wiki Scientific Conference 2019 *will be held on *March
11, 12 and 13, 2019 *in Porto (Portugal) and will take place at the Faculty
of Arts of the University of Porto.
*ORGANIZATION:*
*FLUP - Faculdade de Letras da Universidade do Porto (PT)*
*CIC.DIGITAL Porto (PT)*
*Wiki Educação Brasil (BR)*
*PPGCI - Universidade Federal Fluminense (RJ-BR)*
CALL FOR PAPERS
https://ocs.letras.up.pt/index.php/iwsc2019/IWSC_2019/author/submit?require…
In 2018, after the European Year of Cultural Heritage was celebrated by the
first time and by the initiative of the European Union, the *IWSC 2019*
promotes *Heritage*, both material and intangible, to draw attention to the
presence of Culture and heritage on Wiki platforms and to the contribution
this digital resource/look could have on social and economic development in
Europe and worldwide.
Therefore, works will be considered provided they focus on Wiki culture as
a topic, source or way of spreading scientific knowledge and teaching tool
and, particularly in the scope of heritage. Texts should be written in
English, Spanish or Portuguese and will be assessed by members of the
Scientific Commission.
Submission period starts on the 15 October and ends on the *30 January 2019*.
Authors will be notified whether their works have been accepted by the *10
February 2019*.
*The papers should be submitted in the following thematic groups:*
- Wiki Culture
- Information accreditation on Wikipedia
- Spreading and promoting Science on Wikipedia
- Dissemination and promotion of material and intangible heritage
- Educational Projects on Wiki Platforms
Posters may be presented (up to 3 pages, including references) or articles
(from 8 to 10 pages, including references).
The papers will be published in the digital proceeding of the event. For
this reason, at least one of the authors must register for the event and
perform oral presentation.
In the case of changes requested by the evaluators, the authors have
until *February
20, 2019* to submit their final versions.
*Opening of Call for Papers: *October 15, 2018
*Call for papers closes:* January 30, 2018
Best regards
Rodrigo Padula