Hello,
Recently I'm researching Parsoid's design as MW is migrating to Parsoid. I found out that due to its single-pass tokenizing design, templates are not handled textually as the legacy parser does.
This is good as the HTML now have information about which template they are transcluded from. However, https://www.mediawiki.org/wiki/Parsoid/limitations says "We have since decided to use the PHP preprocessor for template expansions, which side-steps these issues by reverting to the traditional textual preprocessor pass". Is this still true now?
Best regards,
Diskdance
Hi,
I am running https://www,deurnewiki.nl and cannot get visualeditor properly to work on v1.35 hence I had to disable it.
Apparently the issue is that the rest api is not called correctly. I am wondering whether this is due to nginx.... my phpinfo shows:
$_SERVER['SERVER_SOFTWARE'] nginx/1.10.3
Here: Parsoid<https://www.mediawiki.org/wiki/Parsoid> I did read: "If you're serving MediaWiki with Nginx, you'll need to also add something like this to your server conf:"
location /rest.php/ { try_files $uri $uri/ /rest.php?$query_string; }
Do I need to ask my provider to add this in de nginx config in order to make visualeditor work? Is there any way I can find out myself?
Note: I use the standard installed parsoid which is bundled with mediawiki 1.35
Note2: I redirect calls to deurnewiki.nl to deurnewiki.nl/wiki which resulted in showing an empty page in VE when editing a existing page. Removing this redirect results in a "404 Not Found"when editing an page with VE...
Note3: logfile shows:[http] GET: http://www.deurnewiki.nl/wiki/rest.php/www.deurnewiki.nl/v3/page/html/Besta… resulting in 404 in the browser without the redirect mentioned in Note2 (and the mainpage with the redirect :) )
> On Apr 13, 2020, at 3:55 PM, Francis Franck <francis.franck(a)gmail.com> wrote:
>
> Thank you for your reaction .
>
>> It sounds like you're trying to produce a static copy of your site for backup or offline use?
>
> Indeed, my intention is to create a standalone copy of my wiki for archive purposes. Because of its relation with the cities' history I would like to get the wiki stored in the Archive of the city of Alkmaar but the problem is that they are not equipped for a Mediawiki-oriented structure. The require a fully html oriented version.
>
>> If so, maybe see the project here https://github.com/openzim/mwoffliner or this previous discussion https://lists.wikimedia.org/pipermail/wikitext-l/2020-February/000994.html
>
> I have tested mwoffliner but that doesn't incorporate sidebars and seem to have problems with categories and namespaces. I had the impression that Parsoid was much better equipped to make a thrustworthy copy.
mwoffliner builds on the Parsoid output.
Parsoid only handles the page content, not the chrome, so it won't give you the sidebar, etc.
Please see above linked discussion for some ideas on how to make html dumps of your wiki.
Particularly,
https://lists.wikimedia.org/pipermail/wikitext-l/2020-February/000997.htmlhttps://lists.wikimedia.org/pipermail/wikitext-l/2020-February/000998.html
How do I launch Parsoid's conversion of my website?
Apparently, I succeeded in doing it, because I found an html copy of my
site in a directory called localhost. (I'm running Ubuntu under Windows 10).
Parsoid seems properly installed. The command "curl -L
http://localhost:8142/localhost/v3/page/html/Main_Page/ " gives the
expected result and my website "http://localhost/mediawiki/" uses
"VisualEditor".
My first question : how do I instruct Parsoid to start converting my whole
site?
And secondly: how can I learn Parsoid to add ".html" to the links it
creates ? At present it makes links like "<a
href="/mediawiki/index.php/Dossier:Drebbels_Thermometer"
title="Dossier:Drebbels Thermometer">"
Hi Parsoid Support Team,
I am reaching out to you to know about the usage of this tool. We have a
very old version *1.17.5* of Mediawiki in our organization and want to
convert the pages of it to html pages and store it on disk for archiving.
As you know internally Mediawiki stores pages as WikiText.
Can parsoid (https://www.mediawiki.org/wiki/Parsoid) help us here? I also
saw the documentation of VisualEditor extension (
https://www.mediawiki.org/wiki/VisualEditor) which uses parsoid internally
to convert wikitext pages. Which tool among these 2 should we use to do my
job? Can you please suggest? can parsoid be used as a standalone
application or tool instead of VE?
If we use any of them do we need to just provide the url of our Mediawiki
page (example - *https://<our_dns_host>/wiki/TestPage* or do we need to
extract the content from DB which is in WikiText format and feed it to
parsoid for converting it to html page?
Thanks
(Note: This is only an early heads-up, to be prepared. Google Code-in
has NOT been announced yet, but last year, GCI mentors asked for more
time in advance to identify tasks to mentor. Here you are. :)
* You have small, self-contained bugs you'd like to see fixed?
* Your documentation needs specific improvements?
* Your user interface has some smaller design issues?
* Your Outreachy/Summer of Code project welcomes small tweaks?
* You'd enjoy helping someone port your template to Lua?
* Your gadget code uses some deprecated API calls?
* You have tasks in mind that welcome some research?
Google Code-in (GCI) is an annual contest for 13-17 year old students.
GCI 2019 has not yet been announced but usually takes place from late
October to December. It is not only about coding: We also need tasks
about design, docs, outreach/research, QA.
Read https://www.mediawiki.org/wiki/Google_Code-in/Mentors , add
your name to the mentors table, and start tagging tasks in Wikimedia
Phabricator by adding the #gci-2019 project tag.
We will need MANY mentors and MANY tasks, otherwise we cannot make it.
Last year, 199 students successfully worked on 765 tasks supported by
39 mentors. For some achievements from the last round, see
https://wikimediafoundation.org/news/2019/02/20/partnerships-make-it-possib…
Note that "beginner tasks" (e.g. "Set up Vagrant") and generic
tasks are very welcome (like "Choose and replace 2 uses of
Linker::link() from the list in T223010" style).
We also have more than 400 unassigned open #good-first-bug tasks:
https://phabricator.wikimedia.org/maniphest/query/3YnDUWYJfXSo/#R
Can and would you mentor some of these tasks in your area?
Please take a moment to find / update [Phabricator etc.] tasks in your
project(s) which would take an experienced contributor 2-3 hours. Read
https://www.mediawiki.org/wiki/Google_Code-in/Mentors
, ask if you have any questions, and add your name to
https://www.mediawiki.org/wiki/Google_Code-in/2019#List_of_Wikimedia_mentors
Thanks (as we will not be able to run this without your help),
andre
--
Andre Klapper (he/him) | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/
<tl;dr>: Read https://www.mediawiki.org/wiki/Google_Code-in/Mentors and
add your name to the mentors table and start tagging #GCI-2018 tasks.
We'll need MANY mentors and MANY tasks, otherwise we cannot make it.
Google Code-in is an annual contest for 13-17 year old students. It
will take place from Oct23 to Dec13. It's not only about coding:
we also need tasks about design, docs, outreach/research, QA.
Last year, 300 students worked on 760 tasks supported by 51 mentors.
For some achievements from last round, see
https://blog.wikimedia.org/2018/03/20/wikimedia-google-code-in-2017/
While we wait whether Wikimedia will get accepted:
* You have small, self-contained bugs you'd like to see fixed?
* Your documentation needs specific improvements?
* Your user interface has some smaller design issues?
* Your Outreachy/Summer of Code project welcomes small tweaks?
* You'd enjoy helping someone port your template to Lua?
* Your gadget code uses some deprecated API calls?
* You have tasks in mind that welcome some research?
Note that "beginner tasks" (e.g. "Set up Vagrant") and generic
tasks are very welcome (like "Choose and fix 2 PHP7 issues from
the list in https://phabricator.wikimedia.org/T120336" style).
We also have more than 400 unassigned open #easy tasks listed:
https://phabricator.wikimedia.org/maniphest/query/HCyOonSbFn.z/#R
Can you mentor some of those tasks in your area?
Please take a moment to find / update [Phabricator etc.] tasks in your
project(s) which would take an experienced contributor 2-3 hours. Read
https://www.mediawiki.org/wiki/Google_Code-in/Mentors
, ask if you have any questions, and add your name to
https://www.mediawiki.org/wiki/Google_Code-in/2018#List_of_Wikimedia_mentors
(If you have mentored before and have a good overview of our
infrastructure: We also need more organization admins! See
https://www.mediawiki.org/wiki/Google_Code-in/Admins )
Thanks (as we cannot run this without your help),
andre
--
Andre Klapper | Bugwrangler / Developer Advocate
https://blogs.gnome.org/aklapper/
[ Crossposting my wikitech-ambassadors post from y'day for those you
active on different wikis. ]
Hello everyone,
TL:DR;
------
As you are aware from previous postings on this list [1] [2] [3] [4] [5]
[6], we have been progressively replacing Tidy with RemexHtml on all
wikis on the wikimedia cluster. As of today, about 650 wikis have made
the switch that include a number of large wikis. We aim to complete this
switch over on the remaining 250 wikis by end of June 2018. Another 40
or so wikis will be switched on May 2nd.
There are a few large wikis (es, pt, uk, zh especially) that could use
more attention addressing Linter issues so that when we make the switch
end of June, some pages on these wiki don't render differently from how
they do now.
Details:
--------
I started investigating more closely where the remaining large wikis are
with respect to the linter issues (high priority categories on the
Special:LintErrors page) that are pertinent to these wikis. I am listing
below results from running sql queries on quarry.wmflabs.org for these
wikis. If you are a community member on any of these wikis, do try to
address these on your wiki.
15 other large wikis:
See https://quarry.wmflabs.org/query/26474 for counts of linter issues
for each of the 9 categories in the main namespace.
* es, pt, uk, zh wikis have total error counts over 10K and in some
cases, it is usually one category which needs attention.
* vi, ro, sr, sh, ar, tr, id are not too bad but don't seem to have seen
a lot of change which indicates that these wikis aren't looking at
linter issues.
* fr, hu, ja, pl wikis seem to be in good shape overall. There has been
a steady fixing of issues and I think all these will will be in fairly
decent shape for replacing Tidy by end of June.
https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy/FAQ#Simplified_instru…
has some summarized instructions for fixing issues in different categories
English Wikipedia:
See https://quarry.wmflabs.org/query/25665 for counts of linter issues
for each of the 9 categories in the main namespace.
English wp has been making slow and gradual progress. I think overall,
despite there still being ~8300 instances (not pages) that need fixing,
enwp is in pretty good shape for replacing Tidy by end of June.
Commons:
See https://quarry.wmflabs.org/query/25693 for counts of linter issues
for each of the 9 categories in the File (ns6), Gallery (ns0), and
Template (ns10) namespaces.
The vast majority of html5-misnesting errors on commons seem to come
from the use of the {{lang}} template which uses a <span> tag to wrap
content. However, it seems to be extremely common to pass content with
paragraphs into the {{lang}} template. Right now, this doesn't cause any
visible rendering issues and could be ignored temporarily, but we
strongly recommend fixing lang to use <div> or on pages which misuse
{{lang}} this way, replace use of {{lang}} by creating a new template
({{lang-block}} maybe?) that uses a <div> tag.
Some tips:
----------
1. On some wikis, fixing templates usually fixes the problem. Over the
last 6 months, I've personally spent many hours fixing 100s of templates
on 10s of different wikis and can personally attest to the efficacy of
that strategy.
2. A lot of the html5-misnesting errors seem to be from incorrectly
using a <span> tag to wrap content that has paragraphs, lists, tables.
In all these cases, changing them to <div> almost always fixes the problem.
If you need any assistance, please leave a message on
https://www.mediawiki.org/wiki/Help_talk:Extension:Linter. Between 8 am
- 4pm PST, you can also usually find us on IRC on #mediawiki-parsoid.
Thanks,
Subbu.
(on behalf of the Parsing team @ Wikimedia Foundation)
1.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2017-July/001625…
2.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2017-August/0016…
3.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-January/001…
4.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-February/00…
5.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-March/00180…
6.
https://lists.wikimedia.org/pipermail/wikitech-ambassadors/2018-March/00182…
I have now attempted to install the Parsoid service locally under your account following their official documentation here:
> https://www.mediawiki.org/wiki/Parsoid/Developer_Setup
under the "parsoid" directory of your account on port 63800. Everything went smoothly during the installation process, however when I attempted to run the service I have received the following error:
17 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "start" "--production"
18 verbose node v6.13.0
19 verbose npm v5.7.1
20 error code ELIFECYCLE
21 error errno 1
22 error parsoid(a)0.9.0 start: `service-runner`
22 error Exit status 1
23 error Failed at the parsoid(a)0.9.0 start script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]
Then, I attempted multiple times to start the services using "yarn" instead of "npm", but once again the tool was unable to start properly,here is the trace of the error:
Trace:
Error: Command failed.
Exit code: 1
Command: sh
Arguments: -c service-runner
Directory: /home/makercom/parsoid
Output:
at ProcessTermError.Error (native)
at ProcessTermError.MessageError (/home/makercom/.yarn/lib/cli.js:186:110)
at new ProcessTermError (/home/makercom/.yarn/lib/cli.js:226:113)
at ChildProcess.<anonymous> (/home/makercom/.yarn/lib/cli.js:30281:17)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:920:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:230:5)
Please help
Best Regards
Jacksonplp
发送自 Windows 10 版邮件<https://go.microsoft.com/fwlink/?LinkId=550986>应用
The RFC discussion starts 2 hours earlier than usual today.
So, if you are interested, join #wikimedia-office.
--
Daniel Kinzler
Principal Platform Engineer
Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.
