Hi,
I am aware of the various limits in the NewPP report. I'm trying to determine if we currently we have some max page size (before or after processing).
The documentation on mw.org and en.wp is a bit confusing on the subject and personal experimentation shows that substituting templates allows be to go past the 2MiB page size.
If we don't have such a limit, I'm not exactly sure why do we need the Post‐expand include size limit? Why is output generated by transclusion harder on the parser than output directly in the page?
Thanks, Strainu
On Sat, 2022-02-05 at 18:43 +0200, Strainu wrote:
I am aware of the various limits in the NewPP report. I'm trying to determine if we currently we have some max page size (before or after processing).
The documentation on mw.org and en.wp is a bit confusing on the subject and personal experimentation shows that substituting templates allows be to go past the 2MiB page size.
What does "personal experimentation" mean exactly? There might be exceptions like https://phabricator.wikimedia.org/T188852 but generally speaking, as neither https://noc.wikimedia.org/conf/InitialiseSettings.php.txt nor https://noc.wikimedia.org/conf/CommonSettings.php.txt seem to change the MediaWiki software default setting defined in https://phabricator.wikimedia.org/source/mediawiki/browse/master/includes/De... I'd assume that we're at 2MiB.
See also https://phabricator.wikimedia.org/T189108 about requesting an increase, and https://phabricator.wikimedia.org/T181907#3835654 for some more background.
Cheers, andre
În sâm., 5 feb. 2022 la 19:53, Andre Klapper aklapper@wikimedia.org a scris:
On Sat, 2022-02-05 at 18:43 +0200, Strainu wrote:
I am aware of the various limits in the NewPP report. I'm trying to determine if we currently we have some max page size (before or after processing).
The documentation on mw.org and en.wp is a bit confusing on the subject and personal experimentation shows that substituting templates allows be to go past the 2MiB page size.
What does "personal experimentation" mean exactly? There might be exceptions like https://phabricator.wikimedia.org/T188852 but generally speaking, as neither https://noc.wikimedia.org/conf/InitialiseSettings.php.txt nor https://noc.wikimedia.org/conf/CommonSettings.php.txt seem to change the MediaWiki software default setting defined in https://phabricator.wikimedia.org/source/mediawiki/browse/master/includes/De... I'd assume that we're at 2MiB.
Hey Andre,
Thanks for taking the time to respond to my curiosity during the weekend.
Here is what I experimented with: * for post-parser size (which would have been my first guess given that for templates we count the *Post‐expand include size*) I just measured the size of the .mw-parser-output div. For this, I took the output of [1] (which is a mix of included and substituded templates and is displayed just fine) and it was well over 8MiB. * for pre-parser size, I took the same page [1], added some more templates and started saving while substituting them (e.g. I have 1.5 MiB of text and a few thousand templates which I sustitute in one go). One version which goes over the 2MiB limit is [2].
Now, because of the way those templates are written (pretty verbose and with a lot of whitespace) the difference between the template and the result of the substitution is small, so the page size is just over the limit, but one can imagine a template expanding near 2MiB the limit a couple of times could take a page to ~4MiB without much effort.
I understand from your message that these are bugs and the limit is still enforced. However, I still don't understand the logic in having a limit for wikitext in pages, but a limit for post-expand (if I understand correctly, that is after they go through the parser) in templates. Why not have a single limit set to something like 8-12-16 MiB and counted after all the processing is done?
Thanks again, Strainu
[1] https://ro.wikipedia.org/w/index.php?title=Bunuri_mobile_din_domeniul_%C8%99... [2] https://ro.wikipedia.org/w/index.php?title=Utilizator:Strainu/2&oldid=14...
See also https://phabricator.wikimedia.org/T189108 about requesting an increase, and https://phabricator.wikimedia.org/T181907#3835654 for some more background.
Cheers, andre -- Andre Klapper (he/him) | Bugwrangler / Developer Advocate https://blogs.gnome.org/aklapper/ _______________________________________________ Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
It does indeed look like we don't have good documentation for the details you're interested in. Let's see.
post-parser size […]
As far as I know something like this doesn't exist as a technical limitation in MediaWiki. A page's final HTML can be much bigger than 2 MiB.
Post‐expand include size […] added some more templates and started saving while substituting them […]
That's a really nice approach to explain how the "post-expand include size" works. I like it. However, there are some special cases to consider, most notably Lua modules.
The limits for classic templates work as you described. Every single usage of a template (including all nested sub-templates) is effectively substituted. What's counted in the end is the total size of the resulting wikitext. In other words: not only is a template's source code counted. It's counted as often as the template is used.
Lua source code doesn't count. Lua is executed with other limits in mind (runtime and memory usage). What still counts is the size of the wikitext snippet that's returned by the Lua module. I might miss something else, but I guess the reason you get confusing results is because we can't simulate this process by manually substituting {{#invoke:…}} calls.
Best Thiemo
wikitech-l@lists.wikimedia.org