I installed the latest release (1.2) of the DataTransfer Extension via Git. I installed it on a 1.31 MW system per the INSTALL file. I went to the Spezial:XMLview page as instructed. I selected one of the categories and the main namespace and hit 'view XML'. After a few seconds (< 10) I got a 500 page. I see the 500 error in the PHP logs (so this isn't a web timeout). The PHP debug logs show only the following:
PHP Notice: Undefined index: 0505261 in $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php on line 139
($IP is the full installation path, redacted for security reasons)
Help?
On Thu, Jun 24, 2021 at 4:39 AM otheus uibk otheus.uibk@gmail.com wrote:
I installed the latest release (1.2) of the DataTransfer Extension via Git. I installed it on a 1.31 MW system per the INSTALL file. I went to the Spezial:XMLview page as instructed. I selected one of the categories and the main namespace and hit 'view XML'. After a few seconds (< 10) I got a 500 page. I see the 500 error in the PHP logs (so this isn't a web timeout). The PHP debug logs show only the following:
PHP Notice: Undefined index: 0505261 in $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php on line 139
($IP is the full installation path, redacted for security reasons)
Help?
I believe you need to run maintenance/update.php after you install an extension. Can you confirm you have run it?
(I find it is best to run maintenance/update.php after any maintenance, including a Mediawiki upgrade, extension installation, extension removal, etc).
Jeff
Thank Jeffrey, that's a good eye, but unfortunately, the problem persists.
Indeed, update.php had not completed *successfully*. There appears to be a mistake in the documentation (or a shorthand in which it is assumed the admin understands the syntax is not literal). The INSTALL page says to run:
composer require "phpoffice/phpexcel:~1.8"
I had ran this command verbatim. Initially, composer would not complete due to another unrelated module which specified unit-tests in its require/manifest, but the module was not deployed with its test directory. Having fixed that problem in multiple places, the composer command ran successfully. Unfortunately the tilde seems to have been the cause of the failure for running update:
phpoffice/phpexcel: 1.8.2 installed, ~1.8 required. Error: your composer.lock file is not up to date. Run "composer update --no-dev" to install newer dependencies
This is really quite strange. I looked at the composer documentation, and there is definitely some confusion there. In one section, it says that the version attribute must match a regular expression, which does not include the ~. Another section (https://getcomposer.org/doc/articles/versions.md) indicates this is perfectly acceptable, and that "~1.8" should mean ">=1.8.0". However, none of the following combinations in composer.json worked: - "1.8*" - "1.8.*" - "1.8.0" - ">=1.8.0" In the end, I had to change composer.json with the exact version number composer had previously installed. Then I could run update. I also updated the language cache.
On Thu, Jun 24, 2021 at 2:11 PM Jeffrey Walton noloader@gmail.com wrote:
On Thu, Jun 24, 2021 at 4:39 AM otheus uibk otheus.uibk@gmail.com wrote:
I installed the latest release (1.2) of the DataTransfer Extension via
Git. I installed it on a 1.31 MW system per the INSTALL file. I went to the Spezial:XMLview page as instructed. I selected one of the categories and the main namespace and hit 'view XML'. After a few seconds (< 10) I got a 500 page. I see the 500 error in the PHP logs (so this isn't a web timeout). The PHP debug logs show only the following:
PHP Notice: Undefined index: 0505261 in
$IP/local/extensions/DataTransfer/includes/DT_PageStructure.php on line 139
($IP is the full installation path, redacted for security reasons)
Help?
I believe you need to run maintenance/update.php after you install an extension. Can you confirm you have run it?
(I find it is best to run maintenance/update.php after any maintenance, including a Mediawiki upgrade, extension installation, extension removal, etc).
Jeff
Hi,
do yourself a favour and use "phpspreadheet" as in "phpoffice/phpspreadsheet": "~1". :)
Cheers,
Karsten
Karsten / MediaWiki and Semantic MediaWiki enthusiast / → https://professional.wiki/
Am 24.06.21 um 15:39 schrieb otheus uibk:
Thank Jeffrey, that's a good eye, but unfortunately, the problem persists.
Indeed, update.php had not completed *successfully*. There appears to be a mistake in the documentation (or a shorthand in which it is assumed the admin understands the syntax is not literal). The INSTALL page says to run:
composer require "phpoffice/phpexcel:~1.8"
I had ran this command verbatim. Initially, composer would not complete due to another unrelated module which specified unit-tests in its require/manifest, but the module was not deployed with its test directory. Having fixed that problem in multiple places, the composer command ran successfully. Unfortunately the tilde seems to have been the cause of the failure for running update:
phpoffice/phpexcel: 1.8.2 installed, ~1.8 required. Error: your composer.lock file is not up to date. Run "composer update --no-dev" to install newer dependencies
This is really quite strange. I looked at the composer documentation, and there is definitely some confusion there. In one section, it says that the version attribute must match a regular expression, which does not include the ~. Another section (https://getcomposer.org/doc/articles/versions.md https://getcomposer.org/doc/articles/versions.md) indicates this is perfectly acceptable, and that "~1.8" should mean ">=1.8.0". However, none of the following combinations in composer.json worked: - "1.8*" - "1.8.*" - "1.8.0" - ">=1.8.0" In the end, I had to change composer.json with the exact version number composer had previously installed. Then I could run update. I also updated the language cache.
On Thu, Jun 24, 2021 at 2:11 PM Jeffrey Walton <noloader@gmail.com mailto:noloader@gmail.com> wrote:
On Thu, Jun 24, 2021 at 4:39 AM otheus uibk <otheus.uibk@gmail.com <mailto:otheus.uibk@gmail.com>> wrote: > > I installed the latest release (1.2) of the DataTransfer Extension via Git. I installed it on a 1.31 MW system per the INSTALL file. I went to the Spezial:XMLview page as instructed. I selected one of the categories and the main namespace and hit 'view XML'. After a few seconds (< 10) I got a 500 page. I see the 500 error in the PHP logs (so this isn't a web timeout). The PHP debug logs show only the following: > > PHP Notice: Undefined index: 0505261 in $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php on line 139 > > ($IP is the full installation path, redacted for security reasons) > > Help? I believe you need to run maintenance/update.php after you install an extension. Can you confirm you have run it? (I find it is best to run maintenance/update.php after any maintenance, including a Mediawiki upgrade, extension installation, extension removal, etc). Jeff-- Otheus otheus.uibk@gmail.com mailto:otheus.uibk@gmail.com otheus.shelling@uibk.ac.at mailto:otheus.shelling@uibk.ac.at
MediaWiki-l mailing list -- mediawiki-l@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/mediawiki-l.lists.wikimedia.org/
On Thu, Jun 24, 2021 at 9:40 AM otheus uibk otheus.uibk@gmail.com wrote:
Thank Jeffrey, that's a good eye, but unfortunately, the problem persists.
Indeed, update.php had not completed *successfully*. There appears to be a mistake in the documentation (or a shorthand in which it is assumed the admin understands the syntax is not literal). The INSTALL page says to run:
composer require "phpoffice/phpexcel:~1.8"I had ran this command verbatim. Initially, composer would not complete due to another unrelated module which specified unit-tests in its require/manifest, but the module was not deployed with its test directory. Having fixed that problem in multiple places, the composer command ran successfully. Unfortunately the tilde seems to have been the cause of the failure for running update:
phpoffice/phpexcel: 1.8.2 installed, ~1.8 required. Error: your composer.lock file is not up to date. Run "composer update --no-dev" to install newer dependenciesThis is really quite strange. I looked at the composer documentation, and there is definitely some confusion there. In one section, it says that the version attribute must match a regular expression, which does not include the ~. Another section (https://getcomposer.org/doc/articles/versions.md) indicates this is perfectly acceptable, and that "~1.8" should mean ">=1.8.0". However, none of the following combinations in composer.json worked: - "1.8*" - "1.8.*" - "1.8.0" - ">=1.8.0" In the end, I had to change composer.json with the exact version number composer had previously installed. Then I could run update. I also updated the language cache.
Don't get me started on the dev tools on a production server...
Here's what I do for composer:
$ sudo apt-get install -y composer
$ sudo su - # cd /var/www/html/w # rm -rf /var/www/html/w/vendor # php -d extension=phar.so composer.phar update --no-dev # exit
$ sudo apt-get remove -y composer
Then, fix ownership and permissions on the files. We use root:www-data, 0750 and friends. Root owns everything and gets read/write. The webserver is the group owner and only gets read. (The webserver gets read/write on the upload/ and sessions/ directories).
Jeff
Hi Jeff,
Are you suggesting that the root cause is a problem with a dependency provided by composer? Or are you trying to eliminate that as a possibility?
Is there anyway to turn on stack-trace dumping-on-warning or errors in cases like these? I know how to turn on debugging, but that doesn't automatically generate stack traces on a crash.
On Thu, Jun 24, 2021 at 6:18 PM Jeffrey Walton noloader@gmail.com wrote:
On Thu, Jun 24, 2021 at 9:40 AM otheus uibk otheus.uibk@gmail.com wrote:
Thank Jeffrey, that's a good eye, but unfortunately, the problem
persists.
Indeed, update.php had not completed *successfully*. There appears to be
a mistake in the documentation (or a shorthand in which it is assumed the admin understands the syntax is not literal). The INSTALL page says to run:
composer require "phpoffice/phpexcel:~1.8"I had ran this command verbatim. Initially, composer would not complete
due to another unrelated module which specified unit-tests in its require/manifest, but the module was not deployed with its test directory. Having fixed that problem in multiple places, the composer command ran successfully. Unfortunately the tilde seems to have been the cause of the failure for running update:
phpoffice/phpexcel: 1.8.2 installed, ~1.8 required. Error: your composer.lock file is not up to date. Run "composerupdate --no-dev" to install newer dependencies
This is really quite strange. I looked at the composer documentation,
and there is definitely some confusion there. In one section, it says that the version attribute must match a regular expression, which does not include the ~. Another section ( https://getcomposer.org/doc/articles/versions.md) indicates this is perfectly acceptable, and that "~1.8" should mean ">=1.8.0". However, none of the following combinations in composer.json worked:
- "1.8*" - "1.8.*" - "1.8.0" - ">=1.8.0"In the end, I had to change composer.json with the exact version number
composer had previously installed. Then I could run update. I also updated the language cache.
Don't get me started on the dev tools on a production server...
Here's what I do for composer:
$ sudo apt-get install -y composer $ sudo su - # cd /var/www/html/w # rm -rf /var/www/html/w/vendor # php -d extension=phar.so composer.phar update --no-dev # exit $ sudo apt-get remove -y composerThen, fix ownership and permissions on the files. We use root:www-data, 0750 and friends. Root owns everything and gets read/write. The webserver is the group owner and only gets read. (The webserver gets read/write on the upload/ and sessions/ directories).
Jeff _______________________________________________ MediaWiki-l mailing list -- mediawiki-l@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/mediawiki-l.lists.wikimedia.org/
OK, so you get error stack-traces in a *debug* log, and in the error log you get warning notices? *facepalm*
I have a stack-trace which may be relevant.
[error] [YNT6JB0y@yYaYYse7zE8PgAAAIc] /neulatein/index.php?title=Spezial:XML_anzeigen&categories%5BLBI-Authors%5D=on&namespaces%5B0%5D=on ErrorException from line 139 of /sites/ wiki.uibk.ac.at/app/bluespice-20191215/local/extensions/DataTransfer/includes/DT_PageStructure.php: PHP Notice: Undefined index: 0505261 #0 $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php(139): MWExceptionHandler::handleError(integer, string, string, integer, array) #1 $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php(26): DTPageStructure->parsePageContents(string) #2 $IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(138): DTPageStructure::newFromTitle(Title) #3 $IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(230): DTViewXML->getXMLForPage(Title, NULL) #4 $IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(22): DTViewXML->doSpecialViewXML(NULL) #5 $IP/includes/specialpage/SpecialPage.php(565): DTViewXML->execute(NULL) #6 $IP/includes/specialpage/SpecialPageFactory.php(568): SpecialPage->run(NULL) #7 $IP/includes/MediaWiki.php(288): SpecialPageFactory::executePath(Title, RequestContext) #8 $IPincludes/MediaWiki.php(861): MediaWiki->performRequest() #9 $IP/includes/MediaWiki.php(524): MediaWiki->main() #10 $IP/index.php(42): MediaWiki->run() #11 {main}
That index is very suspicious. Perhaps the page content is so big that it triggers some kind of numeric overflow?
On Thu, Jun 24, 2021 at 11:18 PM Otheus otheus@gmail.com wrote:
Hi Jeff,
Are you suggesting that the root cause is a problem with a dependency provided by composer? Or are you trying to eliminate that as a possibility?
Is there anyway to turn on stack-trace dumping-on-warning or errors in cases like these? I know how to turn on debugging, but that doesn't automatically generate stack traces on a crash.
On Thu, Jun 24, 2021 at 6:18 PM Jeffrey Walton noloader@gmail.com wrote:
On Thu, Jun 24, 2021 at 9:40 AM otheus uibk otheus.uibk@gmail.com wrote:
Thank Jeffrey, that's a good eye, but unfortunately, the problem
persists.
Indeed, update.php had not completed *successfully*. There appears to
be a mistake in the documentation (or a shorthand in which it is assumed the admin understands the syntax is not literal). The INSTALL page says to run:
composer require "phpoffice/phpexcel:~1.8"I had ran this command verbatim. Initially, composer would not complete
due to another unrelated module which specified unit-tests in its require/manifest, but the module was not deployed with its test directory. Having fixed that problem in multiple places, the composer command ran successfully. Unfortunately the tilde seems to have been the cause of the failure for running update:
phpoffice/phpexcel: 1.8.2 installed, ~1.8 required. Error: your composer.lock file is not up to date. Run "composerupdate --no-dev" to install newer dependencies
This is really quite strange. I looked at the composer documentation,
and there is definitely some confusion there. In one section, it says that the version attribute must match a regular expression, which does not include the ~. Another section ( https://getcomposer.org/doc/articles/versions.md) indicates this is perfectly acceptable, and that "~1.8" should mean ">=1.8.0". However, none of the following combinations in composer.json worked:
- "1.8*" - "1.8.*" - "1.8.0" - ">=1.8.0"In the end, I had to change composer.json with the exact version number
composer had previously installed. Then I could run update. I also updated the language cache.
Don't get me started on the dev tools on a production server...
Here's what I do for composer:
$ sudo apt-get install -y composer $ sudo su - # cd /var/www/html/w # rm -rf /var/www/html/w/vendor # php -d extension=phar.so composer.phar update --no-dev # exit $ sudo apt-get remove -y composerThen, fix ownership and permissions on the files. We use root:www-data, 0750 and friends. Root owns everything and gets read/write. The webserver is the group owner and only gets read. (The webserver gets read/write on the upload/ and sessions/ directories).
Jeff _______________________________________________ MediaWiki-l mailing list -- mediawiki-l@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/mediawiki-l.lists.wikimedia.org/
-- Otheus otheus@gmail.com +43.699.1049.7813 _______________________________________________ MediaWiki-l mailing list -- mediawiki-l@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/mediawiki-l.lists.wikimedia.org/
On Thu, Jun 24, 2021 at 5:41 PM otheus uibk otheus.uibk@gmail.com wrote:
OK, so you get error stack-traces in a *debug* log, and in the error log you get warning notices? *facepalm*
I have a stack-trace which may be relevant.
[error] [YNT6JB0y@yYaYYse7zE8PgAAAIc] /neulatein/index.php?title=Spezial:XML_anzeigen&categories%5BLBI-Authors%5D=on&namespaces%5B0%5D=on ErrorException from line 139 of /sites/wiki.uibk.ac.at/app/bluespice-20191215/local/extensions/DataTransfer/includes/DT_PageStructure.php: PHP Notice: Undefined index: 0505261 #0 $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php(139): MWExceptionHandler::handleError(integer, string, string, integer, array) #1 $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php(26): DTPageStructure->parsePageContents(string) #2 $IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(138): DTPageStructure::newFromTitle(Title) #3 $IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(230): DTViewXML->getXMLForPage(Title, NULL) #4 $IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(22): DTViewXML->doSpecialViewXML(NULL) #5 $IP/includes/specialpage/SpecialPage.php(565): DTViewXML->execute(NULL) #6 $IP/includes/specialpage/SpecialPageFactory.php(568): SpecialPage->run(NULL) #7 $IP/includes/MediaWiki.php(288): SpecialPageFactory::executePath(Title, RequestContext) #8 $IPincludes/MediaWiki.php(861): MediaWiki->performRequest() #9 $IP/includes/MediaWiki.php(524): MediaWiki->main() #10 $IP/index.php(42): MediaWiki->run() #11 {main}
That index is very suspicious. Perhaps the page content is so big that it triggers some kind of numeric overflow?
It looks like (to me) this is the problem:
PHP Notice: Undefined index ...
The extension needs to be updated for the version of PHP and Mediawiki you are using.
I would disable it in LocalSettings.php until the author updates it. Or, if you know PHP, you can fix it yourself.
Also see https://stackoverflow.com/a/4261200 and https://www.mediawiki.org/wiki/Extension:Data_Transfer .
Jeff
Jeffery, Thanks for being helpful -- I appreciate your helpfulness. This mailing list is the mechanism listed by the authors for this extension to receive help and file bug reports. The Mediawiki version I am using is included within the explicitly supported versions of the extension. Therefore, I have encountered some kind of bug. If you cannot help me resolve this bug yourself, please stay out of the conversation from here on out so that someone with specific knowledge on the topic may assist. Thank you.
On Fri, Jun 25, 2021 at 1:53 AM Jeffrey Walton noloader@gmail.com wrote:
On Thu, Jun 24, 2021 at 5:41 PM otheus uibk otheus.uibk@gmail.com wrote:
OK, so you get error stack-traces in a *debug* log, and in the error log
you get warning notices? *facepalm*
I have a stack-trace which may be relevant.
[error] [YNT6JB0y@yYaYYse7zE8PgAAAIc]
/neulatein/index.php?title=Spezial:XML_anzeigen&categories%5BLBI-Authors%5D=on&namespaces%5B0%5D=on ErrorException from line 139 of /sites/ wiki.uibk.ac.at/app/bluespice-20191215/local/extensions/DataTransfer/includes/DT_PageStructure.php: PHP Notice: Undefined index: 0505261
#0 $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php(139):
MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 $IP/local/extensions/DataTransfer/includes/DT_PageStructure.php(26):
DTPageStructure->parsePageContents(string)
#2
$IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(138): DTPageStructure::newFromTitle(Title)
#3
$IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(230): DTViewXML->getXMLForPage(Title, NULL)
#4
$IP/local/extensions/DataTransfer/includes/specials/DT_ViewXML.php(22): DTViewXML->doSpecialViewXML(NULL)
#5 $IP/includes/specialpage/SpecialPage.php(565):
DTViewXML->execute(NULL)
#6 $IP/includes/specialpage/SpecialPageFactory.php(568):
SpecialPage->run(NULL)
#7 $IP/includes/MediaWiki.php(288):
SpecialPageFactory::executePath(Title, RequestContext)
#8 $IPincludes/MediaWiki.php(861): MediaWiki->performRequest() #9 $IP/includes/MediaWiki.php(524): MediaWiki->main() #10 $IP/index.php(42): MediaWiki->run() #11 {main}
That index is very suspicious. Perhaps the page content is so big that
it triggers some kind of numeric overflow?
It looks like (to me) this is the problem:
PHP Notice: Undefined index ...The extension needs to be updated for the version of PHP and Mediawiki you are using.
I would disable it in LocalSettings.php until the author updates it. Or, if you know PHP, you can fix it yourself.
Also see https://stackoverflow.com/a/4261200 and https://www.mediawiki.org/wiki/Extension:Data_Transfer .
Jeff _______________________________________________ MediaWiki-l mailing list -- mediawiki-l@lists.wikimedia.org List information: https://lists.wikimedia.org/postorius/lists/mediawiki-l.lists.wikimedia.org/
Hi Otheus,
Sorry for not responding before. I'm the main author of Data Transfer, as you probably know. I haven't heard of either of those problems happening before, I don't think: either the 500 error or that PHP notice. I'm guessing that the two are related, and my further guess is that the problem is somehow due to unclosed curly brackets in one or more of the pages - i.e., there is a "{{" that has no corresponding "}}". Which should not crash the extension, of course, but maybe that's what is happening.
Have you tried Special:ViewXML for other categories or namespaces on your wiki, and if so, do those work?
-Yaron
mediawiki-l@lists.wikimedia.org