Hi,
Nicolas (cc-ed) told me that he launched an upload on Tuedsay 24 around 17h20/17h30 (CEST), but that nothing happened.
I was unsure what to look for and where… Is GWToolset supposed to log something somewhere?
Of course, it would log the actual uploads [1] but apart from that?
If there is some internal log available to someone, could you kindly have a look?
Thanks :)
[1] In this case https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Tounoki&am...
Sorry, no.
It is highly likely that the upload fell over on the first record due to a problem with the xml, such as missing the xml header, or not being wrapped in a parent <records> tag.
Debugging is a pain due to this user unfriendliness. My tips would be: 1. Run the xml through the validator (linked on the first page of the GWT forms). 2. Run a small initial set, possibly just 1 or 2 records. Get that working. 3. Run a first tranche, say 100 records. 4. Go full belt. If the upload falls over, check the last successful upload and check over the next record in your input xml file.
Try posting the first couple of records in your xml file here for feedback.
EVERYONE PLEASE NOTE
As the tool falls over with no feedback, I have often had partial uploads. Always check that the uploaded numbers are what you expected. If not, GWT probably fell over at some point without telling you.
Fae
On 26 June 2014 17:45, Jean-Frédéric jeanfrederic.wiki@gmail.com wrote:
Hi,
Nicolas (cc-ed) told me that he launched an upload on Tuedsay 24 around 17h20/17h30 (CEST), but that nothing happened.
I was unsure what to look for and where… Is GWToolset supposed to log something somewhere?
Of course, it would log the actual uploads [1] but apart from that?
If there is some internal log available to someone, could you kindly have a look?
Thanks :)
[1] In this case https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Tounoki&am...
-- Jean-Frédéric
Glamtools mailing list Glamtools@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/glamtools
gwtoolset does log batch uploads to the runJobs.log and any issues it runs into during that upload are recorded there. i was able to access that log on beta, but on production, i have no idea. hashar recommended that we create a separate gwtoolset log, and i agree that we should, but that still wouldn't help an end user.
when we started development, echo, http://www.mediawiki.org/wiki/Echo_(Notifications), wasn't available, so we decided not to spam a user’s talk page with updates from gwtoolset. now that echo is available, it makes sense to create some notification system with it. by happenstance, bawolff created this bug today https://bugzilla.wikimedia.org/show_bug.cgi?id=67144 that makes this request.
fae’s tips below are good. i would add that they should first be tried out on beta before attempting to run the batch on production. if the batch worked on beta, and not commons, then my concern is that something in the production environment may be creating the issue.
in either case, it would be good to see the xml file and metadata mapping Nicolas was using so that i can test it out in my dev environment and see if i can spot the issue. might be best to create a ticket, attach the xml file to it and a link to the metatdata mapping he was using.
with kind regards, dan
On Jun 26, 2014, at 19:08 , Fæ faewik@gmail.com wrote:
Sorry, no.
It is highly likely that the upload fell over on the first record due to a problem with the xml, such as missing the xml header, or not being wrapped in a parent <records> tag.
Debugging is a pain due to this user unfriendliness. My tips would be:
- Run the xml through the validator (linked on the first page of the
GWT forms). 2. Run a small initial set, possibly just 1 or 2 records. Get that working. 3. Run a first tranche, say 100 records. 4. Go full belt. If the upload falls over, check the last successful upload and check over the next record in your input xml file.
Try posting the first couple of records in your xml file here for feedback.
EVERYONE PLEASE NOTE
As the tool falls over with no feedback, I have often had partial uploads. Always check that the uploaded numbers are what you expected. If not, GWT probably fell over at some point without telling you.
Fae
On 26 June 2014 17:45, Jean-Frédéric jeanfrederic.wiki@gmail.com wrote:
Hi,
Nicolas (cc-ed) told me that he launched an upload on Tuedsay 24 around 17h20/17h30 (CEST), but that nothing happened.
I was unsure what to look for and where… Is GWToolset supposed to log something somewhere?
Of course, it would log the actual uploads [1] but apart from that?
If there is some internal log available to someone, could you kindly have a look?
Thanks :)
[1] In this case https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Tounoki&am...
-- Jean-Frédéric
Glamtools mailing list Glamtools@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/glamtools
-- faewik@gmail.com https://commons.wikimedia.org/wiki/User:Fae Personal and confidential, please do not circulate or re-quote.
Glamtools mailing list Glamtools@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/glamtools
On 6/26/14, dan-nl dan.entous.wikimedia@gmail.com wrote:
gwtoolset does log batch uploads to the runJobs.log and any issues it runs into during that upload are recorded there. i was able to access that log on beta, but on production, i have no idea. hashar recommended that we create a separate gwtoolset log, and i agree that we should, but that still wouldn't help an end user.
If its important for debugging something, the people who have access to that log on production may be able to give you (you, as in Dan as the developer of the tool) snippets if you ask nicely on #wikimedia-dev. To get full access to that log you probably at the very least need to sign an NDA, plus probably go through five billion hoops. Anyhow, definitely not something that can help the end users.
--bawolff