Dear Ariel,
I have a questions about the dump file for the `sites' table.
1) Dump file exists, and contains meaningful data.
(shell)$ rsync ftpmirror.your.org::wikimedia-dumps/wikidatawiki/20140508/ | grep sites -rw-rw-r-- 15,817 2014/05/11 00:15:46 wikidatawiki-20140508-sites.sql.gz (shell)$ rsync --progress ftpmirror.your.org::wikimedia-dumps/wikidatawiki/20140508/wikidatawiki-20140508-sites.sql.gz . wikidatawiki-20140508-sites.sql.gz 15,817 100% 15.08MB/s 0:00:00 (xfr#1, to-chk=0/1) (shell)$ zless wikidatawiki-20140508-sites.sql.gz
2) It is mentioned in `dumpruninfo.txt'
(shell)$ rsync --progress ftpmirror.your.org::wikimedia-dumps/wikidatawiki/20140508/dumpruninfo.txt . dumpruninfo.txt 2,293 100% 2.19MB/s 0:00:00 (xfr#1, to-chk=0/1) (shell)$ cat dumpruninfo.txt | grep sitesname:sitestable; status:done; updated:2014-05-11 04:15:48 name:sitestatstable; status:done; updated:2014-05-08 17:39:36
3) But it is not mentioned in `md5sums.txt'
(shell)$ rsync --progress ftpmirror.your.org::wikimedia-dumps/wikidatawiki/20140508/wikidatawiki-20140508-md5sums.txt . wikidatawiki-20140508-md5sums.txt 2,189 100% 2.09MB/s 0:00:00 (xfr#1, to-chk=0/1) (shell)$ cat wikidatawiki-20140508-md5sums.txt | sites bash: sites: command not found
Sincerely Yours, Kent
On May 13, 2014 2:28 AM, "wp mirror" wpmirrordev@gmail.com wrote:
(shell)$ rsync --progress ftpmirror.your.org::wikimedia-dumps/wikidatawiki/20140508/wikidatawiki-20140508-md5sums.txt
.
wikidatawiki-20140508-md5sums.txt 2,189 100% 2.09MB/s 0:00:00 (xfr#1, to-chk=0/1) (shell)$ cat wikidatawiki-20140508-md5sums.txt | sites bash: sites: command not found
Looks like something's missing in your pipeline. md5sums, grep, etc.
Not testing myself as I'm writing from a phone atm.
-Jeremy
Dear Jeremy,
Thank you for pointing out the typo.
The checksum for the `sites' table really is missing. Here in more detail, we see:
(shell)$ cat dumpruninfo.txt | grep sites name:sitestable; status:done; updated:2014-05-11 04:15:48 name:sitestatstable; status:done; updated:2014-05-08 17:39:36
(shell)$ cat wikidatawiki-20140508-md5sums.txt d5ceb46976922ae37bc01a38b72df6c7 wikidatawiki-20140508-site_stats.sql.gz 0545fcd5aab791d181e5622ec4b54323 wikidatawiki-20140508-image.sql.gz b949d4c0aacad32a89d57035e98cea9e wikidatawiki-20140508-pagelinks.sql.gz 7408668171e2092b644d14cc562f4bc3 wikidatawiki-20140508-categorylinks.sql.gz 50634b381bbb2f08568979018e142f4d wikidatawiki-20140508-imagelinks.sql.gz 5ba16e2c14e806188e05b54a8e9282fb wikidatawiki-20140508-templatelinks.sql.gz bd3b09b372eef139fdb3456d37b95303 wikidatawiki-20140508-externallinks.sql.gz 3de576aab1bd8121c81ba4c6ec42996f wikidatawiki-20140508-langlinks.sql.gz 5dcf1cdad9ce56ca34a7599c047111d2 wikidatawiki-20140508-interwiki.sql.gz 6aafd3489caa53231ce5b5c290e22805 wikidatawiki-20140508-user_groups.sql.gz 0b32b77075d971575ead22fe3a1307fc wikidatawiki-20140508-category.sql.gz 122911ee0743812113c1445975e1db3e wikidatawiki-20140508-page.sql.gz d0f8c5807acd62a1303513030429431d wikidatawiki-20140508-page_restrictions.sql.gz 071efe2e58e3d2f8a6c15c79670703a1 wikidatawiki-20140508-page_props.sql.gz 8c971e1bc254fd54990a753e4d95620d wikidatawiki-20140508-protected_titles.sql.gz 44de15a9e110248339bfa9b1c747dbf3 wikidatawiki-20140508-redirect.sql.gz 8d142a14b2e2db09e434b730af472a6b wikidatawiki-20140508-iwlinks.sql.gz d42636a45b10f424155ab022cf441514 wikidatawiki-20140508-all-titles-in-ns0.gz 94029b3b04f56a2c93f05f4bfa47cbef wikidatawiki-20140508-all-titles.gz 579963ee27dcc3073c313a05200215da wikidatawiki-20140508-abstract.xml 7f0a87e0cc3003f4d8b814915d77b886 wikidatawiki-20140508-stub-meta-history.xml.gz f5534bdefe348cce3a254219c2b26457 wikidatawiki-20140508-stub-meta-current.xml.gz 099beec80cd91c2960c258f08af18adf wikidatawiki-20140508-stub-articles.xml.gz 47feb057feb1d0476da5a273ecf9cb60 wikidatawiki-20140508-pages-articles.xml.bz2 8bd673897d715f75a94f59f72b47af69 wikidatawiki-20140508-pages-meta-current.xml.bz2 b3cda5c87bb606cb1c52447753b12da1 wikidatawiki-20140508-pages-logging.xml.gz d543c0146082e34beda9c61bc7063383 wikidatawiki-20140508-wb_items_per_site.sql.gz f3eff696f3e358e099963f6781860ba5 wikidatawiki-20140508-wb_terms.sql.gz a7b5fcc85b0a01c15283d4b006fb7232 wikidatawiki-20140508-wb_entity_per_page.sql.gz
(shell)$ cat wikidatawiki-20140508-md5sums.txt | grep sites (shell)$
Sincerely Yours, Kent
On Tue, May 13, 2014 at 2:53 AM, Jeremy Baron jeremy@tuxmachine.com wrote:
On May 13, 2014 2:28 AM, "wp mirror" wpmirrordev@gmail.com wrote:
(shell)$ rsync --progress ftpmirror.your.org::wikimedia-dumps/wikidatawiki/20140508/wikidatawiki-20140508-md5sums.txt
.
wikidatawiki-20140508-md5sums.txt 2,189 100% 2.09MB/s 0:00:00 (xfr#1, to-chk=0/1) (shell)$ cat wikidatawiki-20140508-md5sums.txt | sites bash: sites: command not found
Looks like something's missing in your pipeline. md5sums, grep, etc.
Not testing myself as I'm writing from a phone atm.
-Jeremy
Dear Jeremy,
I can confirm that checksums for the `sites' table are also missing from: `enwiki-20140502-md5sums.txt', and `simplewiki-20140501-md5sums.txt'.
Sincerely Yours, Kent
On Tue, May 13, 2014 at 3:29 AM, wp mirror wpmirrordev@gmail.com wrote:
Dear Jeremy,
Thank you for pointing out the typo.
The checksum for the `sites' table really is missing. Here in more detail, we see:
(shell)$ cat dumpruninfo.txt | grep sites
name:sitestable; status:done; updated:2014-05-11 04:15:48 name:sitestatstable; status:done; updated:2014-05-08 17:39:36
(shell)$ cat wikidatawiki-20140508-md5sums.txt d5ceb46976922ae37bc01a38b72df6c7 wikidatawiki-20140508-site_stats.sql.gz 0545fcd5aab791d181e5622ec4b54323 wikidatawiki-20140508-image.sql.gz b949d4c0aacad32a89d57035e98cea9e wikidatawiki-20140508-pagelinks.sql.gz 7408668171e2092b644d14cc562f4bc3 wikidatawiki-20140508-categorylinks.sql.gz 50634b381bbb2f08568979018e142f4d wikidatawiki-20140508-imagelinks.sql.gz 5ba16e2c14e806188e05b54a8e9282fb wikidatawiki-20140508-templatelinks.sql.gz bd3b09b372eef139fdb3456d37b95303 wikidatawiki-20140508-externallinks.sql.gz 3de576aab1bd8121c81ba4c6ec42996f wikidatawiki-20140508-langlinks.sql.gz 5dcf1cdad9ce56ca34a7599c047111d2 wikidatawiki-20140508-interwiki.sql.gz 6aafd3489caa53231ce5b5c290e22805 wikidatawiki-20140508-user_groups.sql.gz 0b32b77075d971575ead22fe3a1307fc wikidatawiki-20140508-category.sql.gz 122911ee0743812113c1445975e1db3e wikidatawiki-20140508-page.sql.gz d0f8c5807acd62a1303513030429431d wikidatawiki-20140508-page_restrictions.sql.gz 071efe2e58e3d2f8a6c15c79670703a1 wikidatawiki-20140508-page_props.sql.gz 8c971e1bc254fd54990a753e4d95620d wikidatawiki-20140508-protected_titles.sql.gz 44de15a9e110248339bfa9b1c747dbf3 wikidatawiki-20140508-redirect.sql.gz 8d142a14b2e2db09e434b730af472a6b wikidatawiki-20140508-iwlinks.sql.gz d42636a45b10f424155ab022cf441514 wikidatawiki-20140508-all-titles-in-ns0.gz 94029b3b04f56a2c93f05f4bfa47cbef wikidatawiki-20140508-all-titles.gz 579963ee27dcc3073c313a05200215da wikidatawiki-20140508-abstract.xml 7f0a87e0cc3003f4d8b814915d77b886 wikidatawiki-20140508-stub-meta-history.xml.gz f5534bdefe348cce3a254219c2b26457 wikidatawiki-20140508-stub-meta-current.xml.gz 099beec80cd91c2960c258f08af18adf wikidatawiki-20140508-stub-articles.xml.gz 47feb057feb1d0476da5a273ecf9cb60 wikidatawiki-20140508-pages-articles.xml.bz2 8bd673897d715f75a94f59f72b47af69 wikidatawiki-20140508-pages-meta-current.xml.bz2 b3cda5c87bb606cb1c52447753b12da1 wikidatawiki-20140508-pages-logging.xml.gz d543c0146082e34beda9c61bc7063383 wikidatawiki-20140508-wb_items_per_site.sql.gz f3eff696f3e358e099963f6781860ba5 wikidatawiki-20140508-wb_terms.sql.gz a7b5fcc85b0a01c15283d4b006fb7232 wikidatawiki-20140508-wb_entity_per_page.sql.gz
(shell)$ cat wikidatawiki-20140508-md5sums.txt | grep sites (shell)$
Sincerely Yours, Kent
On Tue, May 13, 2014 at 2:53 AM, Jeremy Baron jeremy@tuxmachine.comwrote:
On May 13, 2014 2:28 AM, "wp mirror" wpmirrordev@gmail.com wrote:
(shell)$ rsync --progress ftpmirror.your.org::wikimedia-dumps/wikidatawiki/20140508/wikidatawiki-20140508-md5sums.txt
.
wikidatawiki-20140508-md5sums.txt 2,189 100% 2.09MB/s 0:00:00 (xfr#1, to-chk=0/1) (shell)$ cat wikidatawiki-20140508-md5sums.txt | sites bash: sites: command not found
Looks like something's missing in your pipeline. md5sums, grep, etc.
Not testing myself as I'm writing from a phone atm.
-Jeremy
xmldatadumps-l@lists.wikimedia.org