Hi,
I am referring to the dump at: http://download.wikimedia.org/zhwiki/20080425/
I can only find the categorylinks.sql.gz (only single level), but not the full category table which contains all the categories.
Any data I have missed?
Thanks.
Howard
On May 27, 2008, at 1:42 PM, howard chen wrote:
Hi,
I am referring to the dump at: http://download.wikimedia.org/zhwiki/ 20080425/
I can only find the categorylinks.sql.gz (only single level), but not the full category table which contains all the categories.
Any data I have missed?
thats 'page' (and should be part of content .xml dump).
On Tue, May 27, 2008 at 6:42 AM, howard chen howachen@gmail.com wrote:
Hi,
I am referring to the dump at: http://download.wikimedia.org/zhwiki/20080425/
I can only find the categorylinks.sql.gz (only single level), but not the full category table which contains all the categories.
The category table, if that's what you really want, can be quickly regenerated (if you have categorylinks) with the maintenance/populateCategory.php script. This will give you different category id's from those used on the real wikis, but category id's aren't currently used elsewhere that I know of, so it shouldn't be a problem for now. I don't know if the category table is explicitly included in any of the dumps at present.
But I'm not sure if the category table is what you want. All its contents are basically part of the categorylinks table, it's just a summary table. If you mean the category *pages*, then as Domas says, those are part of the page table, and should be available in the pages-meta-current dump (or pages-meta-history if you prefer).
Hi Domas & Simetrical,
The category table, if that's what you really want, can be quickly regenerated (if you have categorylinks)
The problem I found is : seems categorylinks is out-of-sync with the category current showing on wikipedia (zhwiki at least, I haven't test other languages)
For example, a singer in Hong Kong, Leon Lai
http://zh.wikipedia.org/w/index.php?title=%E9%BB%8E%E6%98%8E&variant=zh-...
at the bottom, you would find he is belonged to 11 categories, however, I run the SQL against (categorylinks.sql, http://download.wikimedia.org/zhwiki/latest/zhwiki-latest-categorylinks.sql.... )
SELECT * FROM `categorylinks` WHERE `cl_sortkey` ='黎明';
I can only get 2 categories (Note: The queried name is the same in both Traditional & Simplified Chinese, even I looked back to a version in 2007, it also contains 11 categories. )
Now, I query one of the category within the 11 category which not appear in above,
SELECT * FROM `categorylinks` WHERE `cl_to` LIKE '1966年出生';
I get 177 rows, which the wikipedia showing 185 items (http://zh.wikipedia.org/wiki/Category:1966%E5%B9%B4%E5%87%BA%E7%94%9F)
The only reason I can think of is those pages are all human edited, and it might not be reflected in categorylinks table.
Howard
2008/5/27 howard chen howachen@gmail.com:
The problem I found is : seems categorylinks is out-of-sync with the category current showing on wikipedia (zhwiki at least, I haven't test other languages)
Database dumps are not perfectly up-to-date. The dump you downloaded is more than a month old and will of course not correspond exactly with the live database.
Hi
On Wed, May 28, 2008 at 1:45 AM, Simetrical Simetrical+wikilist@gmail.com wrote:
2008/5/27 howard chen howachen@gmail.com: Database dumps are not perfectly up-to-date. The dump you downloaded is more than a month old and will of course not correspond exactly with the live database.
Sure I am aware of this, you can compare the following 2 versions:
1. 2008-03-06, http://zh.wikipedia.org/w/index.php?title=%E9%BB%8E%E6%98%8E&oldid=64039..., 11 Categories 2. 2007-12-05, http://zh.wikipedia.org/w/index.php?title=%E9%BB%8E%E6%98%8E&oldid=57315..., 10 Categories
So where are the missing categories as I can only found 2 in the latest SQL (one month ago).
Howard
wikitech-l@lists.wikimedia.org