Xmldatadumps-l

xmldatadumps-l@lists.wikimedia.org

720 discussions

things look ok after 1.19, so...
by Ariel T. Glenn 08 Mar '12

08 Mar '12

I've cranked up all the usual workers and kicked off an en wp run in addition. If there is anything that squeaked by my spot checks we'll know about it soon... Ariel

1 0

Accessing Wikipedia/Wikitravel dumps
by Ashish Mukherjee 05 Mar '12

05 Mar '12

Hi, I would like to access the Wiki dumps for Wikipedia and Wikitravel. Essentially, I am looking to get dumps for some US cities from both these sources for product research. However, it is not very clear at http://dumps.wikimedia.org/backup-index.html, which are the relevant files to pick up. The other question I had was at what level of atomicity data is available in dumps. The Web Service allows us to retrieve a Wiki entry but it's not easily parsed out into different sections or in more granular form. I was wondering if the dumps solve this problem. Appreciate any help about this. Regards, Ashish

6 5

post-1.19 deployment
by Ariel T. Glenn 25 Feb '12

25 Feb '12

As usual there have been a couple of hiccups after the rollout of mw 1.19wmf1 everywhere. In the expctation that there may be other surprises, I'm running just one process over the weekend and will check on it from time to time. Of course if anyone notices something awry, please bring it up. Thanks, Ariel

1 0

dewiki dump 20120221
by Andreas Meier 25 Feb '12

25 Feb '12

Hello, the dewiki dump 20120221 is complettly broken, see http://dumps.wikimedia.org/dewiki/20120221/ . In my opinion the best thing to do is to stop this dump process and make a new dump. I think it makes no sense to restart the 20120221 dump. Best regards Andreas

2 1

Http external storage
by Christian Aistleitner 10 Feb '12

10 Feb '12

Hello, while working towards the xmldumps-backup test suite, I came across ExternalStorageHttp. While it's easy to setup (read-only), testing it is rather a burden. Is ExternalStorageHttp actually used for some MediaWiki at WikiMedia? Kind regards, Christian -- ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ---- Companies' registry: 360296y in Linz Christian Aistleitner Gruendbergstrasze 65a Email: christian(a)quelltextlich.at 4040 Linz, Austria Phone: +43 732 / 26 95 63 Fax: +43 732 / 26 95 63 Homepage: http://quelltextlich.at/ ---------------------------------------------------------------

1 0

old_flags column of MediaWiki's text table
by Christian Aistleitner 09 Feb '12

09 Feb '12

Hello, I am currently developing a test suite for the XML dumps, and I am curious about the specification of text.old_flags in MediaWiki's maintainance/tables.sql. The file describes the 'object' flag as text field contained a serialized PHP object. object either contains multiple versions compressed to achieve a better compression ratio, or it refers to another row where the text can be found. Is the „multiple versions” part still used in some project? If so, how should this be set up [1]? Kind regards, Christian P.S.: In #wikimedia-dev I was told, to bring up the question on this list. If there are further lists, where I should ask, please let me know. [1] Before r6138 (back then still in Article.php not Revision.php), it seems the text was obtained by $object = unserialize( $text ); $text = $object->getItem( $hash ); . There it is somewhat obvious how a single object may return different texts. However, beginning with p6138 it seems the text is simply fetched by $obj = unserialize( $text ); [...] $text = $obj->getText(); . If a single object should return different texts, how does it determine, which text to return? -- ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ---- Companies' registry: 360296y in Linz Christian Aistleitner Gruendbergstrasze 65a Email: christian(a)quelltextlich.at 4040 Linz, Austria Phone: +43 732 / 26 95 63 Fax: +43 732 / 26 95 63 Homepage: http://quelltextlich.at/ ---------------------------------------------------------------

2 2

POTY 2009 plus a few historical dumps now available
by Ariel T. Glenn 02 Feb '12

02 Feb '12

Good morning campers! :-) At the POTY collection link http://dumps.wikimedia.org/other/poty/ you'll notice that the 2009 files have been added. For people who have wanted older (2002 through 2006) dumps, one or two dumps of the projects for most of that period are now online at our new archive link: http://dumps.wikimedia.org/archive/ I'd love to get a couple dumps of each of the projects for the years 2005, 2007 and 2008. If you have an old mirror on a hard drive gathering dust in your closet, give me a shout. Thanks! Hmm, and one other thing while I'm at it, I've been cleaning up our dumps documentation on wikitech, and there's now a rough outline of the dumps history here: http://wikitech.wikimedia.org/view/Dumps/History If people remember any milestones that should be added there, or if you see any glaring errors, please edit, or send me mail with the corrections. Ariel

2 3

Re: [Xmldatadumps-l] [Wikitech-l] Fwd: Old English Wikipedia image dump from 2005
by John Vandenberg 31 Jan '12

31 Jan '12

On Fri, Nov 11, 2011 at 11:18 PM, emijrp <emijrp(a)gmail.com> wrote: > Forwarding... > > ---------- Forwarded message ---------- > From: emijrp <emijrp(a)gmail.com> > Date: 2011/11/11 > Subject: Old English Wikipedia image dump from 2005 > To: wikiteam-discuss(a)googlegroups.com > > > Hi all; > > I want to share with you this Archive Team link[1]. It is an old English > Wikipedia image dump from 2005. One of the last ones, probably, before > Wikimedia Foundation stopped publishing image dumps. Enjoy. > > Regards, > emijrp > > [1] http://www.archive.org/details/wikimedia-image-dump-2005-11 People interested in image dumps may be also interested in my post relating to the GFDL requirements, which I think mean images need to be included in the dumps. https://meta.wikimedia.org/w/index.php?title=Talk:Terms_of_use&diff=prev&ol… excerpt: "..the [GFDL] license requires that someone can download a ''complete'' Transparent copy for one year after the last Opaque copy is distributed. As a result, I believe the BoT needs to ensure that the dumps are available ''and'' that they can be available for one year after WMF turns of the lights on the core servers (it allows 'agents' to provide this service). As Wikipedia contains images, the images are required to be included. .." discussion continues .. https://meta.wikimedia.org/wiki/Talk:Terms_of_use#Right_to_Fork -- John Vandenberg

7 17

Re: [Xmldatadumps-l] Import of an XML dump of the RU Wiktionary with mwdumper / problem
by Sébastien Druon 24 Jan '12

24 Jan '12

Hello! Taking as example "Дом" from the russian wiktionary, the templates are not rendered (not found by the wikimedia program either, as show the red links): Морфологические и синтаксические свойства Шаблон:сущ ru m ina 1c(1)<http://localhost/mediawiki-1.18.0/index.php?title=%D0%A8%D0%B0%D0%B1%D0%BB%…> Шаблон:морфо<http://localhost/mediawiki-1.18.0/index.php?title=%D0%A8%D0%B0%D0%B1%D0%BB%…> Let's take the example of the template "сущ ru m ina 1c(1)<http://localhost/mediawiki-1.18.0/index.php?title=%D0%A8%D0%B0%D0%B1%D0%BB%…> " I have some questions: - in the SQL generated from mwdumper the following command does not return any result: grep "'сущ ru m ina 1c(1)" ruktionary.sql" ruwiktionary.sql - The string is also not to be found in the generated DB, as a consequence More general: - Is there some special option for mwdumper or a special dump to import to also get the templates?+ - Is there any way to log the requests sent by mediawiki to the DB? Thanks a lot in advance Sebastien On 9 January 2012 22:05, Platonides <platonides(a)gmail.com> wrote: > On 08/01/12 23:03, Sébastien Druon wrote: > >> It seems that mwdumper did not import the templates (select * from page >> where page_title like 'Шаблон:%' does not return anything), though they >> are present in the xml dump. >> Is there some special option to use? >> > > The pages aren't stored with the namespaces name as a literal, but with > the namespace number. > Try SELECT page_title FROM page WHERE page_namespace=10 LIMIT 5; >

2 2

Best place to post questions about mwdumper
by Sébastien Druon 12 Jan '12

12 Jan '12

hello! what is the best place to post questions about mwdumper? regards, sebastien

2 1

← Newer
1
...
55
56
57
58
59
60
61
...
72
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Xmldatadumps-l