Matthew Flaschen wrote:
Here you go:
This is posted here for the foundation to consider rather than on Wikitech as these issues affect the credibility of the Foundation. Posts on these issues to Wikitech get ignored and/or are not responded to at all. I am writing this in response to requests to GFDL enable the wikigadugi.org sites. I am unable to complete the process at present because of severe problems with MediaWiki and the Foundations dumps. I have an extensive set of tools for mirroring wikipedia and they work very well, provided I spend a lot of needless time cleaning up after the MediaWiki developers.
1. The current Mediawiki releases have had breakage with importDump.php since 1.7. I have been unable to make any progress in getting complete runs of the Foundation XML dumps to post without modification to MediaWiki and/or writing programs to clean up NULL titles and other problems with the dumps. Bottom line is that the mediaWiki software DOES NOT WORK with standard XML dumps provided by the foundation AT ALL through its sites, despite the Foundation's clear messages this is in fact the case. This harms the reputation and credibility of the Foundation. It needs to be addressed and fixed.
2. mwdumper does not import the dumps. It processes the files and NOTHING shows up in the database. If these tools do not work, the Meta pages discussing them should say so and the programs should be removed and/or other alternatives suggested or a disclaimer should be placed there that states this. As it stands, the information is misleading and results in a lot of folks wasting a lot of time trying to make tools work which clearly do not. I have found dozens of blog entries and bug reports on these issues and the developers ignore them.
3. The Foundation needs to ask Brion Vibber and the developers to cease posting dumps which are incompatible with MediaWiki releases and which the developers KNOW do not work and force compliance of these processes and mediaWiki compatibility.
In other words, I do not think its too much to ask that the XML dumps posted by the Foundation work with the released MediaWiki versions. It needs to be mandated and complinace required. At present, you almost need a degree in advanced computer science to be able to fix this stuff. I have spent several weekends debugging Brion's PHP code and writing programs to correct these issues just so I can run import on the latest dumps. It would be much nicer if this stuff just worked.
Posts and emails to Wikitech are typically ignored on a lot of these issues. They need to be elevated and corrected.
Jeff
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I have tried to work out such issues with Mr. Merkey before, however he proved unable to follow directions of any kind, so it was a waste of both our and his time to do so.
If someone other than Mr. Merkey is experiencing these problems, I would very much like them to contact me to get them sorted out as soon as possible.
I will not be available during the coming week, however, as I am in the process of moving; I will be available during US east coast office hours from March 5.
Again, if *anyone* *other* than Mr. Merkey is experiencing these problems, please contact me; I would very much like to get them sorted out if they actually exist and affect current release versions of MediaWiki and mwdumper.
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
Brion Vibber wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I have tried to work out such issues with Mr. Merkey before, however he proved unable to follow directions of any kind, so it was a waste of both our and his time to do so.
The traces are posted to wikitech you asked for Brion a very long time ago. This statement is simply not accurate.
If someone other than Mr. Merkey is experiencing these problems, I would very much like them to contact me to get them sorted out as soon as possible.
Exhibit A
http://www.mwusers.com/forums/showthread.php?t=2402
Please fix your code Brion. People are seeing it, you just ignore them.
Jeff
I will not be available during the coming week, however, as I am in the process of moving; I will be available during US east coast office hours from March 5.
Again, if *anyone* *other* than Mr. Merkey is experiencing these problems, please contact me; I would very much like to get them sorted out if they actually exist and affect current release versions of MediaWiki and mwdumper.
- -- brion vibber (brion @ pobox.com / brion @ wikimedia.org)
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFF4NlMwRnhpk1wk44RAjJ6AJ4+l1QepY6hY4iqeUhqk384X6JrUQCghL+D eRuXqQAqVdKttEaVWrmo+gk= =ebLF -----END PGP SIGNATURE-----
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Am Sonntag 25 Februar 2007 03:18 schrieb Jeffrey V. Merkey:
Please fix your code Brion. People are seeing it, you just ignore them.
This seems to be a personal conflict. Please don't expetiate it on that public mailing list, thank you.
-- Leon
Leon Weber wrote:
Am Sonntag 25 Februar 2007 03:18 schrieb Jeffrey V. Merkey:
Please fix your code Brion. People are seeing it, you just ignore them.
This seems to be a personal conflict.
If it is, its one sided. I have no personal issues with Brion. I'm just a MediaWiki user and this is a quality issue. I would agree Brion appears to have some personal issue, but this is simply reports of bugs and issues which have been floating for over a year and have not been closed.
Jeff
Please don't expetiate it on that public mailing list, thank you.
-- Leon
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Dear Jeffrey,
to have some personal issue, but this is simply reports of bugs and issues which have been floating for over a year and have not been closed.
This is open project, quality patches are always welcome.
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this): * Download MediaWiki * Download the dump (for BA wiki and this test it was http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... ) * Install MediaWiki (many tutorials on doing this) * Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux) * Move the extracted xml file to maintenance/dump.xml * In a terminal cd to maintenance * Type in "php importDump.php dump.xml" * Wait for the dump to finish * Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Domas Mituzas wrote:
Dear Jeffrey,
to have some personal issue, but this is simply reports of bugs and issues which have been floating for over a year and have not been closed.
This is open project, quality patches are always welcome.
I am happy to post them.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Now try it with one of the huge enwiki dumps with all the stuff on the talk pages on 1.5 mediawiki that's gone through 1 year on the internet with database upgrades through 1.8.2 and tell me if it still works (will take 2-3 days to run with the enwiki dumps even on a fast system). I have installed applicances in the field and folks are coming back about this issue a lot when they try to use various XML dumps from the Foundation.
When mediawiki gets into low memory conditions of any kind, the wheels fly off when imports, reading, and editing or going on at the same time.
Jeff
Kasimir Gabert wrote:
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this):
- Download MediaWiki
- Download the dump (for BA wiki and this test it was
http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... )
- Install MediaWiki (many tutorials on doing this)
- Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux)
- Move the extracted xml file to maintenance/dump.xml
- In a terminal cd to maintenance
- Type in "php importDump.php dump.xml"
- Wait for the dump to finish
- Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Domas Mituzas wrote:
Dear Jeffrey,
to have some personal issue, but this is simply reports of bugs and issues which have been floating for over a year and have not been closed.
This is open project, quality patches are always welcome.
I am happy to post them.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hello,
Just because of this post I feel compelled to take the time to download and install enwiki dumps...
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Now try it with one of the huge enwiki dumps with all the stuff on the talk pages on 1.5 mediawiki that's gone through 1 year on the internet with database upgrades through 1.8.2 and tell me if it still works (will take 2-3 days to run with the enwiki dumps even on a fast system). I
Okay, we are not talking about whether or not you have destroyed your system because you tried to start it from 1.5 and just upgraded it every time. We are talking about whether or not dumps from Wikimedia work, not whether or not your system is good.
have installed applicances in the field and folks are coming back about this issue a lot when they try to use various XML dumps from the Foundation.
It would be really nice if you could provide some examples. I can test with a larger wiki, if you want... Looking through importDumps.php it does not look like it would stop working after a larger xml file is loaded.
When mediawiki gets into low memory conditions of any kind, the wheels
There is a big difference between whether or not imports work and whether or not your machine can handle them.
fly off when imports, reading, and editing or going on at the same time.
Jeff
I would also like to know what your proposed solution would be? Allow people to dynamically load Wikipedia's content? Do you have any idea how ridiculous that sounds?
Kasimir
Kasimir Gabert wrote:
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this):
- Download MediaWiki
- Download the dump (for BA wiki and this test it was
http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... )
- Install MediaWiki (many tutorials on doing this)
- Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux)
- Move the extracted xml file to maintenance/dump.xml
- In a terminal cd to maintenance
- Type in "php importDump.php dump.xml"
- Wait for the dump to finish
- Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Domas Mituzas wrote:
Dear Jeffrey,
to have some personal issue, but this is simply reports of bugs and issues which have been floating for over a year and have not been closed.
This is open project, quality patches are always welcome.
I am happy to post them.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Kasimir Gabert wrote:
Hello,
Just because of this post I feel compelled to take the time to download and install enwiki dumps...
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Now try it with one of the huge enwiki dumps with all the stuff on the talk pages on 1.5 mediawiki that's gone through 1 year on the internet with database upgrades through 1.8.2 and tell me if it still works (will take 2-3 days to run with the enwiki dumps even on a fast system). I
Okay, we are not talking about whether or not you have destroyed your system because you tried to start it from 1.5 and just upgraded it every time. We are talking about whether or not dumps from Wikimedia work, not whether or not your system is good.
I created Local Area Networking my friend. I think I know if a system works or not.
have installed applicances in the field and folks are coming back about this issue a lot when they try to use various XML dumps from the Foundation.
It would be really nice if you could provide some examples. I can test with a larger wiki, if you want... Looking through importDumps.php it does not look like it would stop working after a larger xml file is loaded.
I have. Go back and read this list for the traces posted.
When mediawiki gets into low memory conditions of any kind, the wheels
There is a big difference between whether or not imports work and whether or not your machine can handle them.
The imports bugs have a large number of pages on google about this and a lot of other places.
fly off when imports, reading, and editing or going on at the same time.
Jeff
I would also like to know what your proposed solution would be? Allow people to dynamically load Wikipedia's content? Do you have any idea how ridiculous that sounds?
Kasimir
Kasimir Gabert wrote:
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this):
- Download MediaWiki
- Download the dump (for BA wiki and this test it was
http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... )
- Install MediaWiki (many tutorials on doing this)
- Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux)
- Move the extracted xml file to maintenance/dump.xml
- In a terminal cd to maintenance
- Type in "php importDump.php dump.xml"
- Wait for the dump to finish
- Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Domas Mituzas wrote:
Dear Jeffrey,
to have some personal issue, but this is simply reports of bugs and issues which have been floating for over a year and have not been closed.
This is open project, quality patches are always welcome.
I am happy to post them.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Kasimir Gabert wrote:
Hello,
Just because of this post I feel compelled to take the time to
[snip]
every time. We are talking about whether or not dumps from Wikimedia work, not whether or not your system is good.
I created Local Area Networking my friend. I think I know if a system works or not.
You invented LAN?? I guess everything I have heard about Robert Metcalfe is completely false because "Jeffrey V. Merkey" posted otherwise.
have installed applicances in the field and folks are coming back about this issue a lot when they try to use various XML dumps from the Foundation.
[snip]
importDumps.php it does not look like it would stop working after a larger xml file is loaded.
I have. Go back and read this list for the traces posted.
"Traces posted"? Do you mean... the debugging output from php5 when it is importing your database? I might be blind, but I can see nothing.
When mediawiki gets into low memory conditions of any kind, the wheels
There is a big difference between whether or not imports work and whether or not your machine can handle them.
The imports bugs have a large number of pages on google about this and a lot of other places.
Nothing critical at http://bugzilla.wikimedia.org/buglist.cgi?query_format=specific&order=re...
You might be providing a true argument, but I am not convinced right now. All of my tests have worked, it definitely could be faster, but I do not understand why you are having issues. It seems to me like it is your fault, not Wikimedia's fault.
fly off when imports, reading, and editing or going on at the same time.
Jeff
I would also like to know what your proposed solution would be? Allow people to dynamically load Wikipedia's content? Do you have any idea how ridiculous that sounds?
Kasimir
Kasimir Gabert wrote:
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this):
- Download MediaWiki
- Download the dump (for BA wiki and this test it was
http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... )
- Install MediaWiki (many tutorials on doing this)
- Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux)
- Move the extracted xml file to maintenance/dump.xml
- In a terminal cd to maintenance
- Type in "php importDump.php dump.xml"
- Wait for the dump to finish
- Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Domas Mituzas wrote:
Dear Jeffrey,
>to have some personal issue, but this is simply reports of bugs and >issues which have been floating for over a year and have not been >closed. > > > > > > This is open project, quality patches are always welcome.
I am happy to post them.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hello,
I imported the dump from enwikibooks to test.kgprog.com and it worked perfectly. This is not one of the smaller wikis and it has just confirmed my belief that Wikimedia dumps work. If you can demonstrate with a specific example what your error is, and provide the appropriate parts of the xml file, then I am sure that the developers will look at your case seriously.
Kasimir
On 2/24/07, Kasimir Gabert kasimir.g@gmail.com wrote:
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Kasimir Gabert wrote:
Hello,
Just because of this post I feel compelled to take the time to
[snip]
every time. We are talking about whether or not dumps from Wikimedia work, not whether or not your system is good.
I created Local Area Networking my friend. I think I know if a system works or not.
You invented LAN?? I guess everything I have heard about Robert Metcalfe is completely false because "Jeffrey V. Merkey" posted otherwise.
have installed applicances in the field and folks are coming back about this issue a lot when they try to use various XML dumps from the Foundation.
[snip]
importDumps.php it does not look like it would stop working after a larger xml file is loaded.
I have. Go back and read this list for the traces posted.
"Traces posted"? Do you mean... the debugging output from php5 when it is importing your database? I might be blind, but I can see nothing.
When mediawiki gets into low memory conditions of any kind, the wheels
There is a big difference between whether or not imports work and whether or not your machine can handle them.
The imports bugs have a large number of pages on google about this and a lot of other places.
Nothing critical at http://bugzilla.wikimedia.org/buglist.cgi?query_format=specific&order=re...
You might be providing a true argument, but I am not convinced right now. All of my tests have worked, it definitely could be faster, but I do not understand why you are having issues. It seems to me like it is your fault, not Wikimedia's fault.
fly off when imports, reading, and editing or going on at the same time.
Jeff
I would also like to know what your proposed solution would be? Allow people to dynamically load Wikipedia's content? Do you have any idea how ridiculous that sounds?
Kasimir
Kasimir Gabert wrote:
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this):
- Download MediaWiki
- Download the dump (for BA wiki and this test it was
http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... )
- Install MediaWiki (many tutorials on doing this)
- Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux)
- Move the extracted xml file to maintenance/dump.xml
- In a terminal cd to maintenance
- Type in "php importDump.php dump.xml"
- Wait for the dump to finish
- Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Domas Mituzas wrote:
>Dear Jeffrey, > > > > > > > >>to have some personal issue, but this is simply reports of bugs and >>issues which have been floating for over a year and have not been >>closed. >> >> >> >> >> >> >This is open project, quality patches are always welcome. > > > > > > > I am happy to post them.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Kasimir Gabert
Kasimir Gabert wrote:
Hello,
I imported the dump from enwikibooks to test.kgprog.com and it worked perfectly. This is not one of the smaller wikis and it has just confirmed my belief that Wikimedia dumps work. If you can demonstrate with a specific example what your error is, and provide the appropriate parts of the xml file, then I am sure that the developers will look at your case seriously.
Kasimir
enwiki, not wikibooks. I can get wikibooks to work too. It is related to large dumps. It behaves like stack corruption. I have posted the logs already. You have to consider I also have all of the images as well in an integrated setup, like wikipedia - squid clusters included, so a workstation test is not probably the same scenario. Try the November and February XML dumps on 1.8.2, not 1.9.3. I have gone through the upgrade path too many times already each time hoping importDump would work better. It has improved, but still is buggy. I doubt its in the platform, or the site would not work at all, and it works.
Jeff
On 2/24/07, Kasimir Gabert kasimir.g@gmail.com wrote:
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Kasimir Gabert wrote:
Hello,
Just because of this post I feel compelled to take the time to
[snip]
every time. We are talking about whether or not dumps from Wikimedia work, not whether or not your system is good.
I created Local Area Networking my friend. I think I know if a system works or not.
You invented LAN?? I guess everything I have heard about Robert Metcalfe is completely false because "Jeffrey V. Merkey" posted otherwise.
have installed applicances in the field and folks are coming back about this issue a lot when they try to use various XML dumps from the Foundation.
[snip]
importDumps.php it does not look like it would stop working after a larger xml file is loaded.
I have. Go back and read this list for the traces posted.
"Traces posted"? Do you mean... the debugging output from php5 when it is importing your database? I might be blind, but I can see nothing.
When mediawiki gets into low memory conditions of any kind, the wheels
There is a big difference between whether or not imports work and whether or not your machine can handle them.
The imports bugs have a large number of pages on google about this and a lot of other places.
Nothing critical at http://bugzilla.wikimedia.org/buglist.cgi?query_format=specific&order=re...
You might be providing a true argument, but I am not convinced right now. All of my tests have worked, it definitely could be faster, but I do not understand why you are having issues. It seems to me like it is your fault, not Wikimedia's fault.
fly off when imports, reading, and editing or going on at the same time.
Jeff
I would also like to know what your proposed solution would be? Allow people to dynamically load Wikipedia's content? Do you have any idea how ridiculous that sounds?
Kasimir
Kasimir Gabert wrote:
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this):
- Download MediaWiki
- Download the dump (for BA wiki and this test it was
http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... )
- Install MediaWiki (many tutorials on doing this)
- Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux)
- Move the extracted xml file to maintenance/dump.xml
- In a terminal cd to maintenance
- Type in "php importDump.php dump.xml"
- Wait for the dump to finish
- Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
>Domas Mituzas wrote: > > > > > > > >>Dear Jeffrey, >> >> >> >> >> >> >> >> >> >>>to have some personal issue, but this is simply reports of bugs and >>>issues which have been floating for over a year and have not been >>>closed. >>> >>> >>> >>> >>> >>> >>> >>> >>This is open project, quality patches are always welcome. >> >> >> >> >> >> >> >> >> >I am happy to post them. > >Jeff > >_______________________________________________ >Wikitech-l mailing list >Wikitech-l@lists.wikimedia.org >http://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > > > >
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Kasimir Gabert
And here is another good one to look into on this problem. The enwiki-20070206 imports FAIL with reports of "NULL title detected" when running importDump.php from 1.8.2. The problem is, there are NO NULL titles contained in this XML file.
I have a tool that strips out the titles from the dumps for translation and link grammar parsing. These tools do not affect or alter the enwiki xml file at all but simply READ FROM IT, I just use them to strip out the article titles.
Here is the output for the enwiki dump when article titles have been striped and redirected to a file. There is not a single NULL title or revision in the entire XMl Dump. This SAME UNMODIFIED DUMP FROM THE FOUNDATION crashes in importDump complaining of NULL titles when THERE ARE NO NULL TITLES IN THE DUMP AT ALL.
Stripped Titles enwiki-20070206:
ftp://www.wikigadugi.org/wiki/xml/titles-enwiki-20070206.txt.bz2
importDump.php Logs with crash and BOGUS NULL title error importing enwiki-20070206:
ftp://www.wikigadugi.org/wiki/xml/log-enwiki-20070206.txt.bz2
Jeff
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160
And here is another good one to look into on this problem. The enwiki-20070206 imports FAIL with reports of "NULL title detected" when running importDump.php from 1.8.2.
Make sure you have a "+" character in your $wgLegalTitleChars
- -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200702250902 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
Greg Sabino Mullane wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160
And here is another good one to look into on this problem. The enwiki-20070206 imports FAIL with reports of "NULL title detected" when running importDump.php from 1.8.2.
Make sure you have a "+" character in your $wgLegalTitleChars
I have a "+" character in wgLegalTitleChars for hthe URL/space expansion thing when I see the problem. I am re-running the importdump with tracing enabled and a debug log. The log is HUGE so far. When it crashes again, I'll post the log.
Jeff
Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200702250902 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE-----
iD8DBQFF4ZcevJuQZxSWSsgRA+12AJ9Lk8J4vc4tDkZNQX8L44CxJWeJwQCgwqNG po+HVYSq7XAwQ9m1z4PJLZk= =PzrW -----END PGP SIGNATURE-----
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
I download the dumps to /wikidump/dump and type from the root wiki dir:
# translate to Cherokee
# -gdfl insert [:en:links] back to wikipedia into each article during translation -nt strip any NULL titles or revisions - syl output in syllabary wikitrans -syl -gfdl -nt /chr/chrlexicon.cherokee_nation /chr/chrlexicon.eastern_band /chr/chrlexicon.oklahoma-missouri -thes chr2en.thesaurus -thes \ en2chr.thesaurus -grammar Otali.bin < enwiki-<date>.xml > sylwiki-<date>.xml wikitrans -gfdl -nt /chr/chrlexicon.cherokee_nation /chr/chrlexicon.eastern_band /chr/chrlexicon.oklahoma-missouri -thes chr2en.thesaurus -thes \ en2chr.thesaurus -grammar Otali.bin < enwiki-<date>.xml > phwiki-<date>.xml
// strip from XML file and download all images from wikipedia commons and wikipedia with 16 daemons (-p parallel mode) wikix -p < /wikidump/dump/enwiki-<date>.xml (this program actually creates parallel scripts which resync images from the Foundation ./image_sh #
php maintenance/importDump.php < /wikidump/enwiki-<date>.xml >& /wikidump/en.log & php maintenance/importDump.php < /wikidump/sylwiki-<date>.xml >& /wikidump/chr.log & php maintenance/importDump.php < /wikidump/phwiki-<date>.xml >& /wikidump/chrp.log &
I think I got it. :-)
Jeff
Kasimir Gabert wrote:
Hello,
I can verify that uploading MediaWiki dumps works.
I was able to successfully import (using *nothing but the built in MW tools*) ba.wikipedia.org to test.kgprog.com in a few minutes with a brand new installation of MW 1.9.3.
The list of things that I had to do (in case Merkey really does want to learn how to do this):
- Download MediaWiki
- Download the dump (for BA wiki and this test it was
http://download.wikipedia.org/bawiki/20070222/bawiki-20070222-pages-articles... )
- Install MediaWiki (many tutorials on doing this)
- Extract the XML file from the bz2 compressed file (I used bunzip2 on Linux)
- Move the extracted xml file to maintenance/dump.xml
- In a terminal cd to maintenance
- Type in "php importDump.php dump.xml"
- Wait for the dump to finish
- Type in "php rebuildrecentchanges.php"
The wiki has now been successfully created with all of the dumped pages from Wikimedia Foundation.
Kasimir
On 2/24/07, Jeffrey V. Merkey jmerkey@wolfmountaingroup.com wrote:
Domas Mituzas wrote:
Dear Jeffrey,
to have some personal issue, but this is simply reports of bugs and issues which have been floating for over a year and have not been closed.
This is open project, quality patches are always welcome.
I am happy to post them.
Jeff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org http://lists.wikimedia.org/mailman/listinfo/wikitech-l
Jeff V. Merkey wrote:
Matthew Flaschen wrote:
Here you go:
This is posted here for the foundation to consider rather than on Wikitech as these issues affect the credibility of the Foundation. Posts on these issues to Wikitech get ignored and/or are not responded to at all. I am writing this in response to requests to GFDL enable the wikigadugi.org sites.
I hope you understand dumps are not a requirement for the GFDL. Special:Export and indeed the XHTML pages themselves count as transparent copies.
This harms the reputation and credibility of the Foundation. It needs to be addressed and fixed.
No, it should be fixed, but it doesn't need to be. Wikimedia has no obligation to provide dumps, working or otherwise.
It would be much nicer if this stuff just worked.
That's true.
Matthew Flaschen
wikitech-l@lists.wikimedia.org