-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Saqib Kadri wrote:
I'm trying to use mwdumper to insert the English Wikipedia enwiki database into MySQL (enwiki-20070908-pages-articles.xml), but the SSH connection seems to timeout/disconnect after about 890K rows (out of about 10 million I believe) have been uploaded. How can I keep SSH from disconnecting?
Is there some mwdumper command line option I can use, or some client or server side SSH setting? (server is Linux with OpenSSH server)
Usually SSH shouldn't drop your connection if data is being actively sent over it, and mwdumper should be happily feeding data over the pipe as fast as the mysql on the other end will take it.
You might try fiddling with some SSH config settings such as ServerAliveInterval and TCPKeepAlive if there is a connection problem.
The problem might be on the MySQL end, though; double-check that you're not getting disconnected from the MySQL socket due to timeouts or oversized packets. Check the MySQL settings net_read_timeout, net_write_timeout, and max_packet_size.
- -- brion vibber (brion @ wikimedia.org)