Hi,
I am hoping that someone here can help me - I realize there is an xmlwikidumps mailing list but it is pretty low volume and expertise relative to this one. THere is a lot of conflicting advice on the mirroring wikipedia page. C I am setting up a local mirror of english Wikipedia pages-articles...xml and I have been stopped by repeated failures in mySQL configuration.
I have downloaded the .gz data from the dumped and run it through mwdumper to create an SQL file without problems, but things keep breaking down on the way into mysql. I have had a lot of agony with inno_db log files, etc. and have learned how to make them bigger, but I'm still apaprently missing some pieces.
my target machine is an AWS instance 32 bit Ubuntu 1.7GB RAM, 1 Core, 200 GB, which is only for this project. I can make it bigger if necessary.
Can someone take a look at this my.cnf file and tell me what I need to change to get this to work?
this is what my.cnf file looks like:
[mysqladmin] user=
[mysqld] basedir=/opt/bitnami/mysql datadir=/opt/bitnami/mysql/data port=3306 socket=/opt/bitnami/mysql/tmp/mysql.sock tmpdir=/opt/bitnami/mysql/tmp
character-set-server=UTF8 collation-server=utf8_general_ci
max_allowed_packet=128M wait_timeout = 120 long_query_time = 1 log_slow_queries log_queries_not_using_indexes query_cache_limit=2M query_cache_type=1 query_cache_size=128M innodb_additional_mem_pool_size=8M innodb_buffer_pool_size=256M innodb_log_file_size=128M #tmp_table_size=64M #max_connections = 2500 #max_user_connections = 2500 innodb_flush_method=O_DIRECT #key_buffer_size=64M
[mysqld_safe] mysqld=mysqld.bin
[client] default-character-set=UTF8 port=3306 socket=/opt/bitnami/mysql/tmp/mysql.sock
[manager] port=3306 socket=/opt/bitnami/mysql/tmp/mysql.sock pid-file=/opt/bitnami/mysql/tmp/manager.pid default-mysqld-path=/opt/bitnami/mysql/bin/mysqld.bin
On Thu, Oct 6, 2011 at 12:54 PM, Fred Zimmerman zimzaz.wfz@gmail.comwrote:
Hi,
I am hoping that someone here can help me - I realize there is an xmlwikidumps mailing list but it is pretty low volume and expertise relative to this one. THere is a lot of conflicting advice on the mirroring wikipedia page. C I am setting up a local mirror of english Wikipedia pages-articles...xml and I have been stopped by repeated failures in mySQL configuration.
I have downloaded the .gz data from the dumped and run it through mwdumper to create an SQL file without problems, but things keep breaking down on the way into mysql. I have had a lot of agony with inno_db log files, etc. and have learned how to make them bigger, but I'm still apaprently missing some pieces.
What fails, exactly? Do you get error messages of some kind? Without knowing what's going wrong, there's little advice that can be given.
my target machine is an AWS instance 32 bit Ubuntu 1.7GB RAM, 1 Core, 200 GB, which is only for this project. I can make it bigger if necessary.
Can someone take a look at this my.cnf file and tell me what I need to change to get this to work?
this is what my.cnf file looks like:
The only requirement I can think of is making sure max_packet_size is biggish so all the pages import; you appear to have set it to 128M which should be more than big enough.
Other settings I would assume should depend on your available memory, workload, etc.
-- brion
I don't have all the log files with me at the moment. I am gathering myself for a fresh start. How about a .cnf file that demonstrably works?
What fails, exactly? Do you get error messages of some kind? Without knowing what's going wrong, there's little advice that can be given.
my target machine is an AWS instance 32 bit Ubuntu 1.7GB RAM, 1 Core, 200 GB, which is only for this project. I can make it bigger if necessary.
Can someone take a look at this my.cnf file and tell me what I need to change to get this to work?
this is what my.cnf file looks like:
The only requirement I can think of is making sure max_packet_size is biggish so all the pages import; you appear to have set it to 128M which should be more than big enough.
Other settings I would assume should depend on your available memory, workload, etc.
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
You can see the one used live on our sites:
https://gerrit.wikimedia.org/r/gitweb?p=operations/puppet.git;a=blob;f=templ...
https://gerrit.wikimedia.org/r/gitweb?p=operations/puppet.git;a=blob;f=manif...
Unfortunately, it's a puppet template, but you can substitute the variables with their values in the related puppet manifest.
- Ryan
On Thu, Oct 6, 2011 at 6:55 PM, Fred Zimmerman zimzaz.wfz@gmail.com wrote:
I don't have all the log files with me at the moment. I am gathering myself for a fresh start. How about a .cnf file that demonstrably works?
What fails, exactly? Do you get error messages of some kind? Without knowing what's going wrong, there's little advice that can be given.
my target machine is an AWS instance 32 bit Ubuntu 1.7GB RAM, 1 Core, 200 GB, which is only for this project. I can make it bigger if necessary.
Can someone take a look at this my.cnf file and tell me what I need to change to get this to work?
this is what my.cnf file looks like:
The only requirement I can think of is making sure max_packet_size is biggish so all the pages import; you appear to have set it to 128M which should be more than big enough.
Other settings I would assume should depend on your available memory, workload, etc.
-- brion _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org