On Thu, Apr 3, 2014 at 11:49 PM, Aaron Halfaker <ahalfaker(a)wikimedia.org>wrote;wrote:
Thanks for the update Sean. This is exciting!
Am I understanding correctly that the two machines will be practically
identical, but that db1047 will contain writeable DBs and the new machine
will not? Is the purpose of this to have higher querying capacity than
write capacity -- to match the hardware capabilities with our usage pattern?
Roughly correct interpretation, yes; but there are other differences, in
hardware for example:
db1047: Dell R510, Xeon E5-640, 64g memory.
dbstore1002: Dell R720, Xeon E5-2650, 128g memory.
The MariaDB setup will also differ between to two:
db1047 will be configured to support heavy and ad-hoc write activity in
your scratch spaces, since I can't predict what you'll do there, and have
normal read-only replicated wikis. The box will be solely for Analytics to
hammer and co-exist upon :-)
dbstore1002 will be configured more like a data warehouse (of one! for
now), optimized for compression on disk and in memory, high-read/low-write
activity, few concurrent threads with large buffers, and minimal locking.
It will, at times, be shared with other slow, read-heavy queries, such as
the xml dump processors.
Custom indexes will be possible for either box (I'd suggest keeping them
documented and in sync). Same for custom aggregated tables, views, or
star/snowflake type stuff on the side.
Sean