I am generally in favor of all of this and in the meeting that proceeded
Rob's email, proposed that we develop a new schema migration tool for
mediawiki along similar lines. Such a beast would have to work in all
deployment cases without modifications (stock single wiki installs and at
wmf with many wikis across multiple masters with tiered replication), be
idempotent when run across many databases, track version and state per
migration, and include up/down steps in every migration.
There are opensource php migration tools modeled along those used by the
popular ruby and python frameworks. I deployed
https://github.com/davejkiger/mysql-php-migrations at
kiva.org a couple
years ago where it worked well and is still in use. Nothing will meet our
needs off the shelf though. A good project could at best be forked into
mediawiki with modifications if the license allows it, or more likely serve
as a model for our own development.
On Tue, Apr 24, 2012 at 11:27 PM, Faidon Liambotis <faidon(a)wikimedia.org>wrote;wrote:
In other systems I've worked before, such problems have been solved by
each schema-breaking version providing schema *and data* migrations for
both forward *and backward* steps.
This means that the upgrade transition mechanism knew
how to add or
remove columns or tables *and* how to fill them with data (say by
concatenating two columns of the old schema). The same program would
also take care to do the exact opposite steps in a the migration's
backward method, in case a rollback was needed.
Down migrations aid development; I find them most useful as documentation
of prior state, making a migration readable as a diff. They generally
aren't useful in production environments at scale though, which developers
removed from the workings of production need to be aware of. Even with
transparent execution of migrations, the time it takes to apply changes
will nearly always be far outside of the acceptable bounds of an emergency
response necessitating a code rollback. So except in obvious cases such as
adding new tables, care is needed to keep forward migration backwards
compatible with code as much as possible.
The migrations themselves can be kept in the source tree, perhaps even
versioned and with the schema version kept in the
database, so that both
us and external users can at any time forward their database to any
later version, automagically.
Yep. That we have to pull in migrations from both core and many extensions
(many projects, one migration system) while also running different sets of
extensions across different wikis intermingling on the same database
servers adds some complexity but we should get there.
-Asher