Here's the migrations library I wrote. :) https://github.com/ottomata/cs_migrations
-Andrew Otto
On Apr 26, 2012, at 11:30 AM, Andrew Otto wrote:
I once wrote a pretty decent schema migration tool that fits most if not all of these requirements. It was built for the Kohana PHP framework, but a lot of it is pretty independent of that. If someone ends up working on this I'd love to help and maybe share some code and ideas.
-Andrew Otto
http://ottomata.org http://www.flickr.com/photos/OttomatonA http://www.couchsurfing.org/people/otto
On Apr 25, 2012, at 12:58 PM, Asher Feldman wrote:
I am generally in favor of all of this and in the meeting that proceeded Rob's email, proposed that we develop a new schema migration tool for mediawiki along similar lines. Such a beast would have to work in all deployment cases without modifications (stock single wiki installs and at wmf with many wikis across multiple masters with tiered replication), be idempotent when run across many databases, track version and state per migration, and include up/down steps in every migration.
There are opensource php migration tools modeled along those used by the popular ruby and python frameworks. I deployed https://github.com/davejkiger/mysql-php-migrations at kiva.org a couple years ago where it worked well and is still in use. Nothing will meet our needs off the shelf though. A good project could at best be forked into mediawiki with modifications if the license allows it, or more likely serve as a model for our own development.
On Tue, Apr 24, 2012 at 11:27 PM, Faidon Liambotis faidon@wikimedia.orgwrote:
In other systems I've worked before, such problems have been solved by each schema-breaking version providing schema *and data* migrations for both forward *and backward* steps.
This means that the upgrade transition mechanism knew how to add or remove columns or tables *and* how to fill them with data (say by concatenating two columns of the old schema). The same program would also take care to do the exact opposite steps in a the migration's backward method, in case a rollback was needed.
Down migrations aid development; I find them most useful as documentation of prior state, making a migration readable as a diff. They generally aren't useful in production environments at scale though, which developers removed from the workings of production need to be aware of. Even with transparent execution of migrations, the time it takes to apply changes will nearly always be far outside of the acceptable bounds of an emergency response necessitating a code rollback. So except in obvious cases such as adding new tables, care is needed to keep forward migration backwards compatible with code as much as possible.
The migrations themselves can be kept in the source tree, perhaps even
versioned and with the schema version kept in the database, so that both us and external users can at any time forward their database to any later version, automagically.
Yep. That we have to pull in migrations from both core and many extensions (many projects, one migration system) while also running different sets of extensions across different wikis intermingling on the same database servers adds some complexity but we should get there.
-Asher _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l