Since the MCR refactor of the Mediawiki database schema has been progressing (https://phabricator.wikimedia.org/T166733 https://phabricator.wikimedia.org/T166732 and many other tickets), one of the last steps is dropping the columns from the wiki replica schema.
The column drops are being tracked and are explained well here https://phabricator.wikimedia.org/T212972 https://phabricator.wikimedia.org/T212972, and currently the change is ready to be applied. It is already applied to two wikis (eswiki and huwiki since there was a column problem that needed fixing). The tables with names such as <tablename>_compat will retain a similar structure if that is needed for refactoring.
From the ticket, this is a summary of what is changing, organized by table name: archive: Remove ar_comment archive_userindex: Remove ar_comment filearchive: Remove fa_deleted_reason Remove fa_description filearchive_userindex: Remove fa_deleted_reason Remove fa_description image: Remove img_description ipblocks: Remove ipb_reason. ipblocks_ipindex: Remove ipb_reason. logging: Remove log_comment. logging_logindex: Remove log_comment. logging_userindex: Remove log_comment. oldimage: Remove oi_description oldimage_userindex: Remove oi_description recentchanges: Remove rc_comment. recentchanges_userindex: Remove rc_comment. revision: Remove rev_comment. revision_userindex: Remove rev_comment.
The changes to the _compat tables should not affect anything.
We will deploy the change early next week (Tuesday - 3/12/2019). In most cases, if a table isn’t working for your tool or app anymore because of the change, you can switch to a table named $tablename_compat and it will appear to have the same schema, but it is recommended that comment references use a join to the new comment table on a “comment_id” field instead where possible.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
On Thu, Mar 7, 2019 at 5:14 PM Brooke Storm bstorm@wikimedia.org wrote:
but it is recommended that comment references use a join to the new comment table on a “comment_id” field instead where possible.
I would like to insist in this part- the change wasn't made to annoy users- we tried to provide compatibility where possible, but some tools users observed reduced performance if something more complex than a point select was being used, so we encourage to use the non-compat ones for performance reasons when possible. Sorry for the inconveniences- unlike an API, while we are happy to provide access to the internal database format, we cannot always guarantee to be stable over the years.
Any chance we can delay the rollout of this? I suspect that this will break a number of tools, mine included. Giving just a week is kinda short notice.
On Thu, Mar 7, 2019 at 11:59 AM Jaime Crespo jcrespo@wikimedia.org wrote:
On Thu, Mar 7, 2019 at 5:14 PM Brooke Storm bstorm@wikimedia.org wrote:
but it is recommended that comment references use a join to the new
comment table on a “comment_id” field instead where possible.
I would like to insist in this part- the change wasn't made to annoy users- we tried to provide compatibility where possible, but some tools users observed reduced performance if something more complex than a point select was being used, so we encourage to use the non-compat ones for performance reasons when possible. Sorry for the inconveniences- unlike an API, while we are happy to provide access to the internal database format, we cannot always guarantee to be stable over the years.
-- Jaime Crespo http://wikimedia.org
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
A delay is certainly possible. I don’t want to hold things up too long on the other end of production, of course. If I roll the change back, temporarily, in puppet, then we could still provide fixes and things without the table changes during the interim. I’d rather not leave it staged and unapplied for too terribly long. As I said, the _compat tables are there to provide a quick fix where possible, and it should be noted that those fields are currently blank (or filled in by the compat tables). This will break things that explicitly reference the fields, though.
I can give it another week, I think, if that helps.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On Mar 7, 2019, at 10:10 AM, John phoenixoverride@gmail.com wrote:
Any chance we can delay the rollout of this? I suspect that this will break a number of tools, mine included. Giving just a week is kinda short notice.
On Thu, Mar 7, 2019 at 11:59 AM Jaime Crespo <jcrespo@wikimedia.org mailto:jcrespo@wikimedia.org> wrote: On Thu, Mar 7, 2019 at 5:14 PM Brooke Storm <bstorm@wikimedia.org mailto:bstorm@wikimedia.org> wrote:
but it is recommended that comment references use a join to the new comment table on a “comment_id” field instead where possible.
I would like to insist in this part- the change wasn't made to annoy users- we tried to provide compatibility where possible, but some tools users observed reduced performance if something more complex than a point select was being used, so we encourage to use the non-compat ones for performance reasons when possible. Sorry for the inconveniences- unlike an API, while we are happy to provide access to the internal database format, we cannot always guarantee to be stable over the years.
-- Jaime Crespo <http://wikimedia.org http://wikimedia.org/>
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org mailto:Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org mailto:labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud https://lists.wikimedia.org/mailman/listinfo/cloud_______________________________________________ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
An addional week or so should be enough. 5 days is just a little rough if you have other commitments.
On Thu, Mar 7, 2019 at 12:33 PM Brooke Storm bstorm@wikimedia.org wrote:
A delay is certainly possible. I don’t want to hold things up too long on the other end of production, of course. If I roll the change back, temporarily, in puppet, then we could still provide fixes and things without the table changes during the interim. I’d rather not leave it staged and unapplied for too terribly long. As I said, the _compat tables are there to provide a quick fix where possible, and it should be noted that those fields are currently blank (or filled in by the compat tables). This will break things that explicitly reference the fields, though.
I can give it another week, I think, if that helps.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org IRC: bstorm_
On Mar 7, 2019, at 10:10 AM, John phoenixoverride@gmail.com wrote:
Any chance we can delay the rollout of this? I suspect that this will break a number of tools, mine included. Giving just a week is kinda short notice.
On Thu, Mar 7, 2019 at 11:59 AM Jaime Crespo jcrespo@wikimedia.org wrote:
On Thu, Mar 7, 2019 at 5:14 PM Brooke Storm bstorm@wikimedia.org wrote:
but it is recommended that comment references use a join to the new
comment table on a “comment_id” field instead where possible.
I would like to insist in this part- the change wasn't made to annoy users- we tried to provide compatibility where possible, but some tools users observed reduced performance if something more complex than a point select was being used, so we encourage to use the non-compat ones for performance reasons when possible. Sorry for the inconveniences- unlike an API, while we are happy to provide access to the internal database format, we cannot always guarantee to be stable over the years.
-- Jaime Crespo http://wikimedia.org
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
After some discussion, we are moving back the date by a week to 3/19/2019 for people to have a bit more time to update tools and such. Thanks everyone!
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On Mar 7, 2019, at 9:13 AM, Brooke Storm bstorm@wikimedia.org wrote:
Since the MCR refactor of the Mediawiki database schema has been progressing (https://phabricator.wikimedia.org/T166733 https://phabricator.wikimedia.org/T166732 and many other tickets), one of the last steps is dropping the columns from the wiki replica schema.
The column drops are being tracked and are explained well here https://phabricator.wikimedia.org/T212972 https://phabricator.wikimedia.org/T212972, and currently the change is ready to be applied. It is already applied to two wikis (eswiki and huwiki since there was a column problem that needed fixing). The tables with names such as <tablename>_compat will retain a similar structure if that is needed for refactoring.
From the ticket, this is a summary of what is changing, organized by table name: archive: Remove ar_comment archive_userindex: Remove ar_comment filearchive: Remove fa_deleted_reason Remove fa_description filearchive_userindex: Remove fa_deleted_reason Remove fa_description image: Remove img_description ipblocks: Remove ipb_reason. ipblocks_ipindex: Remove ipb_reason. logging: Remove log_comment. logging_logindex: Remove log_comment. logging_userindex: Remove log_comment. oldimage: Remove oi_description oldimage_userindex: Remove oi_description recentchanges: Remove rc_comment. recentchanges_userindex: Remove rc_comment. revision: Remove rev_comment. revision_userindex: Remove rev_comment.
The changes to the _compat tables should not affect anything.
We will deploy the change early next week (Tuesday - 3/12/2019). In most cases, if a table isn’t working for your tool or app anymore because of the change, you can switch to a table named $tablename_compat and it will appear to have the same schema, but it is recommended that comment references use a join to the new comment table on a “comment_id” field instead where possible.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Is the comment table fully populated, so we can migrate tools immediately?
Yes, it is fully populated as are the _comment_id fields needed for joins.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On Mar 7, 2019, at 11:21 AM, Jesse Plamondon-Willard pathoschild@gmail.com wrote:
Is the comment table fully populated, so we can migrate tools immediately?
-- Pathoschild (Jesse)
On Thu, 7 Mar 2019 at 12:54, Brooke Storm <bstorm@wikimedia.org mailto:bstorm@wikimedia.org> wrote: After some discussion, we are moving back the date by a week to 3/19/2019 for people to have a bit more time to update tools and such. Thanks everyone!
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On Mar 7, 2019, at 9:13 AM, Brooke Storm <bstorm@wikimedia.org mailto:bstorm@wikimedia.org> wrote:
Since the MCR refactor of the Mediawiki database schema has been progressing (https://phabricator.wikimedia.org/T166733 https://phabricator.wikimedia.org/T166732 and many other tickets), one of the last steps is dropping the columns from the wiki replica schema.
The column drops are being tracked and are explained well here https://phabricator.wikimedia.org/T212972 https://phabricator.wikimedia.org/T212972, and currently the change is ready to be applied. It is already applied to two wikis (eswiki and huwiki since there was a column problem that needed fixing). The tables with names such as <tablename>_compat will retain a similar structure if that is needed for refactoring.
From the ticket, this is a summary of what is changing, organized by table name: archive: Remove ar_comment archive_userindex: Remove ar_comment filearchive: Remove fa_deleted_reason Remove fa_description filearchive_userindex: Remove fa_deleted_reason Remove fa_description image: Remove img_description ipblocks: Remove ipb_reason. ipblocks_ipindex: Remove ipb_reason. logging: Remove log_comment. logging_logindex: Remove log_comment. logging_userindex: Remove log_comment. oldimage: Remove oi_description oldimage_userindex: Remove oi_description recentchanges: Remove rc_comment. recentchanges_userindex: Remove rc_comment. revision: Remove rev_comment. revision_userindex: Remove rev_comment.
The changes to the _compat tables should not affect anything.
We will deploy the change early next week (Tuesday - 3/12/2019). In most cases, if a table isn’t working for your tool or app anymore because of the change, you can switch to a table named $tablename_compat and it will appear to have the same schema, but it is recommended that comment references use a join to the new comment table on a “comment_id” field instead where possible.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org mailto:Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org mailto:labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce https://lists.wikimedia.org/mailman/listinfo/cloud-announce _______________________________________________ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org mailto:Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org mailto:labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud https://lists.wikimedia.org/mailman/listinfo/cloud_______________________________________________ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
Work on deploying this is beginning today. https://phabricator.wikimedia.org/T212972 https://phabricator.wikimedia.org/T212972 will be used to track the work.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On Mar 7, 2019, at 9:13 AM, Brooke Storm bstorm@wikimedia.org wrote:
Since the comment refactor of the Mediawiki database schema has been progressing (https://phabricator.wikimedia.org/T166733 https://phabricator.wikimedia.org/T166732 and many other tickets), one of the last steps is dropping the columns from views on the wiki replica schema.
The column drops are being tracked and are explained well here https://phabricator.wikimedia.org/T212972 https://phabricator.wikimedia.org/T212972, and currently the change is ready to be applied. It is already applied to two wikis (eswiki and huwiki since there was a column problem that needed fixing). The tables with names such as <tablename>_compat will retain a similar structure if that is needed for refactoring.
From the ticket, this is a summary of what is changing, organized by table name: archive: Remove ar_comment archive_userindex: Remove ar_comment filearchive: Remove fa_deleted_reason Remove fa_description filearchive_userindex: Remove fa_deleted_reason Remove fa_description image: Remove img_description ipblocks: Remove ipb_reason. ipblocks_ipindex: Remove ipb_reason. logging: Remove log_comment. logging_logindex: Remove log_comment. logging_userindex: Remove log_comment. oldimage: Remove oi_description oldimage_userindex: Remove oi_description recentchanges: Remove rc_comment. recentchanges_userindex: Remove rc_comment. revision: Remove rev_comment. revision_userindex: Remove rev_comment.
The changes to the _compat tables should not affect anything.
We will deploy the change early next week (Tuesday - 3/12/2019). In most cases, if a table isn’t working for your tool or app anymore because of the change, you can switch to a table named $tablename_compat and it will appear to have the same schema, but it is recommended that comment references use a join to the new comment table on a “comment_id” field instead where possible.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce