Similar to the earlier removal of text fields from the wiki replicas for comment storage refactors in Mediawiki, we are going to remove “user text” columns from the views that are deprecated in the Mediawiki schema to prepare for when they will actually be removed upstream. The column drops are tracked and explained here https://phabricator.wikimedia.org/T223406 https://phabricator.wikimedia.org/T223406. The tables with names such as <tablename>_compat will not see a difference in structure. The change is scheduled for Monday, May 27th.
The fields that are dropping from the views are: revision: rev_user and rev_user_text. archive: ar_user and ar_user_text. ipblocks: ipb_by and ipb_by_text. image: img_user and img_user_text. oldimage: oi_user and oi_user_text. filearchive: fa_user and fa_user_text. recentchanges: rc_user and rc_user_text. logging: log_user and log_user_text. Ideally, tools that connect to the replicas should gather the information from the appropriate entries in the actor table instead, again, this is similar to the change for the comment table. The data is already there for you to start using. The alternative is to try using the related <tablename>_compat table, which won’t be changing in a user-visible way at this time.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Hi, I tried to update one of my tools (intersect-contribs) to use the `actor` table instead of the `revision.rev_user_text` field.
Joining the `revision` and `actor` tables on `revision.rev_actor = actor.actor_id` is painfully slow and makes my tool unusable. I guess that the reason is that there is no index on the `revision.rev_actor` column (which is nullable, by the way).
Would it be possible to have a view like `revision_userindex` (which has index on `rev_user_text`) with an index on the `rev_actor` column instead? In this way, this very common join would be way faster.
Best,
--- Pietro De Nicolao
Il giorno 18 mag 2019, alle ore 00:27, Brooke Storm bstorm@wikimedia.org ha scritto:
Similar to the earlier removal of text fields from the wiki replicas for comment storage refactors in Mediawiki, we are going to remove “user text” columns from the views that are deprecated in the Mediawiki schema to prepare for when they will actually be removed upstream. The column drops are tracked and explained here https://phabricator.wikimedia.org/T223406 https://phabricator.wikimedia.org/T223406. The tables with names such as <tablename>_compat will not see a difference in structure. The change is scheduled for Monday, May 27th.
The fields that are dropping from the views are: revision: rev_user and rev_user_text. archive: ar_user and ar_user_text. ipblocks: ipb_by and ipb_by_text. image: img_user and img_user_text. oldimage: oi_user and oi_user_text. filearchive: fa_user and fa_user_text. recentchanges: rc_user and rc_user_text. logging: log_user and log_user_text. Ideally, tools that connect to the replicas should gather the information from the appropriate entries in the actor table instead, again, this is similar to the change for the comment table. The data is already there for you to start using. The alternative is to try using the related <tablename>_compat table, which won’t be changing in a user-visible way at this time.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
On Sun, May 19, 2019 at 3:07 PM Pietro De Nicolao powerpdn@gmail.com wrote:
Would it be possible to have a view like `revision_userindex` (which has index on `rev_user_text`) with an index on the `rev_actor` column instead? In this way, this very common join would be way faster.
Pietro, that is a useful suggestion, please file a ticket on phabricator or comment on Brooke's ticket: that is something that looks very reasonable to me (although it may had to be done in a different way for now, as far as I can see there is not yet a *real* actor column on the underlying revision table, neither on production nor in labs, and the join on the view may be making it slow).
I’d like to announce that after taking considerable user feedback about it, and fixing a performance issue regarding the revision_userindex table at https://phabricator.wikimedia.org/T221339 https://phabricator.wikimedia.org/T221339 that affects migration of tools to the new schema, that we are extending the date to Monday, June 3rd instead to begin dropping the fields in the views. This should give additional time to fix things up as well as find issues in existing views if any more arise.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On May 17, 2019, at 4:27 PM, Brooke Storm bstorm@wikimedia.org wrote:
Similar to the earlier removal of text fields from the wiki replicas for comment storage refactors in Mediawiki, we are going to remove “user text” columns from the views that are deprecated in the Mediawiki schema to prepare for when they will actually be removed upstream. The column drops are tracked and explained here https://phabricator.wikimedia.org/T223406 https://phabricator.wikimedia.org/T223406. The tables with names such as <tablename>_compat will not see a difference in structure. The change is scheduled for Monday, May 27th.
The fields that are dropping from the views are: revision: rev_user and rev_user_text. archive: ar_user and ar_user_text. ipblocks: ipb_by and ipb_by_text. image: img_user and img_user_text. oldimage: oi_user and oi_user_text. filearchive: fa_user and fa_user_text. recentchanges: rc_user and rc_user_text. logging: log_user and log_user_text. Ideally, tools that connect to the replicas should gather the information from the appropriate entries in the actor table instead, again, this is similar to the change for the comment table. The data is already there for you to start using. The alternative is to try using the related <tablename>_compat table, which won’t be changing in a user-visible way at this time.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
Work on this is beginning.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On May 20, 2019, at 12:38 PM, Brooke Storm bstorm@wikimedia.org wrote:
I’d like to announce that after taking considerable user feedback about it, and fixing a performance issue regarding the revision_userindex table at https://phabricator.wikimedia.org/T221339 https://phabricator.wikimedia.org/T221339 that affects migration of tools to the new schema, that we are extending the date to Monday, June 3rd instead to begin dropping the fields in the views. This should give additional time to fix things up as well as find issues in existing views if any more arise.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On May 17, 2019, at 4:27 PM, Brooke Storm <bstorm@wikimedia.org mailto:bstorm@wikimedia.org> wrote:
Similar to the earlier removal of text fields from the wiki replicas for comment storage refactors in Mediawiki, we are going to remove “user text” columns from the views that are deprecated in the Mediawiki schema to prepare for when they will actually be removed upstream. The column drops are tracked and explained here https://phabricator.wikimedia.org/T223406 https://phabricator.wikimedia.org/T223406. The tables with names such as <tablename>_compat will not see a difference in structure. The change is scheduled for Monday, May 27th.
The fields that are dropping from the views are: revision: rev_user and rev_user_text. archive: ar_user and ar_user_text. ipblocks: ipb_by and ipb_by_text. image: img_user and img_user_text. oldimage: oi_user and oi_user_text. filearchive: fa_user and fa_user_text. recentchanges: rc_user and rc_user_text. logging: log_user and log_user_text. Ideally, tools that connect to the replicas should gather the information from the appropriate entries in the actor table instead, again, this is similar to the change for the comment table. The data is already there for you to start using. The alternative is to try using the related <tablename>_compat table, which won’t be changing in a user-visible way at this time.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce
The work on this is completed across the replicas. We’ve created a wikitech page to describe the change further and help share solutions to problems caused by the changes https://wikitech.wikimedia.org/wiki/News/Actor_storage_changes_on_the_Wiki_R... https://wikitech.wikimedia.org/wiki/News/Actor_storage_changes_on_the_Wiki_Replicas
I’m hoping to collect more useful information and tips there as we go.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On Jun 3, 2019, at 8:27 AM, Brooke Storm bstorm@wikimedia.org wrote:
Work on this is beginning.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On May 20, 2019, at 12:38 PM, Brooke Storm <bstorm@wikimedia.org mailto:bstorm@wikimedia.org> wrote:
I’d like to announce that after taking considerable user feedback about it, and fixing a performance issue regarding the revision_userindex table at https://phabricator.wikimedia.org/T221339 https://phabricator.wikimedia.org/T221339 that affects migration of tools to the new schema, that we are extending the date to Monday, June 3rd instead to begin dropping the fields in the views. This should give additional time to fix things up as well as find issues in existing views if any more arise.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
On May 17, 2019, at 4:27 PM, Brooke Storm <bstorm@wikimedia.org mailto:bstorm@wikimedia.org> wrote:
Similar to the earlier removal of text fields from the wiki replicas for comment storage refactors in Mediawiki, we are going to remove “user text” columns from the views that are deprecated in the Mediawiki schema to prepare for when they will actually be removed upstream. The column drops are tracked and explained here https://phabricator.wikimedia.org/T223406 https://phabricator.wikimedia.org/T223406. The tables with names such as <tablename>_compat will not see a difference in structure. The change is scheduled for Monday, May 27th.
The fields that are dropping from the views are: revision: rev_user and rev_user_text. archive: ar_user and ar_user_text. ipblocks: ipb_by and ipb_by_text. image: img_user and img_user_text. oldimage: oi_user and oi_user_text. filearchive: fa_user and fa_user_text. recentchanges: rc_user and rc_user_text. logging: log_user and log_user_text. Ideally, tools that connect to the replicas should gather the information from the appropriate entries in the actor table instead, again, this is similar to the change for the comment table. The data is already there for you to start using. The alternative is to try using the related <tablename>_compat table, which won’t be changing in a user-visible way at this time.
Brooke Storm Operations Engineer Wikimedia Cloud Services bstorm@wikimedia.org mailto:bstorm@wikimedia.org IRC: bstorm_
_______________________________________________ Wikimedia Cloud Services announce mailing list Cloud-announce@lists.wikimedia.org (formerly labs-announce@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud-announce