I said it before, and I say it again: *some* databases should be available for cross-wiki JOIN everywhere. This would at least include commons_p and centralauth_p but perhaps also enwiki_p and meta_p
I know that we discussed it before and better long-term solutions can be imagined (such as a data lake, etc.) but we need a solution *now*.
Sorry for coming at it a bit passionately. It just feels like we are choosing a path of pain for end users (let their tools break, then let us offer alternatives that they could adopt) over a path of effort for the Clouds team (let them create robust solutions for the end users and give them ample time to transition to the new method before turning off cross-wiki joins).
On Wed, Mar 31, 2021 at 4:57 AM Fastily fastilywp@gmail.com wrote:
A little late to the party, I just learned about this change today.
I maintain a number of bot tasks https://en.wikipedia.org/wiki/User:FastilyBot and database https://fastilybot-reports.toolforge.org reports https://en.wikipedia.org/wiki/Wikipedia:Database_reports on enwp that rely on cross-wiki joins (mostly page title joins between enwp and Commons) to function properly. I didn't find the migration instructions https://wikitech.wikimedia.org/w/index.php?title=News/Wiki_Replicas_2020_Redesign&oldid=1905818#How_do_I_cross_reference_data_between_wikis_like_I_do_with_cross_joins_today? very helpful; I run FastilyBot on a Raspberry Pi, and needless to say it would be grossly impractical for me to perform a "join" in the bot's code.
Is there going to be a replacement for this functionality?
Fastily
On Mon, Mar 15, 2021 at 3:09 PM Dan Andreescu dandreescu@wikimedia.org wrote:
[4] was made to figure out common use cases and possibilities to enable
them again.
...
I just want to highlight this ^ thing Joaquin said and mention that our team (Data Engineering) is also participating in brainstorming ways to bring back not just cross-wiki joins but better datasets to run these queries. We have some good ideas, so please do participate in the task and give us more input so we can pick the best solution quickly. _______________________________________________ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud
Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly labs-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/cloud