(If you don’t work with pagelinks table, feel free to ignore this message)
Hello,
Here is an update and reminder on the previous announcement
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
regarding normalization of links tables that was sent around a year ago.
As part of that work, soon the pl_namespace and pl_title columns of
pagelinks table will be dropped and you will need to use pl_target_id
joining with the linktarget table instead. This is basically identical to
the templatelinks normalization that happened a year ago.
Currently, MediaWiki writes to both data schemes of pagelinks for new rows
in all wikis except English Wikipedia and Wikimedia Commons (we will start
writing to these two wikis next week). We have started to backfill the data
with the new schema but it will take weeks to finish in large wikis.
So if you query this table directly or your tools do, You will need to
update them accordingly. I will write a reminder before dropping the old
columns once the data has been fully backfilled.
You can keep track of the general long-term work in T300222
<https://phabricator.wikimedia.org/T300222> and the specific work for
pagelinks in T299947 <https://phabricator.wikimedia.org/T299947>. You can
also read more on the reasoning in T222224
<https://phabricator.wikimedia.org/T222224> or the previous announcement
<https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/…>
.
Thank you,
--
*Amir Sarabadani (he/him)*
Staff Database Architect
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello,
We are now only three weeks away from the Wikimedia Wishathon! Exciting
news - User:Lucas Werkmeister has signed up to host a piano concert during
a social hour 🎉
Join us and contribute to the development of community wishes between March
15th and 17th! Participate in discussion sessions and work on user scripts,
gadgets, extensions, tools and more!
The full event schedule is available here: <
https://meta.wikimedia.org/wiki/Event:WishathonMarch2024>.
Explore the event wiki for project ideas and keep an eye out for
non-technical tasks (documentation and design-related) that will soon be
added to the Wishathon workboard: <
https://phabricator.wikimedia.org/project/view/5906/>. Project breakouts
will also be added to the schedule, where you can participate in wish
development or explore innovative solutions as a user, developer, or
designer.
We are seeking volunteers to assist with a wide range of activities such as
monitoring discussion channels during hacking hours, answering technical
queries, and helping with session note-taking. Check out the Help desk
schedule and add yourself to a slot where you are available and interested
in providing assistance: <
https://meta.wikimedia.org/wiki/Event:WishathonMarch2024/Help_desk>.
If you have any questions about the Wishathon, reach out via Telegram: <
https://t.me/wmhack>.
Cheers,
Srishti
On behalf of the Wishathon organizing committee
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation <https://wikimediafoundation.org/>
Hi all!
Good news, we have enabled health checks for all the webservices running on
toolforge.
There's no action required on your part, the next time you restart or
stop/start your webservice, it will have a tcp health check by default (just
making sure something is listening).
The most interesting feature though is being able to pass a url to use as HTTP
health check.
To do so you can pass `--health-check-url /path/to/health` to your `toolforge
webservice start` command, and toolforge will automatically restart your
webservice if it stops responding to that path (you can change the path to
whatever you want, ex. `/`).
Note that this url will be queried quite often, so try to avoid hitting a page
that uses many resources.
Also a reminder that you can find this and smaller user-facing updates about
the Toolforge platform features here:
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Changelog
Original task: https://phabricator.wikimedia.org/T341919
Cheers!
--
David Caro
SRE - Cloud Services
Wikimedia Foundation <https://wikimediafoundation.org/>
PGP Signature: 7180 83A2 AC8B 314F B4CE 1171 4071 C7E1 D262 69C3
"Imagine a world in which every single human being can freely share in the
sum of all knowledge. That's our commitment."
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hello all,
The next language community meeting is coming up in a few weeks - *February
21st, 12:00 pm UTC*.
If you're interested, you can sign up on this wiki page: <
https://www.mediawiki.org/w/index.php?title=Wikimedia_Language_engineering/…
>.
This meeting is for individuals involved in creating content or managing
technical aspects across different language communities. This will be a
participant-driven meeting, where we collectively discuss specific
technical issues related to language wikis and work together to find
possible solutions. This could involve anything from fixing a broken
template on the Kurdish wiki to brainstorming ideas for growing content on
the Tulu Wiktionary, currently in the Wikimedia Incubator, or celebrating
the creation of Fon Wikipedia, to using MinT for content translation.
Feel free to add any *technical updates* related to your project or ideas
for *problem-solving* discussion that you would like to share during the
meeting to the notes document here: <
https://etherpad.wikimedia.org/p/language-community-meeting-february-2024>.
If you need interpretation support from English to another language, please
let us know by sending an email to ssethi(a)wikimedia.org.
Looking forward to your participation!
Cheers,
Jon, Mary & Srishti
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello,
Starting on Wednesday(14th February), selected tools will stop running on
Grid Engine.
The tools will be stopped from running but the code and data will not be
deleted.
If you want your tool to be re-enabled, please reach out to the cloud
admins on the mailing list or on the tool's migration ticket.
Those who have reached out to ask for extension are not affected by this.
Here's a reminder of the timeline we are following[0]
[0]:
https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#…
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hello all!
We have been hard at work on our Graph Split experiment [1], and we now
have a working graph split that is loaded onto 3 test servers. We are
running tests on a selection of queries from our logs to help understand
the impact of the split. We need your help to validate the impact of
various use cases and workflows around Wikidata Query Service.
**What is the WDQS Graph Split experiment?**
We want to address the growing size of the Wikidata graph by splitting it
into 2 subgraphs of roughly half the size of the full graph, which should
support the growth of Wikidata for the next 5 years. This experiment is
about splitting the full Wikidata graph into a scholarly articles subgraph
and a “main” graph that contains everything else.
See our previous update for more details [2].
**Who should care?**
Anyone who uses WDQS through the UI or programmatically should check the
impact on their use cases, scripts, bots, code, etc.
**What are those test endpoints?**
We expose 3 test endpoints, for the full, main and scholarly articles
graphs. Those graphs are all created from the same dump and are not live
updated. This allows us to compare queries between the different endpoints,
with stable / non changing data (the data are from the middle of October
2023).
The endpoints are:
* https://query-full-experimental.wikidata.org/
* https://query-main-experimental.wikidata.org/
* https://query-scholarly-experimental.wikidata.org/
Each of the endpoints is backed by a single dedicated server of performance
similar to the production WDQS servers. We don’t expect performance to be
representative of production due to the different load and to the lack of
updates on the test servers.
**What kind of feedback is useful?**
We expect queries that don’t require scholarly articles to work
transparently on the “main” subgraph. We expect queries that require
scholarly articles to need to be rewritten with SPARQL federation between
the “main” and scholarly subgraphs (federation is supported for some
external SPARQL servers already [3], this just happens to be for internal
server-to-server communication). We are doing tests and analysis based on a
sample of query logs.
**We want to hear about:**
General use cases or classes of queries which break under federation
Bots or applications that need significant rewrite of queries to work with
federation
And also about use cases that work just fine!
Examples of queries and pointers to code will be helpful in your feedback.
**Where should feedback be sent?**
You can reach out to us using the project’s talk page [1], the Phabricator
ticket for community feedback [4] or by pinging directly Sannita (WMF) [5].
**Will feedback be taken into account?**
Yes! We will review feedback and it will influence our path forward. That
being said, there are limits to what is possible. The size of the Wikidata
graph is a threat to the stability of WDQS and thus a threat to the whole
Wikidata project. Scholarly articles is the only split we know of that
would reduce the graph size sufficiently. We can work together on providing
support for a migration, on reviewing the rules used for the graph split,
but we can’t just ignore the problem and continue with a WDQS that provides
transparent access to the full Wikidata graph.
Have fun!
Guillaume
[1]
https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_graph_split
[2]
https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_up…
[3]
https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual#Federation
[4] https://phabricator.wikimedia.org/T356773
[5] https://www.wikidata.org/wiki/User:Sannita_(WMF)
--
Guillaume Lederrey (he/him)
Engineering Manager
Wikimedia Foundation
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hello all,
As we continue with work towards the grid engine deprecation[0], we are
following through with the timeline[1] shared.
We have reached out to each maintainer via email and talk pages with
reminders.
On the 14th of this month(February), we will begin to stop tools that are
still running on the grid.
Tools that have had their maintainers reach out and are actively migrating,
can ask for extensions and will not be stopped.
Once a tool is stopped, if the maintainer has a clear plan for migrating,
they can request in the tool-specific migration task for the tool to be
re-enabled (although they will be shut down again if they miss the March
deadline).
If you have a tool that is still on the grid and you cannot meet the above
deadline, kindly reach out via the tool migration phabricator ticket or our
support channels[2]
[0]:
https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation
[1]:
https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation#…
[2]:
https://wikitech.wikimedia.org/wiki/Portal:Toolforge/About_Toolforge#Commun…
--
Seyram Komla Sapaty
Developer Advocate
Wikimedia Cloud Services
_______________________________________________
Cloud-announce mailing list -- cloud-announce(a)lists.wikimedia.org
List information: https://lists.wikimedia.org/postorius/lists/cloud-announce.lists.wikimedia.…
Hello,
Community Wishathon is coming next month: March 15th-17th, 2024! So far, 78
participants have signed up to attend this online event since its first
announcement in December 2023 🎉
You can check out the project ideas and schedule on the event page at <
https://meta.wikimedia.org/wiki/Event:WishathonMarch2024>. These ideas come
from the wishes people share in Community Wishlist Survey <
https://meta.wikimedia.org/wiki/Community_Wishlist_Survey>. If you're a
user, developer, designer, or product lead, you are welcome to join! You
can sign up for the event on the wiki page and add your name, time zone,
and preferred contact method to a project by *February 15th, 2024*.
Before the event, you can learn about the projects and connect with others
who are interested. You can contact them using their preferred contact
method and consider forming project groups on Telegram.
During the event, there will be an opening session to introduce the event
and project ideas, async time for working on projects, self-organized
virtual project group meetings, and a showcase ceremony to share what each
group accomplished during the event. If you want to help with the event or
have questions, let us know on the event's talk page.
Cheers,
Srishti
On behalf of the Wishathon organizing committee
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation <https://wikimediafoundation.org/>
Hello everyone,
The second edition of the Language & Internationalization newsletter
(January 2024) is available at this link: <
https://www.mediawiki.org/wiki/Wikimedia_Language_engineering/Newsletter/20…
>.
This newsletter is compiled by the Wikimedia Language team. It provides
updates from October–December 2023 quarter on new feature development,
improvements in various language-related technical projects and support
efforts, details about community meetings, and contributions ideas to get
involved in projects.
To stay updated, you can subscribe to the newsletter on its wiki page. If
you have any feedback or ideas for topics to feature in the newsletter,
please share them on the discussion page, accessible here: <
https://www.mediawiki.org/w/index.php?title=Talk:Wikimedia_Language_enginee…
>.
Cheers,
Srishti
On behalf of the WMF Language team
*Srishti Sethi*
Senior Developer Advocate
Wikimedia Foundation <https://wikimediafoundation.org/>