Hello Wikidata community,
As mentioned in previous communications [1], we are continuing forward on the WDQS public graph split.
The next stage of our rollout is to change query.wikidata.org to point to the main graph (which can also be reached directly on query-main.wikidata.org), whereas the scholarly graph will be available on query-scholarly.wikidata.org.
Most queries should run fine on the appropriate graph (main vs scholarly), but for queries that must run across the full dataset, the queries must be (re)written to use SPARQL federation [2]. You can ask for help to rewrite your queries [3]. If you encounter issues please feel free to report a problem [4].
If you need more time to do the transition you can use the new legacy endpoint [5] (query-legacy-full.wikidata.org) which contains the full Wikidata graph, but is limited in terms of performance and availability. Please be mindful when using this legacy endpoint and use it only if really needed. This endpoint will be available until December 2025.
We are moving to this next stage on *Monday May 5* (*2025-05-05*).
[1] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_upd... [2] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_graph_split... [3] https://www.wikidata.org/wiki/Wikidata:Request_a_query [4] https://www.wikidata.org/wiki/Wikidata:Report_a_technical_problem/WDQS_and_S...
[5] https://phabricator.wikimedia.org/T384422
Ryan
Brief update:
We're still putting the final pieces in place for the cutover to the main graph, so we're pushing to *Tuesday May 6 (2025-05-06)*.
Ryan
On Thu, May 1, 2025 at 9:44 PM Ryan Kemper rkemper@wikimedia.org wrote:
Hello Wikidata community,
As mentioned in previous communications [1], we are continuing forward on the WDQS public graph split.
The next stage of our rollout is to change query.wikidata.org to point to the main graph (which can also be reached directly on query-main.wikidata.org), whereas the scholarly graph will be available on query-scholarly.wikidata.org.
Most queries should run fine on the appropriate graph (main vs scholarly), but for queries that must run across the full dataset, the queries must be (re)written to use SPARQL federation [2]. You can ask for help to rewrite your queries [3]. If you encounter issues please feel free to report a problem [4].
If you need more time to do the transition you can use the new legacy endpoint [5] (query-legacy-full.wikidata.org) which contains the full Wikidata graph, but is limited in terms of performance and availability. Please be mindful when using this legacy endpoint and use it only if really needed. This endpoint will be available until December 2025.
We are moving to this next stage on *Monday May 5* (*2025-05-05*).
[1] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_upd... [2] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_graph_split... [3] https://www.wikidata.org/wiki/Wikidata:Request_a_query [4] https://www.wikidata.org/wiki/Wikidata:Report_a_technical_problem/WDQS_and_S...
[5] https://phabricator.wikimedia.org/T384422
Ryan
Further update: we began the migration today but had to rollback due to queries receiving 502 errors. You can see the ~30 minute window where queries were affected here https://grafana.wikimedia.org/d/000000489/wikidata-query-service?from=1746565518823&orgId=1&to=1746568158166&var-cluster_name=wdqs&var-graph_type=%289102%7C919%5B35%5D%29&viewPanel=44 .
We're working on figuring out a fix, and we'll likely take another swing at the migration tomorrow (Wednesday).
Ryan
On Mon, May 5, 2025 at 7:19 PM Ryan Kemper rkemper@wikimedia.org wrote:
Brief update:
We're still putting the final pieces in place for the cutover to the main graph, so we're pushing to *Tuesday May 6 (2025-05-06)*.
Ryan
On Thu, May 1, 2025 at 9:44 PM Ryan Kemper rkemper@wikimedia.org wrote:
Hello Wikidata community,
As mentioned in previous communications [1], we are continuing forward on the WDQS public graph split.
The next stage of our rollout is to change query.wikidata.org to point to the main graph (which can also be reached directly on query-main.wikidata.org), whereas the scholarly graph will be available on query-scholarly.wikidata.org.
Most queries should run fine on the appropriate graph (main vs scholarly), but for queries that must run across the full dataset, the queries must be (re)written to use SPARQL federation [2]. You can ask for help to rewrite your queries [3]. If you encounter issues please feel free to report a problem [4].
If you need more time to do the transition you can use the new legacy endpoint [5] (query-legacy-full.wikidata.org) which contains the full Wikidata graph, but is limited in terms of performance and availability. Please be mindful when using this legacy endpoint and use it only if really needed. This endpoint will be available until December 2025.
We are moving to this next stage on *Monday May 5* (*2025-05-05*).
[1] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_upd... [2] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_graph_split... [3] https://www.wikidata.org/wiki/Wikidata:Request_a_query [4] https://www.wikidata.org/wiki/Wikidata:Report_a_technical_problem/WDQS_and_S...
[5] https://phabricator.wikimedia.org/T384422
Ryan
Alright, we've successfully cut over! All queries going through query.wikidata.org are now hitting the main graph. Performance looks acceptable. For the next couple hours we're at a lower total server count in main than we previously had with the full graph; however we're actively repurposing the old full-graph servers so once those are back online as main graph hosts we will be at a higher capacity than we previously had with the full graph.
Thanks for bearing with us, and as always don't hesitate to reach out if you're experiencing issues or have questions that weren't answered by the original communications.
Ryan
On Tue, May 6, 2025 at 3:50 PM Ryan Kemper rkemper@wikimedia.org wrote:
Further update: we began the migration today but had to rollback due to queries receiving 502 errors. You can see the ~30 minute window where queries were affected here https://grafana.wikimedia.org/d/000000489/wikidata-query-service?from=1746565518823&orgId=1&to=1746568158166&var-cluster_name=wdqs&var-graph_type=%289102%7C919%5B35%5D%29&viewPanel=44 .
We're working on figuring out a fix, and we'll likely take another swing at the migration tomorrow (Wednesday).
Ryan
On Mon, May 5, 2025 at 7:19 PM Ryan Kemper rkemper@wikimedia.org wrote:
Brief update:
We're still putting the final pieces in place for the cutover to the main graph, so we're pushing to *Tuesday May 6 (2025-05-06)*.
Ryan
On Thu, May 1, 2025 at 9:44 PM Ryan Kemper rkemper@wikimedia.org wrote:
Hello Wikidata community,
As mentioned in previous communications [1], we are continuing forward on the WDQS public graph split.
The next stage of our rollout is to change query.wikidata.org to point to the main graph (which can also be reached directly on query-main.wikidata.org), whereas the scholarly graph will be available on query-scholarly.wikidata.org.
Most queries should run fine on the appropriate graph (main vs scholarly), but for queries that must run across the full dataset, the queries must be (re)written to use SPARQL federation [2]. You can ask for help to rewrite your queries [3]. If you encounter issues please feel free to report a problem [4].
If you need more time to do the transition you can use the new legacy endpoint [5] (query-legacy-full.wikidata.org) which contains the full Wikidata graph, but is limited in terms of performance and availability. Please be mindful when using this legacy endpoint and use it only if really needed. This endpoint will be available until December 2025.
We are moving to this next stage on *Monday May 5* (*2025-05-05*).
[1] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_upd... [2] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_graph_split... [3] https://www.wikidata.org/wiki/Wikidata:Request_a_query [4] https://www.wikidata.org/wiki/Wikidata:Report_a_technical_problem/WDQS_and_S...
[5] https://phabricator.wikimedia.org/T384422
Ryan
Dear Ryan, thanks for the update and congrats with the launch!
Can you have a look at https://www.wikidata.org/wiki/Q20950365 and see how this needs to be updated? E.g. I just had to add the query-scholarly API endpoint which was not there yet. But then I realized, it also does not describe that query.wikidata.org does not have all data. My point, the split is not reflected in it yet. Actually, reading the documentation and figuring why a SERVICE wikibase:labels is not working in one of my queries on query-scholarly (https://w.wiki/E3vw), I actually realize that a lot of documentation also still needs updating. What is the timeline for that? Who oversees that "split"?
Egon
On Thu, 8 May 2025 at 23:15, Ryan Kemper rkemper@wikimedia.org wrote:
Alright, we've successfully cut over! All queries going through query.wikidata.org are now hitting the main graph. Performance looks acceptable. For the next couple hours we're at a lower total server count in main than we previously had with the full graph; however we're actively repurposing the old full-graph servers so once those are back online as main graph hosts we will be at a higher capacity than we previously had with the full graph.
Thanks for bearing with us, and as always don't hesitate to reach out if you're experiencing issues or have questions that weren't answered by the original communications.
Ryan
On Tue, May 6, 2025 at 3:50 PM Ryan Kemper rkemper@wikimedia.org wrote:
Further update: we began the migration today but had to rollback due to queries receiving 502 errors. You can see the ~30 minute window where queries were affected here https://grafana.wikimedia.org/d/000000489/wikidata-query-service?from=1746565518823&orgId=1&to=1746568158166&var-cluster_name=wdqs&var-graph_type=%289102%7C919%5B35%5D%29&viewPanel=44 .
We're working on figuring out a fix, and we'll likely take another swing at the migration tomorrow (Wednesday).
Ryan
On Mon, May 5, 2025 at 7:19 PM Ryan Kemper rkemper@wikimedia.org wrote:
Brief update:
We're still putting the final pieces in place for the cutover to the main graph, so we're pushing to *Tuesday May 6 (2025-05-06)*.
Ryan
On Thu, May 1, 2025 at 9:44 PM Ryan Kemper rkemper@wikimedia.org wrote:
Hello Wikidata community,
As mentioned in previous communications [1], we are continuing forward on the WDQS public graph split.
The next stage of our rollout is to change query.wikidata.org to point to the main graph (which can also be reached directly on query-main.wikidata.org), whereas the scholarly graph will be available on query-scholarly.wikidata.org.
Most queries should run fine on the appropriate graph (main vs scholarly), but for queries that must run across the full dataset, the queries must be (re)written to use SPARQL federation [2]. You can ask for help to rewrite your queries [3]. If you encounter issues please feel free to report a problem [4].
If you need more time to do the transition you can use the new legacy endpoint [5] (query-legacy-full.wikidata.org) which contains the full Wikidata graph, but is limited in terms of performance and availability. Please be mindful when using this legacy endpoint and use it only if really needed. This endpoint will be available until December 2025.
We are moving to this next stage on *Monday May 5* (*2025-05-05*).
[1] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_upd... [2] https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_graph_split... [3] https://www.wikidata.org/wiki/Wikidata:Request_a_query [4] https://www.wikidata.org/wiki/Wikidata:Report_a_technical_problem/WDQS_and_S...
[5] https://phabricator.wikimedia.org/T384422
Ryan
Wikidata mailing list -- wikidata@lists.wikimedia.org Public archives at https://lists.wikimedia.org/hyperkitty/list/wikidata@lists.wikimedia.org/mes... To unsubscribe send an email to wikidata-leave@lists.wikimedia.org