Hi,
Anyone have any advice regarding upgrading Wikibase? We want to upgrade the install at https://para-sports.es/ but there are a bunch of technical hurdles that are difficult to get past. These include:
1. Upgrading Mediawiki to the latest version in order to be able to upgrade Wikibase. This needs to be done without changing the local customization settings and while trying to maintain a potential Mediawiki farm, as we want to be able to have a local image thing like Commons that can be pulled into Wikibase where Commons would be. 2. Upgrading extensions to be current with new version of Mediawiki. 3. Upgrading Wikibase after upgrading Mediawiki. Our database for Wikibase is over 100,000 items, We don't want to screw this up. 4. Upgrading the query engine after this, and being able to better localize this as the current examples are from Wikidata, and we can't seem to get rid of the Wikidata logo from the Query engine. 5. Installing the OAuth extension. This is required for Quick statements extension. 6. Installing QuickStatements.
Has anyone else who has updated Wikibase dealt with these issues? Is there any advice?
Sincerely, Laura Hale
Hello Laura, and anybody else looking to upgrade their Wikibase install,
At Rhizome we have, with Tom Arrow's help, managed to migrate our custom Wikibase install to the docker-based distribution. While it was a bit of a lift, I certainly don't miss taking care of all the small components you have laid out :) The docker distribution comes with Quickstatements and the Query service pre-configured.
This is how we did it:
1. Create a database dump of the legacy Wikibase.
2. Set up the docker-based Wikibase distribution.
3. Import the database dump
docker exec -i *your_mysql_container* sh -c 'exec mysql -uroot -p"$MYSQL_ROOT_PASSWORD"' < *your-database-dump.sql*
4. Run the maintenance script to fix any database issues with updated environment
docker exec -i *your_wikibase_container* php maintenance/update.php
If all went well, you can proceed to authenticate Quickstatements via OAuth.
5. Getting the data into Blazegraph is not that easy. We had more than a year of edits that should go into the query service, all of which had of course disappeared from the "recent changes" listings the updating service is watching for data to be transported over. So this is what we did:
5.1 Created a directory "dumps" to hold future data exports.
5.2 Edited the docker-compose.yml file to mount that directory in the query container:
wdqs: image: wikibase/wdqs:0.3.0 volumes: - query-service-data:/wdqs/data *- ./dumps:/tmp/dumps* command: /runBlazegraph.sh networks: default: aliases: - wdqs.svc environment: - WIKIBASE_HOST=staging.catalog.rhizome.org - WDQS_HOST=wdqs.svc - WDQS_PORT=9999 expose: - 9999
5.3 Create a ttl dump into that directory:
docker exec *your_wikibase_container* php extensions/Wikibase/repo/maintenance/dumpRdf.php > dumps/*your_dump.ttl*
5.4 import the dump into Blazegraph:
docker exec *your_wdqs_container* curl "http://localhost:9999/bigdata/namespace/wdq/sparql" --data-urlencode "update=DROP ALL; LOAD file:///tmp/dumps/*your_dump.ttl*;"
5.5 Migrate relevant parts of LocalSettings.php into the container. This can be a bit tricky. You basically need to examine the pre-configured settings and your old settings line by line and decide what you'll need to keep and what to throw away. In the end, you'll map a local file into the docker container, similar to how it was done with the dumps directory. Our part of docker-compose.yml looks like this:
wikibase: image: wikibase/wikibase:1.30-bundle links: - mysql ports: - "8181:80" volumes: - ./mediawiki-images-data:/var/www/html/images *- ./LocalSettings.php:/var/www/html/LocalSettings.php* - ./Rhizome_Logo_160.png:/var/www/html/resources/assets/Rhizome_Logo_160.png - ./Styling:/var/www/html/resources/assets/Styling - ./mediawiki-extensions/SendGrid:/var/www/html/extensions/SendGrid - ./mediawiki-extensions/Scribunto:/var/www/html/extensions/Scribunto
That was it basically :) I definitely recommend doing a dry-run on some vm or a local machine.
On the way we had encountered a problem with the PHP bundled in the Wikibase container missing a certain calendaring extension. Tom has fixed that in the docker distribution I believe. Tom, can you confirm?
I hope this helps at least a little bit, Dragan
On 11.11.18 13:02, Laura Hale wrote:
Hi,
Anyone have any advice regarding upgrading Wikibase? We want to upgrade the install at https://para-sports.es/ but there are a bunch of technical hurdles that are difficult to get past. These include:
- Upgrading Mediawiki to the latest version in order to be able to upgrade Wikibase. This needs to be done without changing the local customization settings and while trying to maintain a potential Mediawiki farm, as we want to be able to have a local image thing like Commons that can be pulled into Wikibase where Commons would be.
- Upgrading extensions to be current with new version of Mediawiki.
- Upgrading Wikibase after upgrading Mediawiki. Our database for Wikibase is over 100,000 items, We don't want to screw this up.
- Upgrading the query engine after this, and being able to better localize this as the current examples are from Wikidata, and we can't seem to get rid of the Wikidata logo from the Query engine.
- Installing the OAuth extension. This is required for Quick statements extension.
- Installing QuickStatements.
Has anyone else who has updated Wikibase dealt with these issues? Is there any advice?
Sincerely, Laura Hale -- twitter: purplepopple
Wikibaseug mailing list Wikibaseug@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibaseug
There is even a 1.29 version of the image that you could migrate to. https://github.com/wmde/wikibase-docker/blob/master/wikibase/README.md
@Dragan you should totally write what you just pasted in the email somewhere. A blog post or something perhaps?
On the way we had encountered a problem with the PHP bundled in the
Wikibase container missing a certain calendaring extension. Tom has fixed that in the docker distribution I believe. Tom, can you confirm?
That is indeed now fixed
On Thu, 15 Nov 2018 at 14:53, Dragan Espenschied < dragan.espenschied@rhizome.org> wrote:
Hello Laura, and anybody else looking to upgrade their Wikibase install,
At Rhizome we have, with Tom Arrow's help, managed to migrate our custom Wikibase install to the docker-based distribution. While it was a bit of a lift, I certainly don't miss taking care of all the small components you have laid out :) The docker distribution comes with Quickstatements and the Query service pre-configured.
This is how we did it:
Create a database dump of the legacy Wikibase.
Set up the docker-based Wikibase distribution.
Import the database dump
docker exec -i *your_mysql_container* sh -c 'exec mysql -uroot -p"$MYSQL_ROOT_PASSWORD"' < *your-database-dump.sql*
- Run the maintenance script to fix any database issues with updated
environment
docker exec -i *your_wikibase_container* php maintenance/update.php
If all went well, you can proceed to authenticate Quickstatements via OAuth.
- Getting the data into Blazegraph is not that easy. We had more than a
year of edits that should go into the query service, all of which had of course disappeared from the "recent changes" listings the updating service is watching for data to be transported over. So this is what we did:
5.1 Created a directory "dumps" to hold future data exports.
5.2 Edited the docker-compose.yml file to mount that directory in the query container:
wdqs: image: wikibase/wdqs:0.3.0 volumes: - query-service-data:/wdqs/data *- ./dumps:/tmp/dumps* command: /runBlazegraph.sh networks: default: aliases: - wdqs.svc environment: - WIKIBASE_HOST=staging.catalog.rhizome.org - WDQS_HOST=wdqs.svc - WDQS_PORT=9999 expose: - 9999
5.3 Create a ttl dump into that directory:
docker exec *your_wikibase_container* php extensions/Wikibase/repo/maintenance/dumpRdf.php > dumps/*your_dump.ttl*
5.4 import the dump into Blazegraph:
docker exec *your_wdqs_container* curl "http://localhost:9999/bigdata/namespace/wdq/sparql" --data-urlencode "update=DROP ALL; LOAD file:///tmp/dumps/*your_dump.ttl*;"
5.5 Migrate relevant parts of LocalSettings.php into the container. This can be a bit tricky. You basically need to examine the pre-configured settings and your old settings line by line and decide what you'll need to keep and what to throw away. In the end, you'll map a local file into the docker container, similar to how it was done with the dumps directory. Our part of docker-compose.yml looks like this:
wikibase: image: wikibase/wikibase:1.30-bundle links: - mysql ports: - "8181:80" volumes: - ./mediawiki-images-data:/var/www/html/images *- ./LocalSettings.php:/var/www/html/LocalSettings.php* - ./Rhizome_Logo_160.png:/var/www/html/resources/assets/Rhizome_Logo_160.png - ./Styling:/var/www/html/resources/assets/Styling - ./mediawiki-extensions/SendGrid:/var/www/html/extensions/SendGrid - ./mediawiki-extensions/Scribunto:/var/www/html/extensions/Scribunto
That was it basically :) I definitely recommend doing a dry-run on some vm or a local machine.
On the way we had encountered a problem with the PHP bundled in the Wikibase container missing a certain calendaring extension. Tom has fixed that in the docker distribution I believe. Tom, can you confirm?
I hope this helps at least a little bit, Dragan
On 11.11.18 13:02, Laura Hale wrote:
Hi,
Anyone have any advice regarding upgrading Wikibase? We want to upgrade the install at https://para-sports.es/ but there are a bunch of technical hurdles that are difficult to get past. These include:
- Upgrading Mediawiki to the latest version in order to be able to
upgrade Wikibase. This needs to be done without changing the local customization settings and while trying to maintain a potential Mediawiki farm, as we want to be able to have a local image thing like Commons that can be pulled into Wikibase where Commons would be. 2. Upgrading extensions to be current with new version of Mediawiki. 3. Upgrading Wikibase after upgrading Mediawiki. Our database for Wikibase is over 100,000 items, We don't want to screw this up. 4. Upgrading the query engine after this, and being able to better localize this as the current examples are from Wikidata, and we can't seem to get rid of the Wikidata logo from the Query engine. 5. Installing the OAuth extension. This is required for Quick statements extension. 6. Installing QuickStatements.
Has anyone else who has updated Wikibase dealt with these issues? Is there any advice?
Sincerely, Laura Hale -- twitter: purplepopple
Wikibaseug mailing listWikibaseug@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wikibaseug
-- Dragan Espenschied Preservation Director Rhizome http://rhizome.org at the New Museum
Net Art Anthology http://anthology.rhizome.org | webrecorder.io | oldweb.today _______________________________________________ Wikibaseug mailing list Wikibaseug@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibaseug
wikibaseug@lists.wikimedia.org