TL;DR: Getting started with RESTBase is now easier thanks to a new SQLite3 backend

Hello,

The WMF Services team~[0] is proud to announce the initial release of the SQLite storage back-end~[1] for RESTBase~[2], the service that powers Wikimedia projects’ public REST API~[3]. Traditionally, RESTBase was limited to Cassandra persistence, which, while scalable, presented a weighty and difficult dependency for smaller installations and test environments. This new module allows RESTBase administrators to replace Cassandra with a lightweight, file-based persistence based on SQLite.

This is an exciting milestone in RESTBase’s development. We expect the option to run RESTBase atop an SQLite database to significantly widen the target audience, allowing users with small installations and limited resources to take advantage of this valuable service. We are also hoping to attract more contributors by having a simpler, less resource-intensive alternative to Cassandra; SQLite is now the storage module of choice in MediaWiki-Vagrant~[6] installations with the restbase role enabled. Contributing is just a vagrant up away!

The abstract storage interface~[7] we developed for RESTBase allows users to seamlessly switch between storage back-ends (Cassandra or SQLite). Due to data volumes we are facing in WMF, we are not using the SQLite back-end in production. However, we are testing each RESTBase changeset against both Cassandra and SQLite to ensure they are behaving as expected. Additionally, a 93% code coverage makes us confident to recommend it for production use.

Bringing the SQLite storage module to a usable state represents a significant amount of team effort, but nevertheless we’d like to extend a special thank you to Petr Pchelko for his tireless efforts in driving this project home. Thanks Petr!

This is only the first step in providing better third-party and developer support. One of the next challenges will be basing Parsoid~[8] on service-runner~[9,10], a general-purpose library for running and managing Node.JS services. Amongst other things, it enables bundling and running multiple services together. Packaging and distributing Parsoid and RESTBase together will simplify their installation, configuration and administration in small-setup and development environments.

Best,

Marko Obrovac, PhD Senior Services Engineer Wikimedia Foundation

[0] https://www.mediawiki.org/wiki/Wikimedia_Services [1] https://github.com/wikimedia/restbase-mod-table-sqlite [2] https://www.mediawiki.org/wiki/RESTBase [3] https://en.wikipedia.org/api/rest_v1/?doc (for a complete list of supported domains go to https://rest.wikimedia.org/) [4] https://phabricator.wikimedia.org/tag/restbase/ [5] https://phabricator.wikimedia.org/tag/restbase-api/ [6] https://www.mediawiki.org/wiki/MediaWiki-Vagrant [7] https://github.com/wikimedia/restbase-mod-table-spec [8] https://www.mediawiki.org/wiki/Parsoid [9] https://github.com/wikimedia/service-runner [10] https://phabricator.wikimedia.org/T90668