Hi. The call with the WMF today turned up a pretty big issue.
WikibaseQueryEngine (WQE) depends on two big 3rd party libraries:
* Symphony (used for implementing a command line client for the query engine,
which we use to generate database tables)
* doctine/dbal (a database abstraction layer we use to run queries and create
In order to deploy these, the WMF would require a line-by-line review of
something around 50thousand lines of code (or maybe it was 30thousand, or
80thousand - a lot, in any case). This is not feasible.
We (Katie, Jeroen, Chris, Nik, me, etc) have come up with a plan to get rid of
1) Split the command line interface into a separate component (WQE-CLI perhaps)
that would not be deployed. Symphony is then out of the picture.
2) Go back to using MediaWiki's DB abstraction for running queries. This should
3) For generating DB tables (aka schema creation), we create a separate
component (WQE schema generator or something) that would use dbal to generate
staic sql files for the supported db systems (most importantly, mysql and
sqlite). This would be part of our build step, but the code relying on dbal
would not be deployed. Only the generated sql files would be used for deployment
(either through the update script, or, in the case of WMF, manually).
In the end, WQE could use either MW or dbal for running queries, so we could
deploy it without dbal on the WMF cluster, but it could also be used outside
If there are no objections, I propose to take this on for the next sprint.
Perhaps we can start to split this into tasks on bugzilla already.
Senior Software Developer
Gesellschaft zur Förderung Freien Wissens e.V.