Hey Steffen and Andy,
Continuing what I started on Twitter here, as some more characters might be
helpful :)
It seems that both our projects (FLOW3 and Wikidata) are in a similar
situation. We are using Gerrit as CR tool, and TravisCI to run our tests.
And we both want to have Travis run tests for all patchsets submitted to
Gerrit, and then +1 or -1 on verified based on the build passing or
failing. To what extend have you gotten such a thing to work on your
project? Is there code available anywhere? If both projects can use the
same code for this, I'd be happy to contribute to what you already have.
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
Hello everyone,
I have applied for GSoC 2014 with Mediawiki aiming to create a plugin that
can annotate statements in various websites and feed then as statements (
with references taken as website url and author in the case of Google
books, Wikisource etc.
Project Proposal is currently hosted at :
https://www.mediawiki.org/wiki/Wikidata_annotation_tool
I need to get more feedback from community such as what extra features we
require
from the tool which can prove this project more useful. I hope this project
is considered useful by the community.
So kindly, take a look at proposal and provide any valuable comments you
can.
Thanks
Note:
This mail has been crossposted to wikidata-l and wikitech-l
--
Amanpreet Singh,
IIT Roorkee
Hey all,
A quick update on the status of the new serialization and deserialization
code.
For a long time now, we've had two serialization formats. One intended for
external usage, and one intended for internal usage. The former one is the
format our web API uses. The later one is the format in which entities are
stored in the Wikibase Repo database, and is what ends up in the current
dumps.
The code responsible for serialization and deserialization is for both
formats not reusable and has a high degree of associated technical debt. To
tackle both the reusability and design problems, we started work on two new
components. One for the external format and one for the internal one.
* External one: https://github.com/wmde/WikibaseDataModelSerialization
* Internal one: https://github.com/wmde/WikibaseInternalSerialization
They are now both essentially feature complete. The next step will be to
replace usage of our existing code with usage of these new components, so
the old code can be removed. Once this is done, the components will be 100%
feature complete and very well tested, at which point their 1.0 versions
will be released.
For a description of the responsibility of both components and an
explanation of how to use them, see their README files.
Cheers
--
Jeroen De Dauw - http://www.bn2vs.com
Software craftsmanship advocate
Evil software architect at Wikimedia Germany
~=[,,_,,]:3
The folks of the Wikidata.lib project at the Hasso Plattner Institut have
developed an extension to Wikibase that allows us to suggest properties to add
to items, based on the properties already present (a very cool project, btw).
This is, conceptually, and extension to the Wikibase extension. This raises
problems for managing dependencies:
* conceptually, the extension (property suggester) depends *on* wikibase.
* practically, we want to install the property suggester as an optional
dependency (feature/plugin/extension) *of* wikibase.
So, how do we best express this? How can composer handle this?
I think the most obvious/important thing to do is to have a separate module for
the interface wikibase exposes to plugins/extensions. This would include the
interfaces of the service objects, and some factory/registry classes for
accessing these.
What's the best practice for exposing such a set of interfaces? How is this best
expressed in terms of composer manifests? What are the next steps to resolve the
circular/inverse dependencies we currently have?
-- daniel
PS: Below is an email in which Moritz Finke listed the dependencies the property
suggester currently has:
-------- Original-Nachricht --------
Betreff: PropertySuggester Dependencies
Datum: Thu, 6 Mar 2014 11:07:56 +0000
Von: Finke, Moritz <Moritz.Finke(a)student.hpi.uni-potsdam.de>
An: Daniel Kinzler <daniel.kinzler(a)wikimedia.de>
Hi,
unten sind die Abhängigkeiten des PropertySuggesters nach Klassen sortiert...
Grüße Moritz
Abhängigkeiten PropertySuggester:
GetSuggestions:
use Wikibase\DataModel\Entity\ItemId;
use Wikibase\DataModel\Entity\Property;
use Wikibase\DataModel\Entity\PropertyId;
use Wikibase\EntityLookup;
use Wikibase\Repo\WikibaseRepo;
use Wikibase\StoreFactory;
use Wikibase\Utils;
use ApiBase;
use ApiMain;
use DerivativeRequest;
WikibaseRepo::getDefaultInstance()->getEntityContentFactory();
StoreFactory::getStore( 'sqlstore' )->getEntityLookup();
StoreFactory::getStore()->getTermIndex()->getTermsOfEntities( $ids, 'property',
$language );
Utils::getLanguageCodes()
'type' => Property::ENTITY_TYPE )
SuggesterEngine:
use Wikibase\DataModel\Entity\Item;
use Wikibase\DataModel\Entity\PropertyId;
Suggestion:
use Wikibase\DataModel\Entity\PropertyId;
SimplePHPSuggester:
use Wikibase\DataModel\Entity\Item;
use Wikibase\DataModel\Entity\PropertyId;
use DatabaseBase;
use InvalidArgumentException;
GetSuggestionsTest:
use Wikibase\Test\Api\WikibaseApiTestCase;
SimplePHPSuggesterTest:
use Wikibase\DataModel\Entity\PropertyId;
use Wikibase\DataModel\Entity\Item;
use Wikibase\DataModel\Claim\Statement;
use Wikibase\DataModel\Snak\PropertySomeValueSnak;
use DatabaseBase;
use MediaWikiTestCase;
JavaScript:
wikibase.entityselector
wbEntityId
Hey all,
We recently finished with the initial version of a reusable component
capable of serializating and deserializating our DataModel [0]. This covers
the serialization format we use for our web API, and not the one we use
"internally" for storing our entities. This happens to be the format which
is in our current dumps, and is thus dealt with by various people and tools
outside of our actual core software. We are now starting with the creating
of a serialization component for this format [1].
As before, everyone is welcome to contribute to this component. The work is
very self contained and requires little knowledge of our wider codebase.
The advice I have for the DataModelSerialization component also applies
here, so have a look at if you want to contribute [2].
[0] https://github.com/wmde/WikibaseDataModelSerialization
[1] https://github.com/wmde/WikibaseInternalSerialization
[2]
http://lists.wikimedia.org/pipermail/wikidata-tech/2013-December/000356.html
Cheers
--
Jeroen De Dauw - http://www.bn2vs.com
Software craftsmanship advocate
Evil software architect at Wikimedia Germany
~=[,,_,,]:3
Hey there,
While testing I found that the notations of precision in the internal
dumpfiles is not always what I expected:
My example is the item Zwickau (json and stacktrace attached):
The precision in the geolocation is denoted as an decimal value, while
the implementation of GlobalCoordinatesValueImpl requires a long value
encoding a precision.
How should this discrepancy be handled best?
Greetings,
Fredo
PS: I am at the moment checking all the errors that occur while
converting an earlier dumpfile, so I am likely to stumble upon more
problems soon.
Hey,
(Included wikidata-tech)
Also, while you're at it, if you have a sec, you might like to look at the
> tentative idea for using a separate Wikibase store for the features we're
> working on:
> https://www.mediawiki.org/wiki/Editor_campaigns/Notes#Structured_data_store
>
The spirit is similar in many ways to what's in the EP extension, though
> we're targeting a much wider range of use cases, and the code should be
> much more modular. Also hoping to achieve a very slick UX (or UX's) with
> help from designers.
>
Will just reply to this part here.
>From what I read on this page, I'd not be thinking about using Wikibase. If
anything, Semantic MediaWiki seems like a better fit (though perhaps still
not a good one - I do not suggest you try using it here).
* Wikibase does not support queries and visualizations at this point -
you'd just have entity storage
* Our entity storage code is not reusable at this point
* Our two applications, Wikibase Repo and Wikibase Client are both in bad
shape design wise, making it hard to have them suit your needs (which are
different than the requirements for Wikidata.org). This is something that
should change regardless as it is slowing the Wikidata team down a lot,
though it might well take some time before we got rid of the big issues
there.
I'm also not sure what you'd really gain by having our current entity
storage rather than following a more conventional approach. You do not have
a user defined "schema" right? I'm getting the impression that what you
really want to do is store the data in tripplestore format. In case of the
Wikibase applications, we'll be using Wikibase QueryEngine for this, which
will maintain a second copy of the data that can be queried. Since this
component is not bound to the Wikibase applications, you could potentially
use it at some point (initial development has not quite finished yet).
https://www.mediawiki.org/wiki/Wikibase/Components
Using Wikibase QueryEngine would gain you some amount of interoperability,
esp as you'd essentially be using our DataModel. I'm rather skeptical our
DataModel makes much sense for your use case. You don't need Statements
(that have a rank and references) and perhaps not even claims (which have
qualifiers). This means you'll not have use for Item entities, which means
you'd need a new type of entity.
This does not mean you have to give up on interoperability altogether - you
can still use lower level components. The wiki page mentions geo and time.
You can indeed use our implementations for those by using the appropriate
DataValues libraries (also linked from the Wikibase/Components page).
There definitely is opportunity for reuse and interop here. I suggest we
identify the reusable components of our system that are useful to you, as
well as functionality that is not reusable yet, though would be beneficial
for you if it was.
Quoting from the wiki page itself:
> The code produced during the first few sprints will probably be pretty
straightforward. Still, it would be nice to have an architectural plan to
aim for. Here's a rough proposal:
I'm happy to see you are thinking about this. The diagram and text look
reasonable enough, though neither explains the boundaries (arguably the
most important part of an architecture) in detail. The dependencies
presumably all go inwards towards the "campaigns business logic" part. In
other words, this part does not depend on any of the others. The top layer
contains parts that would each have their own application and domain logic,
so these should definitely not be whole blobs that depend on the "reusable
UI logic" in the layer below. I cannot stress enough how important it is to
get that right - it makes a huge impact on the maintenance cost of the
project and how flexible and powerful it ends up being. (Just look at the
EducationProgram extension or at MW core to see how it can go wrong.)
Quoting from the wiki page itself:
> Question: Do we want to use existing frameworks at all? Composer?
Anything from Symfony?
Composer is a dependency manager (tool), not a framework. If your code is
binding to it, you are very likely doing something wrong.
I'm assuming you will be building this on top of MediaWiki, in which case
you already have a framework.
Generally it is good design practice to bind as little to a framework as
possible (frameworks ought to take the same place as the database and the
UI - at the edge, as an implementation detail that could theoretically be
replaced). It is nicer to depend on libraries, which your application will
control, rather than on a framework, which will control your application.
One of the worst kind of bindings you can have to a framework is to derive
from its concrete classes and put in your domain and application logic.
This is what practically all MediaWiki extensions are doing with special
pages, api modules, jobs, actions, etc. Those APIs are lousily designed,
making binding to them even more problematic. For the Wikibase applications
we made this same mistake, which was identified as such though review by
external experts.
In case of Symfony, which mostly adheres to sane design principles, the
framework is based on libraries. Making use of those can definitely be
beneficial to you. There is however no reason to limit yourself to Symfony
- there are many great libraries out there that are not part of the Symfony
project.
Good luck with your project - I hope it turns into a gem rather than a ball
of mud :)
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
Hi everyone,
I ran into some pretty annoying problems with coordinates specifically
the precision. The valid values of precision aren't properly documented
and enforced at the moment, see
https://bugzilla.wikimedia.org/show_bug.cgi?id=62105
I would like to know what is supposed to be valid so I know what
direction to go in.
Maarten