Hey,
This weekend I stumbled upon Coveralls [0], a service that provides code
coverage statistics for open source projects.
Currently I added it to the TravisCI build of both Ask and Diff, which was
really easy to do [1]. You can find reports on the coveralls site [2], and
can also get a quick idea of the coverage status via the badge on the
documentation page of the relevant component [3].
I've been wanting this kind of reporting for a long time, so we can track
the general trends of our coverage, and spot commits that significantly
lower it. So this coveralls thing is quite awesome \o/
[0] https://coveralls.io and https://coveralls.io/info/features
[1] https://gerrit.wikimedia.org/r/#/c/74960/
[2] https://coveralls.io/r/wikimedia/mediawiki-extensions-Diff
[3] https://github.com/wikimedia/mediawiki-extensions-Diff/
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
Hello,
As far as I checked, we should be using "nb" as the language code of "nowiki". As is known, (see bugs 46455 and 37459), Wikidata allows both codes. I've just come upon a bot request that is over 2 months old that I'm willing to tackle, just that I need verification: terms should use the language code "nb" rather than "no", correct?
As for statistics, there are 534741 uses of the "no" code at present, and only 294725 uses of the "nb" code (at present). As I said, I'm willing to have my bot make the necessary fixes, but I just need to verify i advance.
Thanks.
Hazard-SJ
Agenda as per Etherpad <http://etherpad.wmflabs.org/pad/p/Wikidata>
* Deployment prep / Wikivoyage on Tuesday, July 23
* Solr/ElasticSearch
* SpamBlackList + Wikidata -> backwardscompatibility of SpamBlackList with
pre-ContentHandler MW?
* to do for Berlin team: document all modules, their deployment status,
etc. -> Started it, but where actually to put it?
* https://bugzilla.wikimedia.org/show_bug.cgi?id=41586 -> "On wikidata.org,
api login returns wrong lgtoken" -> update?
If there is anything to add,just add it to the Etherpad or write me.
Until later,
Denny
--
Project director Wikidata
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. +49-30-219 158 26-0 | http://wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Hey,
While considering the moving of the components currently in the DataValues
git repository [0], it occurred to me the current components are not all
that nicely dividing the code. All of these components, with the exception
of DataTypes and ValueView, bundle whole inheritance hierarchies. A classic
package design mistake that leads to various problems. My current plan here
is to have a new component that holds the interfaces, exceptions and
trivial implementations for all of these components (again, except DTs and
VV). Many of the components currently dependent on one of the already
existing ones, would then just need this one, which is much more
abstract/stable, tackling most of the pains of the current component design.
Two questions:
* Any suggestions on what to call this new component to hold the interfaces?
* Any alternate approaches that improve upon the current situation?
There is some urgency to this, as the git repo split should happen soon, to
alleviate the pains currently caused by not having independently deployable
and released versions.
[0] DataValues, ValueParsers, ValueValidators, DataTypes, etc
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
There seems to be an issue with Jenkins. It appears to use an old version of
other extensions under some circumstances.
It's like this:
If you submit change 33 for extension A, which needs change 44 in extension B
(which isn't merged yet), jenkins will fail correctly fail.
BUT: When change 44 got merged into extension B, and you force Jenkins to re-run
(e.g. by rebasing change 33), it will *still* fail, apparently using an old
version of extension B.
It seems this is only the case for the "testextensions-master" job, not the
standalone "repo" and "client" jobs.
Here are some examples:
https://gerrit.wikimedia.org/r/#/c/72962/ fails for no good reason
https://integration.wikimedia.org/ci/job/mwext-Wikibase-testextensions-mast…https://gerrit.wikimedia.org/r/#/c/73772/ fails for no good reason
https://integration.wikimedia.org/ci/job/mwext-Wikibase-testextensions-mast…
Please gather more evidence/insights if you come across this issue.
Thanks,
Daniel
Hey all,
it seems the dispatcher stopped working about 17-18 hours ago (i.e. Friday
5pm Berlin / 8am SF). Could we please, as fast as possible investigate and
get it running again?
(DNS changes have been suggested as the culprits, I have no clue)
Cheers,
Denny
--
Project director Wikidata
Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
Tel. +49-30-219 158 26-0 | http://wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
Körperschaften I Berlin, Steuernummer 27/681/51985.
Hey,
A month ago we had some discussion on our internal list on only having a
single component per git repo. I now want to follow up on this. Since there
is no reason to do this on the internal list, I'm now doing it here and
including the original mail.
WikibaseQueryEngine and WikibaseQuery have been split off, and more
excitingly, we finally managed to do this with the DataModel component as
well. Now the dust has settled from those changes, I think it is time to
look at the next one: DataValues.
Right now this repo contains 6 components. Splitting this up thus means 5
new repos. We can do this one by one, though it probably makes sense to do
it close together. I suggest starting with ValueView and DataTypes, as
these are currently being the most awkward dependency wise.
Any objections against starting work on this next week (ie the one that
starts tomorrow)? (Do read my original mail (below) first in case you have
not yet already done so.)
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
---------- Forwarded message ----------
From: Jeroen De Dauw <jeroendedauw(a)gmail.com>
Date: 9 June 2013 01:28
Subject: One component per repository
To: Wikidata-intern <wikidata-intern(a)wikimedia.de>
Hey,
As you all know, right now we have git repositories containing multiple
components. There are two main reasons for this:
* We started with different repos for client, lib and repo and then had a
lot of hassle with moving code between them. This reason seems mainly
historical at this point. This kind of code moving happening only very
rarely at this point, at least in all components that are not client, lib
or repo. It might still be going on to some extend in those three, though I
suspect this has decreased since the start of the project, probably due to
us more clearly defining the component boundaries. In fact, I think the
primary reason we ended up moving so many things in these components was
caused by those boundaries being so poorly defined.
* Installation hassle. Burdening regular users of the software to know
about all the dependencies, the required versions and having to load them
in a specific version in LocalSettings is a bad idea. Ideally they should
not have to know about the dependencies at all. So putting everything in a
single repo is a reasonable idea if you lack a package management system.
Luckily we are now very close to having Composer work for wikibase, which
enables us to have as many packages as we deem fit, without bothering the
users with them. So that will not only mitigate making the installation
process more complex, it will reduce current complexity (since right now
people do need to care about the dependencies (Diff, Ask, DataValues, ...)).
In short, the main reasons for multiple components per repo no longer apply
in our current situation. OTOH there are some clear disadvantages:
* People in the PHP world tend to assume there is only one component in a
git repo. This is reflected in mainstream tools such as Composer and
TravisCI having this assumption build in. It is not possible (AFAIK) to
properly use Composer if you have all your packages in a single repo. For
instance, it is currently not possible to list ValueParsers as a distinct
component from DataValues in the package repositories. It is also not
easily doable to have Travis do a build for each component, you'd have to
add a two tier build process, which is a lot of added complexity.
* Since the packages contain multiple components, it is less easy to keep
track of changes to a single component. Or to make use of just a single
component, without having to care about the other ones. Reusability and
likelihood of outside contributions are decreased if your package contains
a ton of not applicable stuff to the person that just wants to use one of
the components in there.
* The notion that we cannot create new repositories for new components
promotes harmful practices. These components will just be put somewhere
based on what currently needs them, and possibly need to be moved closer to
the root of the dependency tree. A good example of this is the ValueView
code. If I'm not mistaken, some of this was originally in repo, then in lib
(same package but different component), and then in the DataValues repo.
The last step was made since we happened to be loading the DataValues repo
everywhere we need this JavaScript code. ValueView certainly does not
belong in the same repo however. Both components do very different things
and are even written in different languages, so they will have different
consumers, different contributors and different reasons for change.
* More clear and explicit dependency structure. Makes messing up of
dependencies easier to detect, and makes understanding the individual
components easier.
...
Given that, I suggest we stop the earlier pattern of putting multiple
components in the same repo, and work on moving misplaced components into
new repos. I will be doing this for the Database, Query and QueryEngine
components in the coming days, as these are not needed by any functionality
currently enabled anywhere. I suggest we do the same for the components in
DataValues and for DataModel. We can probably leave lib, repo and client as
they are, at least for now. Those are not really reusable in their current
form anyway, and are the hardest ones to make improvements to. Plus we
might want to tackle fixing issues with lib first. Putting DataModel in its
own repo is currently blocked by remaining legacy dependencies, which need
to be resolved first. That leaves the components in the DataValues repo as
the thing to start with. I suggest we tackle that one as soon as everyone
of the team got a more clear idea of how this will work with Composer and
when there are no new commits on gerrit against the relevant component(s).
"Oh no, we will end up with a million git repos now!" Not really. We have
14 components right now, and this number has not increased a lot lately.
The only recent additions are Database and QueryEngine. We might end up
with a few more over time, but not hundreds, or even dozens.
"I still don't want to care about 14 different repos!" The situation for
developers will not get worse (else I'd not be a fan of the idea to begin
with), and will actually improve in places. For instance, when working on
the QueryEngine component, you will now no longer need all this unrelated
client, lib, repo, datatypes, valueparsers, valuevalidators,
valueformatters and valueview code. If someone breaks the build for one of
those, you can happily not care, since that code is not relevant to the
work you are doing. You'll also have better CI support. And if you are not
working on any of the components you depend on, for instance when working
just on client, you can use Composer much like a regular user and not give
a damn about all indirect dependencies.
</walloftext><other people raging>
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--
Hey,
While working on the serialization and deserialization code for Ask (which
is now finally done - yay!), I figured we'd have use for the same
interfaces, exceptions and basic utilities almost everywhere we do this
kind of work. After bouncing of the idea to put this into its own component
of some people, I went ahead with this. The new component is named
Serialization (we failed to come up with a better name) and is now in a WMF
git repo.
You can browse the code at
https://github.com/wikimedia/mediawiki-extensions-Serialization
For now, the only direct dependency on this component is the Ask library.
It'll need to be included for deployment as soon as we deploy the
WikibaseQuery extension. Or as soon as we decide to load Ask for some other
reason.
Cheers
--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil. ~=[,,_,,]:3
--