On Tue, Feb 17, 2015 at 12:43 PM, Ricordisamoa <ricordisamoa(a)openmailbox.org> wrote: How would you like to see us encourage this more? It is nothing we actively do not want of course.

There are different repositories. They just get merged into one for deployment.

Agreed. Something we want to tackle.

Agreed but as I said this is going to be painful for a little longer until we have done the UI redesign. After that I want it to be more stable again obviously.

Cheers Lydia

Out of curiosity, was GitHub chosen because it fitted with your workflow?

Will you embrace Differential when it comes?

Finally, the ever-changing client-side APIs make gadgets development a pain in the ass. Agreed but as I said this is going to be painful for a little longer until we have done the UI redesign. After that I want it to be more stable again obviously. Thanks. Is there a task/page where progress is tracked?

Hi. I recently started following mediawiki/extensions/Wikibase on Gerrit, and quite astonishingly found that nearly all of the 100 most recently updated changes appear to be owned by WMDE employees (exceptions being one change by Legoktm and some from L10n-bot). This is not the case, for example, with mediawiki/core.

Sorry if this sounds like a slap in the face, but it had to be said.

What bugs me about it is that Wikidata has gone down the same road as Freebase and Neo4J in the sense of developing something ad-hoc that is not well understood. I understand the motivations that lead there, because there are requirements to meet that standards don't necessarily satisfy, plus Wikidata really is doing ambitious things in the sense of capturing provenance information. Perhaps it has come a little too late to help with Wikidata but it seems to me that RDF* and SPARQL* have a lot to offer for "data wikis" in that you can view data as plain ordinary RDF and query with SPARQL but you can also attach provenance and other metadata in a sane way with sweet syntax for writing it in Turtle or querying it in other ways. Another way of thinking about it is that RDF* is formalizing the property graph model which has always been ad hoc in products like Neo4J. I can say that knowing what the algebra is you are implementing helps a lot in getting the tools to work right. So you not only have SPARQL queries as a possibility but also languages like Gremlin and Cypher and this is all pretty exciting. It is also exciting that vendors are getting on board with this and we are going to seeing some stuff that is crazy scalable (way past 10^12 facts on commodity hardware) very soon. On Tue, Feb 17, 2015 at 12:20 PM, Jeroen De Dauw &lt;jeroendedauw(a)gmail.com <mailto:jeroendedauw@gmail.com>> wrote: Hey, As Lydia mentioned, we obviously do not actively discourage outside contributions, and will gladly listen to suggestions on how we can do better. That being said, we are actively taking steps to make it easier for developers not already part of the community to start contributing. For instance, we created a website about our software itself [0], which lists the MediaWiki extensions and the different libraries [1] we created. For most of our libraries, you can just clone the code and run composer install. And then you're all set. You can make changes, run the tests and submit them back. Different workflow than what you as MediaWiki developer are used to perhaps, though quite a bit simpler. Furthermore, we've been quite progressive in adopting practices and tools from the wider PHP community. I definitely do not disagree with you that some things could, and should, be improved. Like you I'd like to see the Wikibase git repository and naming of the extensions be aligned more, since it indeed is confusing. Increased API stability, especially the JavaScript one, is something else on my wish-list, amongst a lot of other things. There are always reasons of why things are the way they are now and why they did not improve yet. So I suggest to look at specific pain points and see how things can be improved there. This will get us much further than looking at the general state, concluding people do not want third party contributions, and then protesting against that. [0] http://wikiba.se/ [1] http://wikiba.se/components/ Cheers -- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate Evil software architect at Wikimedia Germany ~=[,,_,,]:3 _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org <mailto:Wikidata-l@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata-l -- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2(a)gmail.com <mailto:ontology2@gmail.com> http://legalentityidentifier.info/lei/lookup _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l

What bugs me about it is that Wikidata has gone down the same road as Freebase and Neo4J in the sense of developing something ad-hoc that is not well understood. I understand the motivations that lead there, because there are requirements to meet that standards don't necessarily satisfy, plus Wikidata really is doing ambitious things in the sense of capturing provenance information. Perhaps it has come a little too late to help with Wikidata but it seems to me that RDF* and SPARQL* have a lot to offer for "data wikis" in that you can view data as plain ordinary RDF and query with SPARQL but you can also attach provenance and other metadata in a sane way with sweet syntax for writing it in Turtle or querying it in other ways. Another way of thinking about it is that RDF* is formalizing the property graph model which has always been ad hoc in products like Neo4J. I can say that knowing what the algebra is you are implementing helps a lot in getting the tools to work right. So you not only have SPARQL queries as a possibility but also languages like Gremlin and Cypher and this is all pretty exciting. It is also exciting that vendors are getting on board with this and we are going to seeing some stuff that is crazy scalable (way past 10^12 facts on commodity hardware) very soon. On Tue, Feb 17, 2015 at 12:20 PM, Jeroen De Dauw &lt;jeroendedauw(a)gmail.com&gt; wrote: -- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2(a)gmail.com http://legalentityidentifier.info/lei/lookup _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l

Dear Gerard: ... It seems that, in this bright future, you are forgetting our past. Those very "ivory towers" that you so scold are where Wikidata has been conceived. RDF is the reason why we have properties as first-class objects in Wikidata. Even before such technical "details", the vision of a Semantic Web that enables the free exchange of information beyond system boundaries for Denny and myself has been the first and foremost inspiration for much of the work that went into preparing and realizing Wikidata. Surely not all of Wikidata came from this one source of inspiration -- e.g., the crucial insight that all of this should be in a single multilingual site is due to Erik Moeller -- but without all of the work in semantic technologies we would not have Wikidata today. People from different backgrounds are working together here. If you want to be part of such a community, you should abandon outdated stereotypes, and in particular stop using "academic" as a pejorative. Markus _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l

Gerard, I should probably keep my mouth shut about this but I am so offended but what you say that I am not. I am not an academic. The people behind Wikidata are. I am a professional programmer who has spent a lot of time being the guy who finishes what other people started; I typically come on when a project has been two years late for two years and I do whatever it takes to get the product in front of the customer. I know that building a system in PHP to do what Wikidata is the road to hell not because I hate PHP but because I have done it myself and learned it through experience. I first heard about Wikidata at SemTech in San Francisco and I was told very directly that they were not interested in working with anybody who was experienced with putting data from generic database in front of users because they had worked so hard to get academic positions and get a grant from the Allen Institute and it is more cost-effective and more compatible with academic advancement to hire a bunch of young people who don't know anything but will follow orders. If you hire 3x the people you need and have good management you can make that work; just the fact that the project has a heavyweight project manager is a very good sign. I mean that is how the CMM 5 shops in India do it, and perhaps they have done that because actually Wikidata has succeeded quite well from a software engineering perspective. Now so far as RDF and SPARQL go if you'd seen my history you'd see I am an American in the Winston Churchill sense that I've tried everything except for the right thing and finally settled on it. I really had my conversion when I discovered I could take data from Freebase and put it through something more like a reconstruction than a transformation, convert it to RDF and I could write SPARQL queries that "just worked". RDF* and SPARQL* do not come from an academic background but from a commercial organization that expects to make money by satisfying people's needs and it is being supported by a number of other commercial organizations. See http://wiki.bigdata.com/wiki/index.php/Reification_Done_Right This is something that builds on everything successful about RDF and SPARQL and adds the "missing links" that it takes to implement data wikis. If somebody was starting Wikidata today or if the kind of billionaire who buys sports teams the way I might buy a game console wanted to fund an effort to keep Freebase going, RDF*/SPARQL* is the way to do it. What happens when you build a half-baked system from scratch and don't what algebra is using is that you run into problems that get progressively worse and you wind up like woman who ate the cat because she ate the rat and so forth. If I had a dime for every time I had to fix up some application where people could not figure out how to make primary keys that are unique or every boss who didn't want me to take my time to understand a race condition and wished I would be like the guy who made the race conditions, just trying random things until it sorta works I would be a billionaire and I would take Freebase over and fix all the things that are wrong with it. (Which are really not that bad, but never happened because Google didn't have any incentive to say, improve the book database.) Or would I? A structural problem with open data is that people are NOT paying for it. If you were paying for it, the publishers of the data would have an incentive to serve the PEOPLE who are using it. Wikidata is playing to whims of a few rich people and it could disappear at any time when those people get tired of it or decide they have what they want and don't want to make it any easier for competitors to follow them. Most conventional forms of academic funding that come from governments have the same problems. I mean, you get your grant, you publish a paper, it doesn't particularly matter that what you did worked or not. There is also the perpetual "project" orientation which is not suitable for things like arXiv.org or DBpedia which are really programs or operations. You can have a very hard time finding $400k a year for something that is high impact (i.e. 50,000 scientists use it every day), while next door there is somebody who got $5 million to make something that goes nowhere (i.e. the postdoc wants to use Hadoop to process the server logs but the number of hits is small enough you could do it by hand.) In terms of producing a usable product Wikidata has made some very good progress in terms of having data that is clean (if not copious), but in terms of having a usable query facility or dump files that are easy to work with, it is still behind DBpedia. I mean, you can do a lot with the DBpedia files with grep and awk and tools like that and it is not that hard to load it into a triple store and you have SPARQL 1.1 which is eminently practical because you can use your whole bag of tricks that you use with relational databases. In the big picture though, quality is in the eye of the end user, so it only happens if you have a closed feedback loop where the end user has a major influence on the behavior of the producer and certainly the possibility of making more money if you do a better job and (even more so) going out of business if you fail to do so is a powerful way to do it. The trouble is that most people interested in open data seem to think their time is worth nothing and other people's time is worth nothing and aren't interested in paying even a small amount for services so the producers throw stuff that almost works over the wall. I don't think it would be all that difficult for me to do for Wikidata what I did for Freebase but I am not doing it because you aren't going to pay for it. [... GOES BACK TO WORK ON A "SKUNK WORKS" PROJECT THAT JUST MIGHT PAY OFF] On Fri, Feb 20, 2015 at 8:09 AM, Gerard Meijssen < gerard.meijssen(a)gmail.com&gt; wrote: -- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2(a)gmail.com http://legalentityidentifier.info/lei/lookup _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l

>>

>>

Hi Paul! I understand your frustration, but let me put a few things into perspective. For reference: I'm employed by WMDE and work on wikibase/wikidata. I have been working on MediaWiki since 2005, and am being payed for it since 2008. Am 20.02.2015 um 19:14 schrieb Paul Houle: To the extend that most of us have some college degree. The only "full" academic involved is Markus Krötzsch, who together with Denny Vrandecic developed many of the concepts behind Wikidata. He acts as an advisor to the Wikidata project, but doesn't have any formal position. Oh, we also have a group of students working on their bachelor project with us. very because the Allen will Auch. Working with such people would be a drag. Luckily, we have an awesome team of full blooded programmers. Not that we get everything right, or done in time... commercial it is You'll be happy to hear that we are working with high priority to finally provide full query functionality. We are still evaluating options (WMF's Nik and Stas have been visiting the WMDE office for this, just this week - have a safe trip home, guys!), but the current favorite is, in fact, BlazeGraph, formerly BigData, by the people who came up with RDF* and RDR. If we end up using that, chances are good that we will be exposing a SPARQL endpoint directly. We may still find a deal breaker though, so no promise. Another option would be Neo4J, using a graph oriented mapping. We could still expose SPARQL (building upon Gremlin, IIRC), but I suspect that we'd probably rather expose something more domain specific, perhaps based on Magnus' WDQ syntax, that operates directly on the graph. SPARQL and somebody was teams the going, I still stand by the decision not to use a triple store as the primary storage for wikidata, for various reasons (MediaWiki integration, especially versioning, being among the most important ones). But I'm all for mapping our internal model to RDF, and exposing a SPARQL endpoint, if we can do that in a reliable manner with the available resources. I'd rather have limited query functionality with five nines uptime than a SPARQL endpoint that is down half the time. Speaking of mapping to RDF: Have you read <http://korrekt.org/papers/Wikidata-RDF-export-2014.pdf>? it or Wikidata development and hosting is funded by donations to Wikimedia, like all Wikimedia projects. The first year of development was indeed funded by large companies and trusts (AI2, Google, and the Moore Foundation), but to my knowledge they never tried to influence our decisions. We have never had academic funding. I don't think we are going to say "no" if we can get any, though. their time interested that difficult for me you If you mail me an application/offer, I'm happy to forward and, depending on content, champion it. Wikimedia doesn't pay as well as big tech companies (Wikimedia operates on a shoe string budget, compared to other sites with upwards of 100k hits per second), but the pay isn't shoddy either. Come and visit! Let's talk! -- Daniel Kinzler Senior Software Developer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V. _______________________________________________ Wikidata-l mailing list Wikidata-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l

Hoi, Obviously you forgot about OmegaWiki. It can still do things Wikidata is incapable of.

Il 17/02/2015 12:53, Lydia Pintscher ha scritto: Using a single code review system and a simpler repository structure will indirectly encourage them. I'm now seeing Wikibase/Programmer's guide to Wikibase <https://www.mediawiki.org/wiki/Wikibase/Programmer%27s_guide_to_Wikibase>, which seems fairly detailed but partly duplicates the Gerrit help pages. Really? AFAICS development occurs on mediawiki/extensions/Wikibase <https://git.wikimedia.org/summary/?r=mediawiki/extensions/Wikibase.git> and on GitHub. mediawiki/extensions/WikibaseRepository <https://git.wikimedia.org/summary/?r=mediawiki/extensions/WikibaseRepository.git> and mediawiki/extensions/WikibaseClient <https://git.wikimedia.org/summary/?r=mediawiki/extensions/WikibaseClient.git> also exist but have always been empty.