Hi, I am a 3rd year undergraduate student of computer science, pursuing my B.Tech degree at RCC Institute of Information Technology. I am proficient in Java, PHP and C#. Among the project ideas on the GSoC 2013 ideas page, the one particular idea that seemed really interesting to me is developing an Entity Suggester for Wikidata. I want to work on it. I am passionate about data mining, big data and recommendation engines, therefore this idea naturally appeals to me a lot. I have experience with building music and people recommendation systems, and have worked with Myrrix and Apache Mahout. I recently designed and implemented such a recommendation system and deployed it on a live production site, where I'm interning at, to recommend Facebook users to each other depending upon their interests. The problem is, the documentation for Wikidata and the Wikibase extension seems pretty daunting to me since I have not ever configured a mediawiki instance or actually used it. (I am on my way to try it out following the instructions at http://www.mediawiki.org/wiki/Summer_of_Code_2013#Where_to_start.) I can easily build a recommendation system and create a web-service or REST based API through which the engine can be trained with existing data, and queried and all. This seems to be a collaborative filtering problem (people who bought x also bought y). It'll be easier if I could get some help about the part where/how I need to integrate it with Wikidata. Also, some sample datasets (csv files?) or schemas (just the column names and data types?) would help a lot, for me to figure this out. I have added this email as a comment on the bug report at https://bugzilla.wikimedia.org/show_bug.cgi?id=46555#c1. Please ask me if you have any questions. :-) Thanks, Nilesh -- A quest eternal, a life so small! So don't just play the guitar, build one. You can also email me at contact(a)nileshc.com or visit my website<http://www.nileshc.com/> _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

I have a question - when someone creates a new statement, for suggesting "properties", I can use collaborative filtering to make suggestions. Example, explained in the simplest terms - suppose there are X cities in the dataset. The user is adding another city (writes 'city in Australia' for short description). The system checks all other cities, figures out the common properties and suggests them. Cool. But I can't get any "exact" ideas off the top of my head that can used to suggest "values" for the properties. Suppose one of the recommended properties is "population". How can I make the system guess its value? (Am I getting this right?) Have you guys got anything on your minds regarding this? Please point me to the right direction. :)

Hi Lydia, That helps a lot, and makes it way more interesting. Rather than being a one-size-fits-all solution, as it seems to me, each property or each type of property (eg. different relationships) will need individual attention and different methods/metrics for recommendation. The examples you gave, like continents, sex, relations like father/son, uncle/aunt/spouse, or place-oriented properties like place of birth, country of citizenship, ethnic group etc. - each type has a certain pattern to it (if a person was born in the US, US should be one of the countries he was a citizen of; US census/ethnicity statistics may be used to predict ethnic group etc.) I'm already starting to chalk out a few patterns and how they can be used for recommendation. In my proposal, should I go into details regarding these? Or should I just give a few examples and explain how the algorithms would work, to explain the idea?

Awesome. Got it. I see what you mean, great, thank you. :) Cheers, Nilesh On Apr 28, 2013 2:56 AM, "Lydia Pintscher" &lt;lydia.pintscher(a)wikimedia.de&gt; wrote:

On Fri, May 3, 2013 at 5:39 AM, Nilesh Chakraborty &lt;nilesh(a)nileshc.com once could server long up dump) or One of the best examples of a MediaWiki extension interacting with a Java service is how Solr is used. Solr is still pretty new at Wikimedia, though. It is used with the GeoData extension and then Solr is used by geodata api modules. I think Solr gets updated via a cronjob (solrupdate.php) which creates jobs in the job queue. Not 100% sure of the exact details. I do not think direct access to the live database is very practical. I think anyway the data (json blobs) would need indexing in some particular way to support what the entity selector needs to do. http://www.mediawiki.org/wiki/Extension:GeoData The Translate extension also uses Solr in some way, though I am not very familiar with the details. On the operations side, puppet is used to configure everything. The puppet git repo is available to see how things are done. https://gerrit.wikimedia.org/r/gitweb?p=operations/puppet.git;a=tree;f=modu… inside wikidata-vagrant is maintained in github, though I think might not work perfectly right now. We need to update it and it's on our to-do, and perhaps could be moved to gerrit. I do not know about integrating the jars, but should be possible. Cheers, Katie Filbert [answering from this email, as I am not subscribed to wikitech-l on my wikimedia.de email] > > Thanks, > Nilesh > > > > On Sun, Apr 28, 2013 at 3:01 AM, Nilesh Chakraborty &lt;nilesh(a)nileshc.com > > > > Awesome. Got it. > > > > I see what you mean, great, thank you. :) > > > > Cheers, > > Nilesh > > On Apr 28, 2013 2:56 AM, "Lydia Pintscher" < lydia.pintscher(a)wikimedia.de into V. one. -- @wikimediadc / @wikidata _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

One more thing - should I create a new thread to discuss on prototyping my project (entity suggester) and to discuss any issues I'm facing along the way or ask for help? Or should I just stick to this old thread?