Hi everyone,
I'm organizing a contest for people in Romania willing to contribute to Wikimedia code. [1] In order to automatically grade the contributions, we're using a tool already developed be our partners, ROSEdu, which reviews changes made on github [2][3].
The current (github-based) workflow is: 1. The admins add a number of repositories that qualify for the contest 2. The paticipants login with their github account (using oauth) 3. The software retrieves all the pull requests they made to the relevant projects. 4. A number of points is assigned for each pull request using a predefined formula (based on the number of touched lines, if the change was merged etc.; can by customized)
I need some guidance on how to replicate this workflow to Wikimedia's gerrit.
I've read the API docs [4] and looked at the gerrit uploader [5] and it seems that retrieving the reviews is fairly straightforward, since all the reviews seem to be available through unauthenticated access.
The real issue is how to match the user in the tool with the reviews without user intervention. Any ideas or advice are appreciated, but here are my thoughts on the issue:
1. Gerrit does not seem to support oauth authentication. I vaguely remember that the gerrit account used to be linked to the mw.org account. Is there any way I could use the mw.org auth to retrieve the gerrit account and/or authenticate to gerrit with it? The gerrit uploader seems to only use the mw account to put the username in the committer field and then uploads the change as itself.
2. The simplest (although not so secure) solution would be to ask people to submit their changes using the same email address used for their github account. This will only work if the user is willing to make their github address public (I'm not doing that, for instance).
3. Another idea would be to match the gerrit account with the github account. This sounds even less reliable.
4. Give up and ask the users to submit the email/user used for gerrit and check for cheaters manually (this should work as long as the number of contributors is small)
Thanks, Strainu
[1] https://www.mediawiki.org/wiki/Wikimedia_Challenge_powered_by_ROSEdu [2] http://challenge.rosedu.org/ [3] https://github.com/rosedu/challenge [4] https://gerrit.wikimedia.org/r/Documentation/rest-api.html [5] https://github.com/valhallasw/gerrit-patch-uploader
You can add multiple e-mails both to gerrit [0] and github [1]. As long as the e-mail address you are making commits with is added to both accounts, you can likely use your preexisting software directly on the mirrored github repos[2]. For example, my contributions to the citoid repo, all of which were made on gerrit, are also automatically* associated with my github account [3]. You could add a throwaway email to both both gerrit and github and set this as your git email [4] and then your e-mail will not be publicly exposed anywhere. In both gerrit and github, the email you set as your email in git is visible in your commits, as this is a property of git itself, not of gerrit/github in particular.
*Because these are mirrored repositories, contestants *must star the mirrored repository on github* [5] in order to have it associated with their github account, because they didn't open the pull request directly on github. This is the main issue as I see it, as it is an extra step and people are fundamentally bad at following rules :D.
[0] https://gerrit.wikimedia.org/r/#/settings/web-identities [1] https://help.github.com/articles/adding-an-email-address -to-your-github-account/ [2] https://github.com/wikimedia [3] https://github.com/wikimedia/citoid/commits?author=mvolz [4] https://help.github.com/articles/setting-your-email-in-git/ [5] https://help.github.com/articles/why-are-my-contribution s-not-showing-up-on-my-profile/
On Fri, Oct 21, 2016 at 10:13 AM, Strainu strainu10@gmail.com wrote:
Hi everyone,
I'm organizing a contest for people in Romania willing to contribute to Wikimedia code. [1] In order to automatically grade the contributions, we're using a tool already developed be our partners, ROSEdu, which reviews changes made on github [2][3].
The current (github-based) workflow is:
- The admins add a number of repositories that qualify for the contest
- The paticipants login with their github account (using oauth)
- The software retrieves all the pull requests they made to the
relevant projects. 4. A number of points is assigned for each pull request using a predefined formula (based on the number of touched lines, if the change was merged etc.; can by customized)
I need some guidance on how to replicate this workflow to Wikimedia's gerrit.
I've read the API docs [4] and looked at the gerrit uploader [5] and it seems that retrieving the reviews is fairly straightforward, since all the reviews seem to be available through unauthenticated access.
The real issue is how to match the user in the tool with the reviews without user intervention. Any ideas or advice are appreciated, but here are my thoughts on the issue:
- Gerrit does not seem to support oauth authentication. I vaguely
remember that the gerrit account used to be linked to the mw.org account. Is there any way I could use the mw.org auth to retrieve the gerrit account and/or authenticate to gerrit with it? The gerrit uploader seems to only use the mw account to put the username in the committer field and then uploads the change as itself.
- The simplest (although not so secure) solution would be to ask
people to submit their changes using the same email address used for their github account. This will only work if the user is willing to make their github address public (I'm not doing that, for instance).
- Another idea would be to match the gerrit account with the github
account. This sounds even less reliable.
- Give up and ask the users to submit the email/user used for gerrit
and check for cheaters manually (this should work as long as the number of contributors is small)
Thanks, Strainu
[1] https://www.mediawiki.org/wiki/Wikimedia_Challenge_powered_by_ROSEdu [2] http://challenge.rosedu.org/ [3] https://github.com/rosedu/challenge [4] https://gerrit.wikimedia.org/r/Documentation/rest-api.html [5] https://github.com/valhallasw/gerrit-patch-uploader
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
2016-10-21 16:08 GMT+03:00 Marielle Volz mvolz@wikimedia.org:
You can add multiple e-mails both to gerrit [0] and github [1]. As long as the e-mail address you are making commits with is added to both accounts, you can likely use your preexisting software directly on the mirrored github repos[2]. For example, my contributions to the citoid repo, all of which were made on gerrit, are also automatically* associated with my github account [3]. You could add a throwaway email to both both gerrit and github and set this as your git email [4] and then your e-mail will not be publicly exposed anywhere.
Hi Marielle,
Thank you for your response, it was really informative. Your solution seems basically equivalent to skipping gerrit entirely, right? The big downside of that is that we can't evaluate changes that were not merged. We also can't score the commit based on parameters from the review (such as how many versions were uploaded, etc.)
Strainu
Yes, that would mean there would be no information from gerrit. including information about unmerged reviews. In that case it is probably less than ideal :).
On Fri, Oct 21, 2016 at 3:54 PM, Strainu strainu10@gmail.com wrote:
2016-10-21 16:08 GMT+03:00 Marielle Volz mvolz@wikimedia.org:
You can add multiple e-mails both to gerrit [0] and github [1]. As long
as
the e-mail address you are making commits with is added to both accounts, you can likely use your preexisting software directly on the mirrored github repos[2]. For example, my contributions to the citoid repo, all of which were made on gerrit, are also automatically* associated with my github account [3]. You could add a throwaway email to both both gerrit
and
github and set this as your git email [4] and then your e-mail will not
be
publicly exposed anywhere.
Hi Marielle,
Thank you for your response, it was really informative. Your solution seems basically equivalent to skipping gerrit entirely, right? The big downside of that is that we can't evaluate changes that were not merged. We also can't score the commit based on parameters from the review (such as how many versions were uploaded, etc.)
Strainu
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 21 October 2016 at 10:13, Strainu strainu10@gmail.com wrote:
- Gerrit does not seem to support oauth authentication. I vaguely
remember that the gerrit account used to be linked to the mw.org account. Is there any way I could use the mw.org auth to retrieve the gerrit account and/or authenticate to gerrit with it? The gerrit uploader seems to only use the mw account to put the username in the committer field and then uploads the change as itself.
Gerrit uses LDAP authentication, which is controlled by wikitech.wikimedia.org and matches accounts there. This is separate to Wikimedia SUL which controls accounts on wikis like mediawiki.org. See https://phabricator.wikimedia.org/T148048 and the bottom of https://meta.wikimedia.org/wiki/Community_Tech/Tool_Labs_support/Tool_Labs_v...
2016-10-21 19:03 GMT+03:00 Alex Monk krenair@gmail.com:
On 21 October 2016 at 10:13, Strainu strainu10@gmail.com wrote:
- Gerrit does not seem to support oauth authentication. I vaguely
remember that the gerrit account used to be linked to the mw.org account. Is there any way I could use the mw.org auth to retrieve the gerrit account and/or authenticate to gerrit with it? The gerrit uploader seems to only use the mw account to put the username in the committer field and then uploads the change as itself.
Gerrit uses LDAP authentication, which is controlled by wikitech.wikimedia.org and matches accounts there. This is separate to Wikimedia SUL which controls accounts on wikis like mediawiki.org. See https://phabricator.wikimedia.org/T148048 and the bottom of https://meta.wikimedia.org/wiki/Community_Tech/Tool_Labs_support/Tool_Labs_v...
Thanks Alex, that's very nice! I have some follow-up questions: 1. What's the best way to match users between wiki and gerrit? I suspect the answer is username, since on gerrit one can register multple emails and I can't find a full/real name on wiki, but what about if a user is moved? Also, does the shell username has anything to do with gerrit? 2. When requesting a new OAuth consumer, what should I choose as the Type of grants being requested? - Authentication only, no API access. - Authentication only with access to real name and email address via Special:OAuth/identify, no API access. - Request authorization for specific permissions.
Thank you, Strainu
On Fri, Oct 21, 2016 at 2:13 AM, Strainu strainu10@gmail.com wrote:
The real issue is how to match the user in the tool with the reviews without user intervention. Any ideas or advice are appreciated, but here are my thoughts on the issue:
Can you better explain what you are after? Are you worried that the users are going to give positive reviews to themselves to bias the scores? That seems fairly easy to spot manually (and if they really want to do it, they could always register a seemingly unrelated gerrit account).
2016-10-22 1:16 GMT+03:00 Gergo Tisza gtisza@wikimedia.org:
Are you worried that the users are going to give positive reviews to themselves to bias the scores?
Authentication is used only to ensure they don't claim somebody else's submissions (say, Gerrit Patch Uploader's :) ). Yes, this could probably be detected manually, but we're trying to go with an automated workflow where manual interventions are at a minimum.
Can you better explain what you are after?
I'm simply trying to make it easy for the users. In the current version of the tool, they login with the github account and the rest happens "magically": the tool retrieves their pull requests and scores them according to a predefined set of criteria - no need for user input of any kind. I just want the same workflow for patches submitted to gerrit and I needed a way to authenticate the users and match the information I have from the OAuth endpoint with reviews from gerrit.
On Fri, Oct 21, 2016 at 4:38 PM, Strainu strainu10@gmail.com wrote:
2016-10-22 1:16 GMT+03:00 Gergo Tisza gtisza@wikimedia.org:
Are you worried that the users are going to give positive reviews to themselves to bias the scores?
Authentication is used only to ensure they don't claim somebody else's submissions (say, Gerrit Patch Uploader's :) ). Yes, this could probably be detected manually, but we're trying to go with an automated workflow where manual interventions are at a minimum.
Can you better explain what you are after?
I'm simply trying to make it easy for the users. In the current version of the tool, they login with the github account and the rest happens "magically": the tool retrieves their pull requests and scores them according to a predefined set of criteria - no need for user input of any kind. I just want the same workflow for patches submitted to gerrit and I needed a way to authenticate the users and match the information I have from the OAuth endpoint with reviews from gerrit.
Today there is no accessible mapping between Wikimedia unified accounts (the account you use on en.wikipedia.org as an example) and Wikitech/Gerrit accounts. As Alex pointed out earlier in the thread there is some work being done to unify these systems, but that unification is quite far off currently.
There is however a one to one mapping between a Wikitech username and Gerrit username. My Wikitech username is "BryanDavis" and so is my Gerrit username (https://gerrit.wikimedia.org/r/#/q/owner:BryanDavis). If the mapping is not an identity mapping, then it would be still be contained in the LDAP directory that any Labs project or Tool Labs tool can query. The "cn" LDAP attribute is a user's Wikitech username, so you can search for a Wikitech user's LDAP record with something like `ldapsearch -xLLL cn=BryanDavis` from a command line or a similar query using an LDAP library. I am unsure if Gerrit uses the "cn" or "sn" attribute of the same record as the account's login name. For many records in our LDAP directory it would not matter as the values are the same, but I know I ran across some records when I was deploying https://labsadmin.wikimedia.org/ where the two values differ.
OAuth was recently re-enabled on the Wikitech server, so you would need to register your OAuth consumer there (https://wikitech.wikimedia.org/wiki/Special:OAuthConsumerRegistration) and interact with wikitech.wikimedia.org in your client code.
Bryan
Me and Bryan have been discussing this. There might be a hacky way to map SUL accounts to LDAP accounts - Phabricator allows users to login via LDAP as you'd expect, but it also allows OAuth against Wikimedia SUL accounts. Therefore the mapping can exist there, where users have set it up. You'd probably have to: * Use https://phabricator.wikimedia.org/conduit/method/user.mediawikiquery/ to get a Phabricator account from a Wikimedia SUL username * Scrape their Phabricator userpage to find their LDAP name, as I don't believe Phabricator provides this via the API (but I could be wrong)
On Fri, Oct 21, 2016 at 3:38 PM, Strainu strainu10@gmail.com wrote:
I'm simply trying to make it easy for the users. In the current version of the tool, they login with the github account and the rest happens "magically": the tool retrieves their pull requests and scores them according to a predefined set of criteria - no need for user input of any kind. I just want the same workflow for patches submitted to gerrit and I needed a way to authenticate the users and match the information I have from the OAuth endpoint with reviews from gerrit.
In that case, I would require them to prove account ownership by sending an email to the gerrit email address with a verification link.
Or you could require that that email address is present in Github (it does not have to be the primary address, and this is a good practice anyway as it will ensure that the clone repo on Github attributes the patch to them correctly once it gets merged - although in theory the Gerrit owner, committer and author email address could be three different things, but that's unlikely to happen) and then verify that somehow. You can probably just upload a gist with that address and check whether Github attributes it to them.
Thank you all for your suggestions. I'll discuss with the people who implemented the original application and decide on the best approach given the limited resources and time.
Strainu
2016-10-22 3:23 GMT+03:00 Gergo Tisza gtisza@wikimedia.org:
On Fri, Oct 21, 2016 at 3:38 PM, Strainu strainu10@gmail.com wrote:
I'm simply trying to make it easy for the users. In the current version of the tool, they login with the github account and the rest happens "magically": the tool retrieves their pull requests and scores them according to a predefined set of criteria - no need for user input of any kind. I just want the same workflow for patches submitted to gerrit and I needed a way to authenticate the users and match the information I have from the OAuth endpoint with reviews from gerrit.
In that case, I would require them to prove account ownership by sending an email to the gerrit email address with a verification link.
Or you could require that that email address is present in Github (it does not have to be the primary address, and this is a good practice anyway as it will ensure that the clone repo on Github attributes the patch to them correctly once it gets merged - although in theory the Gerrit owner, committer and author email address could be three different things, but that's unlikely to happen) and then verify that somehow. You can probably just upload a gist with that address and check whether Github attributes it to them. _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Hi,
After extensive discussions with my fellow organizers, we decided to go with an oauth consumer on wikitech, and I submitted a new consumer. It would be great if Bryan or another Oauth admin could take a look at it (we're waaay overdue with that already).
Thank you, Strainu
2016-10-22 17:06 GMT+03:00 Strainu strainu10@gmail.com:
Thank you all for your suggestions. I'll discuss with the people who implemented the original application and decide on the best approach given the limited resources and time.
Strainu
2016-10-22 3:23 GMT+03:00 Gergo Tisza gtisza@wikimedia.org:
On Fri, Oct 21, 2016 at 3:38 PM, Strainu strainu10@gmail.com wrote:
I'm simply trying to make it easy for the users. In the current version of the tool, they login with the github account and the rest happens "magically": the tool retrieves their pull requests and scores them according to a predefined set of criteria - no need for user input of any kind. I just want the same workflow for patches submitted to gerrit and I needed a way to authenticate the users and match the information I have from the OAuth endpoint with reviews from gerrit.
In that case, I would require them to prove account ownership by sending an email to the gerrit email address with a verification link.
Or you could require that that email address is present in Github (it does not have to be the primary address, and this is a good practice anyway as it will ensure that the clone repo on Github attributes the patch to them correctly once it gets merged - although in theory the Gerrit owner, committer and author email address could be three different things, but that's unlikely to happen) and then verify that somehow. You can probably just upload a gist with that address and check whether Github attributes it to them. _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org