There seems to be extremely little documentation on Wikimedia's GitHub project https://github.com/wikimedia ... I can only find https://www.mediawiki.org/wiki/Gerrit/GitHub which mostly says we mirror a bunch of stuff from gerrit. And I know we have continuous integration of some kind set up for some projects, but it doesn't seem to be well documented in a place I could find.
There are also some repos that are mirrors of gerrit, and other repos that are primary repos, and it's a bit unclear what's what. More importantly, when folks have repos that they've been running on GitHub already and want to move into the wikimedia project (rather than switch to gerrit), what's the procedure? I'm an admin/owner so I can manually import people's repos but I'm not sure whether I'm supposed to... :)
We also have a lot of admins, which I wonder is necessary: https://github.com/orgs/wikimedia/people?utf8=%E2%9C%93&query=role%3Aown... https://github.com/orgs/wikimedia/people?utf8=%E2%9C%93&query=role%3Aowner+ Do we do any security review / removal of old accounts, or have a procedure for adding new admins?
-- brion
On Mon, Apr 25, 2016 at 9:19 AM, Brion Vibber bvibber@wikimedia.org wrote:
There seems to be extremely little documentation on Wikimedia's GitHub project https://github.com/wikimedia ... I can only find https://www.mediawiki.org/wiki/Gerrit/GitHub which mostly says we mirror a bunch of stuff from gerrit. And I know we have continuous integration of some kind set up for some projects, but it doesn't seem to be well documented in a place I could find.
There are also some repos that are mirrors of gerrit, and other repos that are primary repos, and it's a bit unclear what's what. More importantly, when folks have repos that they've been running on GitHub already and want to move into the wikimedia project (rather than switch to gerrit), what's the procedure? I'm an admin/owner so I can manually import people's repos but I'm not sure whether I'm supposed to... :)
The really brief procedure Timo came up with during the Librarization project is documented on wiki [0] but probably kind of buried in the details about developing new libraries.
We also have a lot of admins, which I wonder is necessary: https://github.com/orgs/wikimedia/people?utf8=%E2%9C%93&query=role%3Aown... https://github.com/orgs/wikimedia/people?utf8=%E2%9C%93&query=role%3Aowner+ Do we do any security review / removal of old accounts, or have a procedure for adding new admins?
Not that I am aware of. Rights there tend to work a lot like getting elevated rights on mediawiki.org: the rights are handed out by existing admins when somebody asks for something that will be easily solved by giving them rights. I think there was some amount of cleanup done a few months ago that got admins to either add 2fa to their github account or be removed.
[0]: https://www.mediawiki.org/wiki/Manual:Developing_libraries#Transferring_an_e...
Bryan
On Mon, Apr 25, 2016 at 8:34 AM, Bryan Davis bd808@wikimedia.org wrote:
Not that I am aware of. Rights there tend to work a lot like getting elevated rights on mediawiki.org: the rights are handed out by existing admins when somebody asks for something that will be easily solved by giving them rights. I think there was some amount of cleanup done a few months ago that got admins to either add 2fa to their github account or be removed.
Correct, all admins should have two-factor setup. I believe everyone who is an admin there has +2 in gerrit, and a reason to have the rights in Github. I'd propose those 3 things as a minimal standard, since I don't think we ever defined one.
On 25 April 2016 at 17:01, Chris Steipp csteipp@wikimedia.org wrote:
Correct, all admins should have two-factor setup. I believe everyone who is an admin there has +2 in gerrit, and a reason to have the rights in Github. I'd propose those 3 things as a minimal standard, since I don't think we ever defined one
I don't think everyone with admin there at the moment meets those criteria.
On Mon, Apr 25, 2016 at 9:30 AM Alex Monk krenair@gmail.com wrote:
On 25 April 2016 at 17:01, Chris Steipp csteipp@wikimedia.org wrote:
Correct, all admins should have two-factor setup. I believe everyone who
is
an admin there has +2 in gerrit, and a reason to have the rights in
Github.
I'd propose those 3 things as a minimal standard, since I don't think we ever defined one
I don't think everyone with admin there at the moment meets those criteria.
What Chris said. And what Alex said.
-Chad
Le 25/04/2016 17:34, Bryan Davis a écrit :
We also have a lot of admins, which I wonder is necessary: https://github.com/orgs/wikimedia/people?utf8=%E2%9C%93&query=role%3Aown... https://github.com/orgs/wikimedia/people?utf8=%E2%9C%93&query=role%3Aowner+ Do we do any security review / removal of old accounts, or have a procedure for adding new admins?
Not that I am aware of. Rights there tend to work a lot like getting elevated rights on mediawiki.org: the rights are handed out by existing admins when somebody asks for something that will be easily solved by giving them rights. I think there was some amount of cleanup done a few months ago that got admins to either add 2fa to their github account or be removed.
The private task is: https://phabricator.wikimedia.org/T118946
I havent followed up on it though but it definitely prompted a lot of admin to enable 2FA.
The up-to-date list is available if you get admin on Github: https://github.com/orgs/wikimedia/people?query=role%3Aowner
On Mon, Apr 25, 2016 at 8:20 AM Brion Vibber bvibber@wikimedia.org wrote:
There seems to be extremely little documentation on Wikimedia's GitHub project https://github.com/wikimedia ... I can only find https://www.mediawiki.org/wiki/Gerrit/GitHub which mostly says we mirror a bunch of stuff from gerrit. And I know we have continuous integration of some kind set up for some projects, but it doesn't seem to be well documented in a place I could find.
There are also some repos that are mirrors of gerrit, and other repos that are primary repos, and it's a bit unclear what's what. More importantly, when folks have repos that they've been running on GitHub already and want to move into the wikimedia project (rather than switch to gerrit), what's the procedure? I'm an admin/owner so I can manually import people's repos but I'm not sure whether I'm supposed to... :)
Honestly, I'm not entirely convinced that "mirror everything" is all that useful. It mostly results in a ton of unused repos cluttering up lists.
Not opposed to mirroring, but I'm wondering if we can be more selective.
-Chad
On 2016-04-25 19:01, Chad wrote:
Honestly, I'm not entirely convinced that "mirror everything" is all that useful. It mostly results in a ton of unused repos cluttering up lists.
I, for one, appreciate it. GitHub's interface is unfortunately a lot more convenient than any of the repository viewers we host ourselves. :( And fairly often I need to give somebody a link to a code snippet in one of our repos.
On Mon, Apr 25, 2016 at 7:33 PM, Bartosz Dziewoński matma.rex@gmail.com wrote:
I, for one, appreciate it. GitHub's interface is unfortunately a lot more convenient than any of the repository viewers we host ourselves. :( And fairly often I need to give somebody a link to a code snippet in one of our repos.
+1, it's very convenient for linking to code, or even just looking code up for something that's not locally installed. Gitblit can take several seconds per page view, and the Phabricator repo browser interface is one of the sadder components of that software.
If we truly care about repo discoverability on github, using Github Pages is probably a better option than trying to filter out unused ones (and still ending up with an unmanageably large list).
Hi!
On 2016-04-25 19:01, Chad wrote:
Honestly, I'm not entirely convinced that "mirror everything" is all that useful. It mostly results in a ton of unused repos cluttering up lists.
I, for one, appreciate it. GitHub's interface is unfortunately a lot more convenient than any of the repository viewers we host ourselves. :(
Same here. Especially if I need to browse code and/or to communicate to somebody about it - especially somebody outside MediaWiki community (which means probably not trained with tools like gerrit but most probably having some familiarity with github because who doesn't have it these days?) - I almost always use github. If we have something better, I'd use it - but right now neither gerrit, nor gitblt, nor phabricator's code browser are superior to what github offers.
Now, having something like wikimedia.github.io would be an excellent idea. If somebody would do the design, loading up repos list and displaying them with a nice structure - given that we actually have pretty structured names already, so we could start from that - should not be super-hard?
On Tue, Apr 26, 2016 at 5:24 PM, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
On 2016-04-25 19:01, Chad wrote:
Honestly, I'm not entirely convinced that "mirror everything" is all that useful. It mostly results in a ton of unused repos cluttering up lists.
I, for one, appreciate it. GitHub's interface is unfortunately a lot more convenient than any of the repository viewers we host ourselves. :(
Same here. Especially if I need to browse code and/or to communicate to somebody about it - especially somebody outside MediaWiki community (which means probably not trained with tools like gerrit but most probably having some familiarity with github because who doesn't have it these days?) - I almost always use github. If we have something better, I'd use it - but right now neither gerrit, nor gitblt, nor phabricator's code browser are superior to what github offers.
The Wikimedia GitHub project gives me two things in one place that I don't get elsewhere: * Find a repo based on some partial name I remember it probably has using the "Find a repository..." filtering at https://github.com/wikimedia/ * Search all the things at once using https://github.com/search?q=org:wikimedia
Now, having something like wikimedia.github.io would be an excellent idea. If somebody would do the design, loading up repos list and displaying them with a nice structure - given that we actually have pretty structured names already, so we could start from that - should not be super-hard?
github.io pages are static html from the gh-pages branch of some repository. They can use javascript and connect to the github api however, so it might be possible to build a useful browser interface that filtered based on the '-' separated name components. You could at least setup really broad categories like "Extensions", "Skins", "Operations", "...". The "other" bucket of that interface would still be pretty big however since we have been creating a lot of "un-namespaced" repos in the last year or two. That change started happening when the migration off of Gerrit was deemed an eventuality as the organization of repos in Gerrit is not very friendly to other hosting platforms.
Bryan
On 27 April 2016 at 01:15, Bryan Davis bd808@wikimedia.org wrote:
The Wikimedia GitHub project gives me two things in one place that I don't get elsewhere:
- Find a repo based on some partial name I remember it probably has
using the "Find a repository..." filtering at https://github.com/wikimedia/
https://gerrit.wikimedia.org/r/#/admin/projects/?filter= lets you do this
On Tue, Apr 26, 2016 at 5:20 PM Alex Monk krenair@gmail.com wrote:
On 27 April 2016 at 01:15, Bryan Davis bd808@wikimedia.org wrote:
The Wikimedia GitHub project gives me two things in one place that I don't get elsewhere:
- Find a repo based on some partial name I remember it probably has
using the "Find a repository..." filtering at https://github.com/wikimedia/
https://gerrit.wikimedia.org/r/#/admin/projects/?filter= lets you do this
As does https://phabricator.wikimedia.org/diffusion/query/advanced/
-Chad
On Tue, Apr 26, 2016 at 6:20 PM, Chad innocentkiller@gmail.com wrote:
On Tue, Apr 26, 2016 at 5:20 PM Alex Monk krenair@gmail.com wrote:
On 27 April 2016 at 01:15, Bryan Davis bd808@wikimedia.org wrote:
The Wikimedia GitHub project gives me two things in one place that I don't get elsewhere:
- Find a repo based on some partial name I remember it probably has
using the "Find a repository..." filtering at https://github.com/wikimedia/
https://gerrit.wikimedia.org/r/#/admin/projects/?filter= lets you do this
As does https://phabricator.wikimedia.org/diffusion/query/advanced/
Neither of which is as usable as the github in my personal opinion. Gerrit will lead you into a twisty maze of dead ends if you are trying to get to a repo browser. The diffusion search is several clicks deep in the UI and not type ahead filtering system. I could learn to live with the diffusion UX if I didn't have github as a more familiar and polished interface.
Bryan
On Tue, Apr 26, 2016 at 5:32 PM Bryan Davis bd808@wikimedia.org wrote:
On Tue, Apr 26, 2016 at 6:20 PM, Chad innocentkiller@gmail.com wrote:
On Tue, Apr 26, 2016 at 5:20 PM Alex Monk krenair@gmail.com wrote:
On 27 April 2016 at 01:15, Bryan Davis bd808@wikimedia.org wrote:
The Wikimedia GitHub project gives me two things in one place that I don't get elsewhere:
- Find a repo based on some partial name I remember it probably has
using the "Find a repository..." filtering at https://github.com/wikimedia/
https://gerrit.wikimedia.org/r/#/admin/projects/?filter= lets you do
this
As does https://phabricator.wikimedia.org/diffusion/query/advanced/
Neither of which is as usable as the github in my personal opinion. Gerrit will lead you into a twisty maze of dead ends if you are trying to get to a repo browser. The diffusion search is several clicks deep in the UI and not type ahead filtering system. I could learn to live with the diffusion UX if I didn't have github as a more familiar and polished interface.
Yeah, making those more discoverable from the front page of diffusion would be ideal. The default query isn't terribly useful without filtering.
-Chad
Hi!
repository. They can use javascript and connect to the github api however, so it might be possible to build a useful browser interface that filtered based on the '-' separated name components. You could at
Exactly! I've (ab)used github API for a bit, and found this (please correct me if the figures look weird, I trusted API passed through several quick-n-dirty scripts so mistakes possible): - we have 1780 repos - 900 of them are extensions, so putting extensions in separate group cuts it by half - 102 repos do not have "-" in their name, meaning those would be hardest to classify - 1016 have "mediawiki" prefix - 243 are "operations" - 76 are "analytics" - 48 are "labs" - 20 are "pywikibot" - 18 are "thumbor" - 18 are "integration" - 15 are "apps" - 10 are "phabricator"
The rest of the groups are smaller or don't look like a good unifying principle immediately. Thus, even with very naive approach we could categorize at least 1464 or over 80% of the repos, leaving just about 300 non-obvious ones, of which probably about 100-200 will end up in "other". Not ideal, but at least better than 1780 :) The list of names is at https://gist.github.com/smalyshev/dfa72c79d9f750058262b902f47c0130 if anybody wants to play with it and save time on extracting it :)
On Tue, Apr 26, 2016 at 4:24 PM, Stas Malyshev smalyshev@wikimedia.org wrote:
Now, having something like wikimedia.github.io would be an excellent idea. If somebody would do the design, loading up repos list and displaying them with a nice structure - given that we actually have pretty structured names already, so we could start from that - should not be super-hard?
I've pitched this idea before (to your team, no less :P). I'm not sure what came out of it. Maybe Julien or Moiz can say. My suggestion was to take http://twitter.github.io/ as a starting-point; the source code for that is Apache-licensed (https://github.com/twitter/twitter.github.com).
As mentioned before Github's interface for searching, browsing, reading and linking to code is great, so it would be great to keep the mirrors. An option could be moving them to another organization, like wikimedia-mirrors maybe.
Another very nice feature in the mirror repos IMO is the graphs tab. For example https://github.com/wikimedia/mediawiki/graphs/contributors
On Wed, Apr 27, 2016 at 4:09 AM, Ori Livneh ori@wikimedia.org wrote:
On Tue, Apr 26, 2016 at 4:24 PM, Stas Malyshev smalyshev@wikimedia.org wrote:
Now, having something like wikimedia.github.io would be an excellent idea. If somebody would do the design, loading up repos list and displaying them with a nice structure - given that we actually have pretty structured names already, so we could start from that - should not be super-hard?
I've pitched this idea before (to your team, no less :P). I'm not sure what came out of it. Maybe Julien or Moiz can say. My suggestion was to take http://twitter.github.io/ as a starting-point; the source code for that is Apache-licensed (https://github.com/twitter/twitter.github.com). _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Le 27/04/2016 04:09, Ori Livneh a écrit :
I've pitched this idea before (to your team, no less :P). I'm not sure what came out of it. Maybe Julien or Moiz can say. My suggestion was to take http://twitter.github.io/ as a starting-point; the source code for that is Apache-licensed (https://github.com/twitter/twitter.github.com).
Hello,
Since both Ori and Gergo suggested that I have forked their repo, updated a few things to match Wikimedia and bam done.
Page: https://wikimedia.github.io/ Repo: https://github.com/wikimedia/wikimedia.github.io/
Le 25/04/2016 19:01, Chad a écrit :
Honestly, I'm not entirely convinced that "mirror everything" is all that useful. It mostly results in a ton of unused repos cluttering up lists.
Not opposed to mirroring, but I'm wondering if we can be more selective.
Hello,
https://github.com/wikimedia/ is surely a mess. If GitHub had a way to flag a repo as being a mirror, they could be filtered out easily (maybe one can fill a feature request to them).
Another approach would be to split mirrors to a different organization such as 'wikimedia-mirror'. Admins would solely be people dealing with mirroriing. That will nicely cleanup the 'wikimedia' org.
On Mon, Apr 25, 2016 at 11:34 PM, Antoine Musso hashar+wmf@free.fr wrote:
https://github.com/wikimedia/ is surely a mess. If GitHub had a way to flag a repo as being a mirror, they could be filtered out easily (maybe one can fill a feature request to them).
Github's repo lists are in general not very helpful. If we filter out all the mirrors, the rest might be of a manageable size, but then most people are probably interested in one of the mirrored projects (MediaWiki, or some popular extension). If we really care, we should use Github Pages like other orgs do, e.g. http://microsoft.github.io/ http://yelp.github.io/ http://esri.github.io/
Guys, how ı can leave this e-mail list? I do not therewith :) Please send link for me.. thanks.
2016-04-26 15:03 GMT+03:00 Gergo Tisza gtisza@wikimedia.org:
On Mon, Apr 25, 2016 at 11:34 PM, Antoine Musso hashar+wmf@free.fr wrote:
https://github.com/wikimedia/ is surely a mess. If GitHub had a way to flag a repo as being a mirror, they could be filtered out easily (maybe one can fill a feature request to them).
Github's repo lists are in general not very helpful. If we filter out all the mirrors, the rest might be of a manageable size, but then most people are probably interested in one of the mirrored projects (MediaWiki, or some popular extension). If we really care, we should use Github Pages like other orgs do, e.g. http://microsoft.github.io/ http://yelp.github.io/ http://esri.github.io/ _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Tue, Apr 26, 2016 at 5:46 PM, Muhammed Tatlısu <tatlisumuhammed@gmail.com
wrote:
Guys, how ı can leave this e-mail list? I do not therewith :) Please send link for me.. thanks.
I see that your query was already answered in your separate thread at https://lists.wikimedia.org/pipermail/wikitech-l/2016-April/085387.html. Please do not kill the thread.
Thanks, Tony Thomas https://www.mediawiki.org/wiki/User:01tonythomas Home http://www.thomastony.me | Blog http://blog.thomastony.me | ThinkFOSS http://www.thinkfoss.com
Le 25/04/2016 17:19, Brion Vibber a écrit : <snip>
More importantly, when folks have repos that they've been running on GitHub already and want to move into the wikimedia project (rather than switch to gerrit), what's the procedure? I'm an admin/owner so I can manually import people's repos but I'm not sure whether I'm supposed to... :)
I don't think there is any. If you are an admin for the 'wikimedia' github organization, you could fork their repository under 'wikimedia/' namespace. That will leave some kind of audit trail and indicate the original repository.
The Jenkins community does that when they incorporate a third party plugin in their ecosystem. An example:
https://github.com/jenkinsci/git-plugin/
Got forked from https://github.com/magnayn/Hudson-GIT-plugin
And can then be community managed.
On Mon, Apr 25, 2016 at 8:19 AM, Brion Vibber bvibber@wikimedia.org wrote:
More importantly, when folks have repos that they've been running on GitHub already and want to move into the wikimedia project (rather than switch to gerrit), what's the procedure? I'm an admin/owner so I can manually import people's repos but I'm not sure whether I'm supposed to... :)
The method we have been using is via 'transfer ownership' in the original repo settings. I believe moving the repo to the wikimedia org requires owner permissions, so for repos owned by non-owners this might require two transfers: One to an owner, then from owner to the org.
wikitech-l@lists.wikimedia.org