Is AWS (Amazon Web Services) being used by Wikimedia directly; US Foundation, or by other affiliates?
While trawling around for AWS related tasks, I saw on T74501 that Sage Ross' team was using AWS late last year, perhaps only temporarily due to the bug. https://phabricator.wikimedia.org/T74501 (I assume from the date the task was raised that it was a team at Wiki Education Foundation).
All use of Travis-CI is indirectly using AWS. Are there other indirect uses?
Is it still in use at all in a direct manner, i.e. using our own account?
-- John Vandenberg
<quote name="John Mark Vandenberg" date="2015-10-07" time="16:41:01 +1100">
Is AWS (Amazon Web Services) being used by Wikimedia directly; US Foundation, or by other affiliates?
Not for anything user facing. I *think* the Performance team uses AWS for a testing hosting (it runs Windows, I think, or I'm confusing it with another team).
All use of Travis-CI is indirectly using AWS. Are there other indirect uses?
Travis-CI is only currently used by a very small number of repositories, luckily.
Greg
Hi,
Why do you even care? Is this question directed to foundation only or community developers as well? "Used by wikimedia" is very broad term.
On Wed, Oct 7, 2015 at 4:42 PM, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="John Mark Vandenberg" date="2015-10-07" time="16:41:01 +1100"> > Is AWS (Amazon Web Services) being used by Wikimedia directly; US > Foundation, or by other affiliates?
Not for anything user facing. I *think* the Performance team uses AWS for a testing hosting (it runs Windows, I think, or I'm confusing it with another team).
All use of Travis-CI is indirectly using AWS. Are there other indirect uses?
Travis-CI is only currently used by a very small number of repositories, luckily.
Greg
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
<quote name="Petr Bena" date="2015-10-07" time="16:54:47 +0200">
Why do you even care? Is this question directed to foundation only or community developers as well? "Used by wikimedia" is very broad term.
I interpreted "Wikimedia" as WMF, for better or worse, in my email, just so it's clear.
Sorry, I mentioned the question for OP of this thread.
On Wed, Oct 7, 2015 at 5:06 PM, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="Petr Bena" date="2015-10-07" time="16:54:47 +0200"> > Why do you even care? Is this question directed to foundation only or > community developers as well? "Used by wikimedia" is very broad term.
I interpreted "Wikimedia" as WMF, for better or worse, in my email, just so it's clear.
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Oct 8, 2015 at 1:54 AM, Petr Bena benapetr@gmail.com wrote:
Hi,
Why do you even care? Is this question directed to foundation only or community developers as well? "Used by wikimedia" is very broad term.
Pywikibot uses Travis-CI, and I was looking at their S3 artefacts integration for storage of coverage data. It turns out to be quite a bit more expensive than I had assumed. https://phabricator.wikimedia.org/T74863#1710830
Also of interest was that given some people are dead against allowing Travis-CI to do builds for free, it appears parts of our community are using AWS directly instead of Labs, which is not free.
I used the broad term "Wikimedia", as it is the broader community that could be using Labs. It seems to be worth understanding why they are choosing to not use Labs, and how much it is costing "us".
-- John Vandenberg
As to why:
AWS is more flexible and more reliable than wikimedia-labs, other than that it's basically the same. If they really need to use AWS it's probably because they don't like the restrictions that come with wikimedia labs (stuff must be open source, comply with policies, only ubuntu or debian, no proprietary software, hard-to-get public IP, long waiting time for stuff that you can't do yourself (request new project), no IPv6 and many others)
On Thu, Oct 8, 2015 at 6:08 AM, John Mark Vandenberg jayvdb@gmail.com wrote:
On Thu, Oct 8, 2015 at 1:54 AM, Petr Bena benapetr@gmail.com wrote:
Hi,
Why do you even care? Is this question directed to foundation only or community developers as well? "Used by wikimedia" is very broad term.
Pywikibot uses Travis-CI, and I was looking at their S3 artefacts integration for storage of coverage data. It turns out to be quite a bit more expensive than I had assumed. https://phabricator.wikimedia.org/T74863#1710830
Also of interest was that given some people are dead against allowing Travis-CI to do builds for free, it appears parts of our community are using AWS directly instead of Labs, which is not free.
I used the broad term "Wikimedia", as it is the broader community that could be using Labs. It seems to be worth understanding why they are choosing to not use Labs, and how much it is costing "us".
-- John Vandenberg
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Oct 8, 2015 at 9:13 AM, Petr Bena benapetr@gmail.com wrote:
As to why:
AWS is more flexible and more reliable than wikimedia-labs, other than that it's basically the same. If they really need to use AWS it's probably because they don't like the restrictions that come with wikimedia labs (stuff must be open source, comply with policies, only ubuntu or debian, no proprietary software, hard-to-get public IP, long waiting time for stuff that you can't do yourself (request new project), no IPv6 and many others)
Assuming you don't use NFS, Labs is likely as reliable or possibly more reliable than EC2, since effort is put into saving instances when hardware is having issues (which AWS does not do). Project creation is usually pretty fast in Labs. When you create a new account in AWS, you have to get approved for most of the features before you can use them, so it's not quite as quick as you're making out. Getting public IPs in Labs is restricted, but it's also the most restricted thing in AWS too. For the most part it's unnecessary to get/use public IPs. In AWS basically everything goes in through ELBs and in Labs there's an ELB equivalent.
As for the rest of the restrictions, yep. Those totally make sense and fit with the intent of Labs.
I think some of the benefits of Labs are being ignored here. You may be limited to ubuntu/debian, but you also create an instance that's pre-configured, that you (and any other project member) can immediately SSH into. DNS is handled for you, load balancers are easy to use, cross-project access is way easier (no need to manage IAM and VPCs), and networking is pre-configured (which is actually non-trivial in AWS). Most importantly it's free for end-users.
- Ryan
Hi,
Travis-CI is only currently used by a very small number of repositories, luckily.
I'm not sure I understand the underlying connotation in your comment but some non-core developers are happy about Travis-CI [0] and how it allows them to run a test environment independent from WMF (and/or WMF Jenkins).
[0] https://cloud.githubusercontent.com/assets/1245473/10268587/d1b30c78-6abd-11...
Cheers
On 10/7/15, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="John Mark Vandenberg" date="2015-10-07" time="16:41:01 +1100"> > Is AWS (Amazon Web Services) being used by Wikimedia directly; US > Foundation, or by other affiliates?
Not for anything user facing. I *think* the Performance team uses AWS for a testing hosting (it runs Windows, I think, or I'm confusing it with another team).
All use of Travis-CI is indirectly using AWS. Are there other indirect uses?
Travis-CI is only currently used by a very small number of repositories, luckily.
Greg
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
<quote name="James HK" date="2015-10-07" time="17:03:18 +0200">
Hi,
Travis-CI is only currently used by a very small number of repositories, luckily.
I'm not sure I understand the underlying connotation in your comment but some non-core developers are happy about Travis-CI [0] and how it allows them to run a test environment independent from WMF (and/or WMF Jenkins).
As in, they would not be experiencing any outage because they are not using Travis. The very topic of this thread.
Greg
On Wed, Oct 7, 2015 at 11:27 AM, Greg Grossmeier greg@wikimedia.org wrote:
As in, they would not be experiencing any outage because they are not using Travis. The very topic of this thread.
Nowhere in this thread is the uptime of Travis or AWS mentioned as far as I see, so it's understandable to be confused as to why we're "lucky" to not be using Travis.
*-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2016 Major in Computer Science
<quote name="Tyler Romeo" date="2015-10-07" time="11:41:41 -0400">
On Wed, Oct 7, 2015 at 11:27 AM, Greg Grossmeier greg@wikimedia.org wrote:
As in, they would not be experiencing any outage because they are not using Travis. The very topic of this thread.
Nowhere in this thread is the uptime of Travis or AWS mentioned as far as I see, so it's understandable to be confused as to why we're "lucky" to not be using Travis.
Hah! I thought the thread subject was "AWS outage" not "usage".
Sorry, my bad :)
On Tue, Oct 6, 2015 at 10:41 PM, John Mark Vandenberg jayvdb@gmail.com wrote:
Is AWS (Amazon Web Services) being used by Wikimedia directly; US Foundation, or by other affiliates?
While trawling around for AWS related tasks, I saw on T74501 that Sage Ross' team was using AWS late last year, perhaps only temporarily due to the bug. https://phabricator.wikimedia.org/T74501 (I assume from the date the task was raised that it was a team at Wiki Education Foundation).
Your assumptions here are correct: Wiki Education Foundation initially tried using Heroku (AWS) to host an OAuth app, and then we switched to a more traditional hosting provider that does not run on AWS.
-Sage
wikitech-l@lists.wikimedia.org