[QA] CI web migrated behind Varnish

[Antoine Musso]

Hello,

The continuous integration server hosts the CI website
integration.wikimedia.org as well as  doc.wikimedia.org. We also have
permanent redirects for mediawiki.org entries.

We served both HTTP and HTTPS directly from gallium.  This has been
slightly changed and we are no pointing to varnish caches which connect
to gallium over HTTP (no more SSL on gallium box).

The new layout is:

{doc,integration}.mediawiki.org points to the production text varnish.
The query hit the application servers Apaches which issues redirects to
the {doc,integration}.wikimedia.org entries.

{doc,integration}.wikimedia.org is pointing to the misc varnishes
caches. They will terminate SSL connection if needed then make HTTP
request to gallium.


The end result is that we no more have to take of SSL  / certs on
gallium and we benefit from varnish caching.

We might encounter issues with Varnish caching pages.  The way to fix it
is to serve appropriate Cache-Control header. I did such a fix to Zuul
web service to ensure it is serving uncacheable pages. Upstream fix is:
https://review.openstack.org/#/c/66583/

cheers,

-- 
Antoine "hashar" Musso