>Browser bias: I was under the impression that Nav timing was only
> available for modern browsers, so that might affect a bit of a bias into the situation.
It is well supported for IE >8 . The notable exception are iOS devices.
>Browser cache: Browser cache is going to reduce any gains we might see due to reduced
>latency or increased bandwidth. It would be nice if we could compare the
>speed changes of cached/non-cached page/asset loading.
Actually this is not an issue if you can assume cache hits/misses are the same before and after deploying ulsfo. What you care about in this case are the network times more than anything, as Faidon's changes do not affect the rendering of the page once all assets are downloaded. Now, if his changes also have the effect to clear cache, then we need to compare a data before his changes to data after his changes once caches have warmed up.
If there is an impact for network times should be "measurable" via transport times (they should be lower). If these smaller transport times have an effect on user experience (which is an entirely different matter) should be measurable via DOMContentReady (should be faster). Whether there is a real impact might get muddled as time passes and new features are deployed that affect number of assets on the page and rendering times.
I would analyze desktop and mobile data separately.
Nuria