Joshua,
This is awesome. Now, let's make sure we are not running into throttling limits. Could we get an estimate of the number of requests we expect from the apps?
Given that we have about 400.000 daily uniques in IOS daily we would expect about 5 requests per second coming from the app IF traffic is widespread through the day and app is doing 1 request per day. If traffic concentrates in some hours request ratio might be higher
We would also like to double check that caching headers are respected in the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
Thanks,
Nuria
IF traffic is widespread through the day and app is doing 1 request per day. If traffic concentrates in some hours request ratio might be higher
It largely boils down to "it depends" on:
- Number of "Top read sections" that are in the user's feed on that device - Which sections they scroll to (we fetch lazily) - Whether the app is ever purged from memory (i.e. terminated by the OS)
The app will keep previous pageviews responses in memory, so as long as the app lives, it won't make another request until it reaches the next day, and attempts to fetch pageviews data for it.
We would also like to double check that caching headers are respected in
the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
We can look into it, but it might not really be necessary in practice since we'd only re-fetch data if the app was forcibly terminated by the user or OS, and the user scrolled to view older sections of the feed. IOW, persistenting previous responses to disk with cache control information will only save user data (and server bandwidth) in the off chance that our existing in-memory data is purged.
That said, iOS already does have mechanisms for respecting HTTP cache control headers. We just need to (finally) take advantage of them.
On Mon, Feb 22, 2016 at 11:55 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Joshua,
This is awesome. Now, let's make sure we are not running into throttling limits. Could we get an estimate of the number of requests we expect from the apps?
Given that we have about 400.000 daily uniques in IOS daily we would expect about 5 requests per second coming from the app IF traffic is widespread through the day and app is doing 1 request per day. If traffic concentrates in some hours request ratio might be higher
We would also like to double check that caching headers are respected in the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
Thanks,
Nuria
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Thanks, Brian, that should be fine. We're going to up the cache for all requests to 24 hours, so if you do start respecting the headers keep that in mind.
On Mon, Feb 22, 2016 at 6:43 PM, Brian Gerstle bgerstle@wikimedia.org wrote:
IF traffic is widespread through the day and app is doing 1 request per
day. If traffic concentrates in some hours request ratio might be higher
It largely boils down to "it depends" on:
- Number of "Top read sections" that are in the user's feed on that
device
- Which sections they scroll to (we fetch lazily)
- Whether the app is ever purged from memory (i.e. terminated by the
OS)
The app will keep previous pageviews responses in memory, so as long as the app lives, it won't make another request until it reaches the next day, and attempts to fetch pageviews data for it.
We would also like to double check that caching headers are respected in
the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
We can look into it, but it might not really be necessary in practice since we'd only re-fetch data if the app was forcibly terminated by the user or OS, and the user scrolled to view older sections of the feed. IOW, persistenting previous responses to disk with cache control information will only save user data (and server bandwidth) in the off chance that our existing in-memory data is purged.
That said, iOS already does have mechanisms for respecting HTTP cache control headers. We just need to (finally) take advantage of them.
On Mon, Feb 22, 2016 at 11:55 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Joshua,
This is awesome. Now, let's make sure we are not running into throttling limits. Could we get an estimate of the number of requests we expect from the apps?
Given that we have about 400.000 daily uniques in IOS daily we would expect about 5 requests per second coming from the app IF traffic is widespread through the day and app is doing 1 request per day. If traffic concentrates in some hours request ratio might be higher
We would also like to double check that caching headers are respected in the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
Thanks,
Nuria
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Analytics-internal mailing list Analytics-internal@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics-internal
Ok, also worth noting is that the pageview response expiration is different than the additional "action=query" response we're retrieving for each title in the pageview results. For example, if pageviews API response for 2016/02/22 expires 24hrs after retrieval, we won't hit pageviews again for 24hrs if the cache control information is respected. However, the "previews" we obtain via "action=query" might not be cached as long.
On Mon, Feb 22, 2016 at 6:55 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Thanks, Brian, that should be fine. We're going to up the cache for all requests to 24 hours, so if you do start respecting the headers keep that in mind.
On Mon, Feb 22, 2016 at 6:43 PM, Brian Gerstle bgerstle@wikimedia.org wrote:
IF traffic is widespread through the day and app is doing 1 request per
day. If traffic concentrates in some hours request ratio might be higher
It largely boils down to "it depends" on:
- Number of "Top read sections" that are in the user's feed on that
device
- Which sections they scroll to (we fetch lazily)
- Whether the app is ever purged from memory (i.e. terminated by the
OS)
The app will keep previous pageviews responses in memory, so as long as the app lives, it won't make another request until it reaches the next day, and attempts to fetch pageviews data for it.
We would also like to double check that caching headers are respected in
the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
We can look into it, but it might not really be necessary in practice since we'd only re-fetch data if the app was forcibly terminated by the user or OS, and the user scrolled to view older sections of the feed. IOW, persistenting previous responses to disk with cache control information will only save user data (and server bandwidth) in the off chance that our existing in-memory data is purged.
That said, iOS already does have mechanisms for respecting HTTP cache control headers. We just need to (finally) take advantage of them.
On Mon, Feb 22, 2016 at 11:55 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Joshua,
This is awesome. Now, let's make sure we are not running into throttling limits. Could we get an estimate of the number of requests we expect from the apps?
Given that we have about 400.000 daily uniques in IOS daily we would expect about 5 requests per second coming from the app IF traffic is widespread through the day and app is doing 1 request per day. If traffic concentrates in some hours request ratio might be higher
We would also like to double check that caching headers are respected in the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
Thanks,
Nuria
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Analytics-internal mailing list Analytics-internal@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics-internal
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
However, the "previews" we obtain via "action=query" might not be cached
as long. But these are not fetched from pageview API, are they?
On Tue, Feb 23, 2016 at 7:05 AM, Brian Gerstle bgerstle@wikimedia.org wrote:
Ok, also worth noting is that the pageview response expiration is different than the additional "action=query" response we're retrieving for each title in the pageview results. For example, if pageviews API response for 2016/02/22 expires 24hrs after retrieval, we won't hit pageviews again for 24hrs if the cache control information is respected. However, the "previews" we obtain via "action=query" might not be cached as long.
On Mon, Feb 22, 2016 at 6:55 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Thanks, Brian, that should be fine. We're going to up the cache for all requests to 24 hours, so if you do start respecting the headers keep that in mind.
On Mon, Feb 22, 2016 at 6:43 PM, Brian Gerstle bgerstle@wikimedia.org wrote:
IF traffic is widespread through the day and app is doing 1 request per
day. If traffic concentrates in some hours request ratio might be higher
It largely boils down to "it depends" on:
- Number of "Top read sections" that are in the user's feed on that
device
- Which sections they scroll to (we fetch lazily)
- Whether the app is ever purged from memory (i.e. terminated by the
OS)
The app will keep previous pageviews responses in memory, so as long as the app lives, it won't make another request until it reaches the next day, and attempts to fetch pageviews data for it.
We would also like to double check that caching headers are respected in
the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
We can look into it, but it might not really be necessary in practice since we'd only re-fetch data if the app was forcibly terminated by the user or OS, and the user scrolled to view older sections of the feed. IOW, persistenting previous responses to disk with cache control information will only save user data (and server bandwidth) in the off chance that our existing in-memory data is purged.
That said, iOS already does have mechanisms for respecting HTTP cache control headers. We just need to (finally) take advantage of them.
On Mon, Feb 22, 2016 at 11:55 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Joshua,
This is awesome. Now, let's make sure we are not running into throttling limits. Could we get an estimate of the number of requests we expect from the apps?
Given that we have about 400.000 daily uniques in IOS daily we would expect about 5 requests per second coming from the app IF traffic is widespread through the day and app is doing 1 request per day. If traffic concentrates in some hours request ratio might be higher
We would also like to double check that caching headers are respected in the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
Thanks,
Nuria
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Analytics-internal mailing list Analytics-internal@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics-internal
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Correct. Sorry I didn't mean to confuse the issue, just pointing out for clarity that pageview is not the only request we're making for this feature which should be cached intelligently.
On Tue, Feb 23, 2016 at 10:54 AM, Nuria Ruiz nuria@wikimedia.org wrote:
However, the "previews" we obtain via "action=query" might not be
cached as long. But these are not fetched from pageview API, are they?
On Tue, Feb 23, 2016 at 7:05 AM, Brian Gerstle bgerstle@wikimedia.org wrote:
Ok, also worth noting is that the pageview response expiration is different than the additional "action=query" response we're retrieving for each title in the pageview results. For example, if pageviews API response for 2016/02/22 expires 24hrs after retrieval, we won't hit pageviews again for 24hrs if the cache control information is respected. However, the "previews" we obtain via "action=query" might not be cached as long.
On Mon, Feb 22, 2016 at 6:55 PM, Dan Andreescu dandreescu@wikimedia.org wrote:
Thanks, Brian, that should be fine. We're going to up the cache for all requests to 24 hours, so if you do start respecting the headers keep that in mind.
On Mon, Feb 22, 2016 at 6:43 PM, Brian Gerstle bgerstle@wikimedia.org wrote:
IF traffic is widespread through the day and app is doing 1 request per
day. If traffic concentrates in some hours request ratio might be higher
It largely boils down to "it depends" on:
- Number of "Top read sections" that are in the user's feed on that
device
- Which sections they scroll to (we fetch lazily)
- Whether the app is ever purged from memory (i.e. terminated by
the OS)
The app will keep previous pageviews responses in memory, so as long as the app lives, it won't make another request until it reaches the next day, and attempts to fetch pageviews data for it.
We would also like to double check that caching headers are respected
in the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
We can look into it, but it might not really be necessary in practice since we'd only re-fetch data if the app was forcibly terminated by the user or OS, and the user scrolled to view older sections of the feed. IOW, persistenting previous responses to disk with cache control information will only save user data (and server bandwidth) in the off chance that our existing in-memory data is purged.
That said, iOS already does have mechanisms for respecting HTTP cache control headers. We just need to (finally) take advantage of them.
On Mon, Feb 22, 2016 at 11:55 AM, Nuria Ruiz nuria@wikimedia.org wrote:
Joshua,
This is awesome. Now, let's make sure we are not running into throttling limits. Could we get an estimate of the number of requests we expect from the apps?
Given that we have about 400.000 daily uniques in IOS daily we would expect about 5 requests per second coming from the app IF traffic is widespread through the day and app is doing 1 request per day. If traffic concentrates in some hours request ratio might be higher
We would also like to double check that caching headers are respected in the app and requests are not retried if they are mean to be cached. Can developers confirm this is the case?
Thanks,
Nuria
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Analytics-internal mailing list Analytics-internal@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics-internal
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
-- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l