I'm having some positive results in preliminary tests simply switching the cache mode on the WebView.
Getting rid of the urlcache bits which we've had problems with, and adding this into WikipediaActivity.java.onCreate():
this.appView.getSettings().setCacheMode(WebSettings.LOAD_CACHE_ELSE_NETWORK);
lets me go back to previously-visited pages without complaint after switching on airplane mode -- and very quickly!
However, any loading that *fails* (such as a page that hasn't been cached) triggers an application-fatal error, as far as I can tell because DroidGap assumes that iframes don't exist and the only thing that might fail to load is the entire application. :P This may just have to be hacked around by modifying the phonegap infrastructure. :P
Some potential issues with this approach: * not sure if/how to control the amount of space used for caching * (though you should be able to clear it manually from the standard Manage Applications panel on the phone) * presumably we should load from network first if we're online, so would need to swap this mode dynamically or else find a better behavior * not sure if any way to tell what's in cache ahead of time * of course since it's platform-dependent, similar behavior would need to be tweaked on other platforms
A couple other possibilities include:
1) Making an "HTTP caching proxy"-like layer that we slide under the ability to override how requests are made.
Again, this would be platform-specific and may need to be reimplemented in the backend for other platforms. Advantage: makes our JS-side code super easy as it would just try to load things and they'd show up. We can control the return results, so errors wouldn't have to trigger the fatal code as above.
2) Do platform-independent 'caching' by slurping the contents of pages within JavaScript after loading, and managing the re-loading from cache ourselves when offline
* Still needs the android fix for not dying when a connection fails. :P * may not be able to handle images this way * could explode with JS stuff :)
-- brion
On Fri, Dec 9, 2011 at 2:40 PM, Brion Vibber bvibber@wikimedia.org wrote:
However, any loading that *fails* (such as a page that hasn't been cached) triggers an application-fatal error, as far as I can tell because DroidGap assumes that iframes don't exist and the only thing that might fail to load is the entire application. :P This may just have to be hacked around by modifying the phonegap infrastructure. :P
Found this was easy to fix in WikipediaActivity by overriding onReceivedError to do nothing. We get the generic Android browser error page on fail instead, nice!
Work in progress: https://github.com/brion/Wikipedia/commits/cache-setcachemode
-- brion
On Fri, Dec 9, 2011 at 3:04 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Fri, Dec 9, 2011 at 2:40 PM, Brion Vibber bvibber@wikimedia.orgwrote:
However, any loading that *fails* (such as a page that hasn't been cached) triggers an application-fatal error, as far as I can tell because DroidGap assumes that iframes don't exist and the only thing that might fail to load is the entire application. :P This may just have to be hacked around by modifying the phonegap infrastructure. :P
Found this was easy to fix in WikipediaActivity by overriding onReceivedError to do nothing. We get the generic Android browser error page on fail instead, nice!
Work in progress: https://github.com/brion/Wikipedia/commits/cache-setcachemode
Using setCacheMode to set it to load from cache works nicely, but for only a limited amount of data. It seems that pages get pushed out of cache very quickly -- only a few megabytes seem to be stored and there's no obvious way to increase it.
To ensure that bookmarked or 'saved' pages are consistently available, either a homebrew loader cache with some smarts or something like the existing URLCache should help; but it'd need to be changed to handle images better etc.
I prefer the homebrew loader cache, since it allows standard HTML security separations to stay in effect (assuming PhoneGap doesn't disable them all, which it might considering that the file: pages can do anything to the iframe).
The error fix definitely needs to be kept so things don't esplode ridiculously, or else covered with an alternate error return.
-- brion
On Fri, Dec 9, 2011 at 4:15 PM, Brion Vibber bvibber@wikimedia.org wrote:
To ensure that bookmarked or 'saved' pages are consistently available, either a homebrew loader cache with some smarts or something like the existing URLCache should help; but it'd need to be changed to handle images better etc.
I prefer the homebrew loader cache, since it allows standard HTML security separations to stay in effect (assuming PhoneGap doesn't disable them all, which it might considering that the file: pages can do anything to the iframe).
Bad news: that (WebViewClient.shouldInterceptRequest) seems to only be available on Android 3 and later. Manually loading as with URLCache should work though. Sigh...
-- brion
On Fri, Dec 9, 2011 at 4:48 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Fri, Dec 9, 2011 at 4:15 PM, Brion Vibber bvibber@wikimedia.org wrote:
To ensure that bookmarked or 'saved' pages are consistently available, either a homebrew loader cache with some smarts or something like the existing URLCache should help; but it'd need to be changed to handle images better etc.
I prefer the homebrew loader cache, since it allows standard HTML security separations to stay in effect (assuming PhoneGap doesn't disable them all, which it might considering that the file: pages can do anything to the iframe).
Bad news: that (WebViewClient.shouldInterceptRequest) seems to only be available on Android 3 and later. Manually loading as with URLCache should work though. Sigh...
Sigh indeed. We don't have to go back too far but whatever we do has to be Android 2.2.x and above. I double checked with Amit and he confirmed that we shouldn't have to support anything older then that.
--tomasz
On Fri, Dec 9, 2011 at 4:48 PM, Brion Vibber bvibber@wikimedia.org wrote:
On Fri, Dec 9, 2011 at 4:15 PM, Brion Vibber bvibber@wikimedia.orgwrote:
To ensure that bookmarked or 'saved' pages are consistently available, either a homebrew loader cache with some smarts or something like the existing URLCache should help; but it'd need to be changed to handle images better etc.
I prefer the homebrew loader cache, since it allows standard HTML security separations to stay in effect (assuming PhoneGap doesn't disable them all, which it might considering that the file: pages can do anything to the iframe).
Bad news: that (WebViewClient.shouldInterceptRequest) seems to only be available on Android 3 and later. Manually loading as with URLCache should work though. Sigh...
Good news: good results so far using URLCache plugin to save just the bookmarked pages.
Pushed to same branch: https://github.com/brion/Wikipedia/commits/cache-setcachemode
Likely next steps on the cache front: * make sure things work consistently ;) * make bookmarked pages update in UrlCache when we load them again * make bookmarks/saved pages list searchable when offline * highlight bookmarked/saved pages in search results, history * consider if renaming 'bookmarks' to 'saved pages' would make more sense * make UrlCache usage behave nicer for bookmark saves -- there are flashes as images and styles load, etc, probably due to the way things get loaded in separate pieces (switch to XHR and preload the styles?) * (Java side?) make sure we understand the cache size limits on UrlCache * ensure that UrlCache is safe-ish * (Java side) see if we can switch in the 'load from cache first' mode only when offline
And related: * general bookmarks cleanup & testing * general history view cleanup & testing
-- brion