Surprisingly, it was very easy to do that. Just a quick look through the WebView/WebViewClient docs and you've quickly got your answer.
Set up your WebView with a new WebViewClient by calling:
webView.setWebViewClient(new NoErrorWebViewClient());   And here is the subclassed WebViewClient which handles errors:
 class NoErrorWebViewClient extends WebViewClient {
   @Override
   public void onReceivedError(WebView view, int errorCode, String description, String failingUrl) {
     Log.e(String.valueOf(errorCode), description);
 
     // API level 5: WebViewClient.ERROR_HOST_LOOKUP
     if (errorCode == -2) {
       String summary = "<html><body style='background: black;'><p style='color: red;'>Unable to load information. Please check if your network connection is working properly or try again later.</p></body></html>";       view.loadData(summary, "text/html", null);
       return;
     }
 
     // Default behaviour
     super.onReceivedError(view, errorCode, description, failingUrl);
   }
 }    And the result is...
Just a reminder that within the error handler you can also load a local resource/asset rather than building the HTML string.
Update 24/01/2013: Moved super.onReceivedError() to the end of the call to prevent flickering.
