On Thu, Mar 03, 2011 at 05:23:45AM -0500, Alexander Zuban wrote:
> We have quite busy phpbb based forum. I found that it is quite heavy
> loaded with bots(Google, AdSense, Yahoo. It seems that AdSense bot
> duplicate every user's request). I have added confuguration to cache all
> dynamic requests and return them to bots only(see config below). But I
> faced strange issue, when page returned from cache - nginx doesn't pass
> any headers back.
> So questions are:
> * Are there anything bad that response considered as http 0.9?
> * Is it possible configure it to return headers in case when page was
> returned from cache so response will be considered as 1.1?
> * It seems that cache refreshed evertime when page was hit by user(not
> bot). Is it possible to configure it not to refresh it till it is
> valid(72h in config). To not produce not necessary IO.
> proxy_cache_bypass $crawlernocache;
You are using proxy_cache_bypass without identical proxy_no_cache,
it's known to have problems (and that's why you see http 0.9
replies). You have to write something like
instead. This way a) http 0.9 problem will be fixed and b) normal
users hits won't touch cache at all.