2011年3月5日星期六

Re: Monthly Gateway Timeout

Hello!

On Sat, Mar 05, 2011 at 06:18:00AM -0500, Marc Kramis wrote:

[...]

> log_format LOG '$remote_addr - $remote_user
> [$time_local] "$request" $status $body_bytes_sent "$http_referer"
> "$http_user_agent"';
> access_log /nginx/logs/$host.access.log LOG;

Just a side note: this, along with no root specified in server{}
blocks, makes your system vulnerable to inode exhaustion attack
(i.e. attacker may create arbitrary number of log files on your
system, eventually bringing it down).

[...]

> proxy_read_timeout 3600;

Uhm... Its really big one. It's unlikely that clients will wait
for so long for an answer.

[...]

> location /bar/ {
> proxy_pass http://10.10.10.1:8080/bar/;

With this configuration nginx will convert ip/port to binary form
once (while reading config), and will use it (binary) while doing
connect()'s. It is very unlikely that something bad will happen
in nginx at this point.

[...]

> The error log is full of the following error (only during the
> problematic hour):
>
> 2011/03/04 08:40:28 [error] 20062#0: *507995 upstream timed out (145:
> Connection timed out) while reading response header from upstream,
> client: ***IP***, server: ***SERVER***, request: "GET ***URL***
> HTTP/1.1", upstream: "***UPSTREAM***", host: "***HOST***", referrer:
> "***REFERER"

Words "while reading response header" in error_log suggest that
connection to upstream was established (and request was sent), but
upstream failed to generate an answer in time.

> I just realized that only during this hour, the firewall lists blocked
> outgoing traffic exactly to the client IPs of the error log at random
> ports, i.e., I assume that during this hour, nginx mistakenly sends the
> proxied request back to the client instead of the internal server.

With your proxy_read_timeout it's very likely that appropriate
firewall states for connections with clients were already expired,
and that's why your firewall complains about nginx trying to
reply "504 Gateway timeout" back to clients once it detects
timeout.

>From here it looks like it's your backend problem, not nginx. And
strange firwall complains you see are due to timeouts
misconfiguration (you have to configure timeouts on your firewall
at least as big as ones in nginx).

Maxim Dounin

_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx

没有评论:

发表评论