[alibaba/tengine]ngx_http_upstream_check_module导致recv() failed (104: Connection reset by peer)

upstream配置：check interval=10000 rise=2 fall=2 timeout=5000 type=tcp; nginx错误日志（部分）： 2017/06/16 14:28:54 [error] 5626#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:29:04 [error] 5624#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:29:14 [error] 5630#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:29:24 [error] 5619#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:29:34 [error] 5628#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:29:44 [error] 5629#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:29:54 [error] 5618#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:30:04 [error] 5621#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:30:14 [error] 5616#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:30:24 [error] 5623#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:30:34 [error] 5620#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:30:44 [error] 5631#0: recv() failed (104: Connection reset by peer) 2017/06/16 14:30:54 [error] 5627#0: recv() failed (104: Connection reset by peer)

nginx版本1.10.3 patch版本1.9.2+

mjwtc0722

这个日志显示读后端数据时，后端的连接异常断开了（发了TCP RST包给你的nginx），具体可以再抓包确认下

tcpdump -i any -s0 -A port <port>

chobits

首先感谢chobits的提示，根据抓包的结果分析，ngx_http_upstream_check_module这个模块，在使用tcp检测后端状态时，只进行了TCP的三次握手，没有主动断开这个连接，而是等待服务端来断开。当后端是nginx或者tomcat时（linux上），超时后后端会发fin包关闭这个连接。这个错误日志recv() failed (104: Connection reset by peer)是在后端为IIS的情况下抛出的，抓包发现IIS并不会发fin包来断开链接，而是在超时后发RST包重置连接，所以导致了这个问题。从这个问题也反应出ngx_http_upstream_check_module这个模块还是需要完善下检测机制的，如果是在检测后端状态后主动关闭这个连接，应该就不会出现connect reset这个问题

mjwtc0722

通过修改源代码已经解决了该问题 static ngx_check_conf_t ngx_check_types[] = { { NGX_HTTP_CHECK_TCP, ngx_string("tcp"), ngx_null_string, 0, ngx_http_upstream_check_peek_handler, ngx_http_upstream_check_peek_handler, NULL, NULL, NULL, 0, 1 }, 将最后一行的1改为0即可，根据数据结构分析可得知，这个1代表启用keepalived，所以客户端才不会主动断开连接，因为这是tcp的端口连通性检查，不需要keepalived，将其改为0禁止keepalived即可修改之后的代码如下： static ngx_check_conf_t ngx_check_types[] = { { NGX_HTTP_CHECK_TCP, ngx_string("tcp"), ngx_null_string, 0, ngx_http_upstream_check_peek_handler, ngx_http_upstream_check_peek_handler, NULL, NULL, NULL, 0, 0 },

mjwtc0722

666

chenzhangfeng

?，完美的解决了我的问题，感谢

weifan01

[alibaba/tengine]ngx_http_upstream_check_module导致recv() failed (104: Connection reset by peer)

回答