Nginx 请求处理拾遗

原创
2018/01/08 11:18
阅读数 188

Nginx配置指令(directives)的执行顺序和处理阶段(phase)绑定,而不是配置文件中先后顺序决定。当你了解大部分的指令执行的阶段后,最佳实践是将配置文件中配置指令的顺序调整为处理阶段的顺序。

处理阶段总览

Nginx 处理请求的过程一共划分为 11 个阶段,按照执行顺序依次是: post-read、server-rewrite、find-config、rewrite、post-rewrite、 preaccess、access、post-access、try-files、content、log.具体每个阶段的主要工作可以查看

nginx debug log反映出指令执行的阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 http process request line
2018/01/05 10:50:35 [debug] 14057#0: *203 http request line: "GET / HTTP/1.1"
2018/01/05 10:50:35 [debug] 14057#0: *203 http uri: "/"
2018/01/05 10:50:35 [debug] 14057#0: *203 http args: ""
2018/01/05 10:50:35 [debug] 14057#0: *203 http exten: ""
2018/01/05 10:50:35 [debug] 14057#0: *203 posix_memalign: 0000000000914B90:4096 [@16](https://my.oschina.net/u/2771800)
2018/01/05 10:50:35 [debug] 14057#0: *203 http process request header line
2018/01/05 10:50:35 [debug] 14057#0: *203 http header: "User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"
2018/01/05 10:50:35 [debug] 14057#0: *203 http header: "Host: 172.19.23.208:8087"
2018/01/05 10:50:35 [debug] 14057#0: *203 http header: "Accept: */*"
2018/01/05 10:50:35 [debug] 14057#0: *203 http header done
post-read阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 event timer del: 4: 1515120695127
2018/01/05 10:50:35 [debug] 14057#0: *203 rewrite phase: 0
2018/01/05 10:50:35 [debug] 14057#0: *203 rewrite phase: 1
2018/01/05 10:50:35 [debug] 14057#0: *203 http script value: "dafa"
2018/01/05 10:50:35 [debug] 14057#0: *203 http script set $a
server-rewrite阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 test location: ~ "/waf_admin/templates/(css|js)"
2018/01/05 10:50:35 [debug] 14057#0: *203 test location: ~ "/"
2018/01/05 10:50:35 [debug] 14057#0: *203 using configuration "/"
2018/01/05 10:50:35 [debug] 14057#0: *203 http cl:-1 max:4194304
find-config阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 rewrite phase: 3
2018/01/05 10:50:35 [debug] 14057#0: *203 rewrite phase: 4
2018/01/05 10:50:35 [debug] 14057#0: *203 http script regex: "(.*)"
2018/01/05 10:50:35 [notice] 14057#0: *203 "(.*)" matches "/", client: 172.19.23.208, server: 172.19.23.208, request: "GET / HTTP/1.1", host: "172.19.23.208:8087"
2018/01/05 10:50:35 [debug] 14057#0: *203 http script copy: "/waf_admin/templates/css/a.html"
2018/01/05 10:50:35 [debug] 14057#0: *203 http script regex end
2018/01/05 10:50:35 [notice] 14057#0: *203 rewritten data: "/waf_admin/templates/css/a.html", args: "", client: 172.19.23.208, server: 172.19.23.208, request: "GET / HTTP/1.1", host: "172.19.23.208:8087"
rewrite阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 post rewrite phase: 5
post-rewrite阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 uri changes: 11
2018/01/05 10:50:35 [debug] 14057#0: *203 test location: ~ "/waf_admin/templates/(css|js)"
2018/01/05 10:50:35 [debug] 14057#0: *203 using configuration "/waf_admin/templates/(css|js)"
2018/01/05 10:50:35 [debug] 14057#0: *203 http cl:-1 max:4194304
find-config阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 rewrite phase: 3
2018/01/05 10:50:35 [debug] 14057#0: *203 rewrite phase: 4
rewrite阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 post rewrite phase: 5
post-rewrite阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 generic phase: 6
2018/01/05 10:50:35 [debug] 14057#0: *203 generic phase: 7

2018/01/05 10:50:35 [debug] 14057#0: *203 access phase: 8
2018/01/05 10:50:35 [debug] 14057#0: *203 access phase: 9
access阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 post access phase: 10
post-access阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 try files phase: 11
try-files阶段

2018/01/05 10:50:35 [debug] 14057#0: *203 content phase: 12
2018/01/05 10:50:35 [debug] 14057#0: *203 content phase: 13
2018/01/05 10:50:35 [debug] 14057#0: *203 content phase: 14
content阶段 ngx_index模块-->ngx_autoindex模块-->ngx_static模块


2018/01/05 10:50:35 [debug] 14057#0: *203 http filename: "/opt/openresty/nginx/conf/waf_admin/templates/css/a.html"
2018/01/05 10:50:35 [debug] 14057#0: *203 add cleanup: 00000000008A3E68
2018/01/05 10:50:35 [error] 14057#0: *203 open() "/opt/openresty/nginx/conf/waf_admin/templates/css/a.html" failed (2: No such file or directory), client: 172.19.23.208, server: 172.19.23.208, request: "GET / HTTP/1.1", host: "172.19.23.208:8087"
2018/01/05 10:50:35 [debug] 14057#0: *203 http finalize request: 404, "/waf_admin/templates/css/a.html?" a:1, c:1
2018/01/05 10:50:35 [debug] 14057#0: *203 http special response: 404, "/waf_admin/templates/css/a.html?"
2018/01/05 10:50:35 [debug] 14057#0: *203 http set discard body
2018/01/05 10:50:35 [debug] 14057#0: *203 lua capture header filter, uri "/waf_admin/templates/css/a.html"
2018/01/05 10:50:35 [debug] 14057#0: *203 HTTP/1.1 404 Not Found
Server: openresty/1.9.7.2
Date: Fri, 05 Jan 2018 02:50:35 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding

关于重写

rewrite指令

Syntax:	rewrite regex replacement [flag];
Default:	—
Context:	server, location, if

rewrite(ngx_http_rewrite_module)能够rewritten当前请求,分为内部rewrite和外部rewrite,内部rewrite浏览器用户不会看到访问链接发生的改变;而外部rewrite则会使浏览器url发生变化,并伴随着状态码的变化。

rewrite指令有四种flag,分别为last、break、redirect、permanent,前两种为内部rewrite,后两种为外部。值得注意的是,当不使用任何flag时默认为last。

  • 外部rewrite返回的状态码不同,permanent(永久301),redirect(临时302)在seo中起关键作用。

  • 内部rewrite,last重写url后请求阶段转到find-config阶段,即进行location再匹配。break重写rul后结束rewrite阶段,进行下一阶段(post-rewrite)


server {
    listen 8087;
    server_name 172.19.23.208;
    default_type text/html;

    set $a 'dafa';
    location ~ /waf_admin/templates/(css|js) {
        root  /opt/openresty/nginx/conf/;
    }

    location ~ / {
        rewrite "(.*)" /waf_admin/templates/css/a.html break; # rewrite阶段,优先级高于index
        index index.html index.html1;# index指令属于ngx_index模块,运行在content阶段
    }
}

debug log

2018/01/05 10:53:03 [debug] 14155#0: *204 rewrite phase: 0
2018/01/05 10:53:03 [debug] 14155#0: *204 rewrite phase: 1
2018/01/05 10:53:03 [debug] 14155#0: *204 http script value: "dafa"
2018/01/05 10:53:03 [debug] 14155#0: *204 http script set $a
2018/01/05 10:53:03 [debug] 14155#0: *204 test location: ~ "/waf_admin/templates/(css|js)"
2018/01/05 10:53:03 [debug] 14155#0: *204 test location: ~ "/"
2018/01/05 10:53:03 [debug] 14155#0: *204 using configuration "/"
2018/01/05 10:53:03 [debug] 14155#0: *204 http cl:-1 max:4194304
2018/01/05 10:53:03 [debug] 14155#0: *204 rewrite phase: 3
2018/01/05 10:53:03 [debug] 14155#0: *204 rewrite phase: 4
2018/01/05 10:53:03 [debug] 14155#0: *204 http script regex: "(.*)"
2018/01/05 10:53:03 [notice] 14155#0: *204 "(.*)" matches "/", client: 172.19.23.208, server: 172.19.23.208, request: "GET / HTTP/1.1", host: "172.19.23.208:8087"
2018/01/05 10:53:03 [debug] 14155#0: *204 http script copy: "/waf_admin/templates/css/a.html"
2018/01/05 10:53:03 [debug] 14155#0: *204 http script regex end
2018/01/05 10:53:03 [notice] 14155#0: *204 rewritten data: "/waf_admin/templates/css/a.html", args: "", client: 172.19.23.208, server: 172.19.23.208, request: "GET / HTTP/1.1", host: "172.19.23.208:8087"
2018/01/05 10:53:03 [debug] 14155#0: *204 post rewrite phase: 5
2018/01/05 10:53:03 [debug] 14155#0: *204 generic phase: 6
2018/01/05 10:53:03 [debug] 14155#0: *204 generic phase: 7
2018/01/05 10:53:03 [debug] 14155#0: *204 access phase: 8
2018/01/05 10:53:03 [debug] 14155#0: *204 access phase: 9
2018/01/05 10:53:03 [debug] 14155#0: *204 post access phase: 10
2018/01/05 10:53:03 [debug] 14155#0: *204 try files phase: 11
2018/01/05 10:53:03 [debug] 14155#0: *204 content phase: 12
2018/01/05 10:53:03 [debug] 14155#0: *204 content phase: 13
2018/01/05 10:53:03 [debug] 14155#0: *204 content phase: 14
2018/01/05 10:53:03 [debug] 14155#0: *204 http filename: "/opt/openresty//nginx/html/waf_admin/templates/css/a.html"

index指令

index(ngx_http_index_module)官方文档

Syntax:	index file ...;
Default:	
index index.html;
Context:	http, server, location

It should be noted that using an index file causes an internal redirect, and the request can be processed in a different location. For example, with the following configuration

有如下nginx配置文件

server {
    listen 8087;
    server_name 172.19.23.208;
    default_type text/html;

    set $a 'dafa';
    location ~ /waf_admin/templates/(css|js) {
        root  /opt/openresty/nginx/conf/;
    }

    location ~ / {
        index index.html index.html1;
    }
}

[root@slave2 conf]# curl -i  http://172.19.23.208:8087/

debug log

2018/01/05 11:52:08 [debug] 16838#0: *205 http process request line
2018/01/05 11:52:08 [debug] 16838#0: *205 http request line: "GET / HTTP/1.1"
2018/01/05 11:52:08 [debug] 16838#0: *205 http uri: "/"
2018/01/05 11:52:08 [debug] 16838#0: *205 http args: ""
2018/01/05 11:52:08 [debug] 16838#0: *205 http exten: ""
2018/01/05 11:52:08 [debug] 16838#0: *205 posix_memalign: 00000000009D4A60:4096 [@16](https://my.oschina.net/u/2771800)
2018/01/05 11:52:08 [debug] 16838#0: *205 http process request header line
2018/01/05 11:52:08 [debug] 16838#0: *205 http header: "User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"
2018/01/05 11:52:08 [debug] 16838#0: *205 http header: "Host: 172.19.23.208:8087"
2018/01/05 11:52:08 [debug] 16838#0: *205 http header: "Accept: */*"
2018/01/05 11:52:08 [debug] 16838#0: *205 http header done
2018/01/05 11:52:08 [debug] 16838#0: *205 event timer del: 3: 1515124388853
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 0
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 1
2018/01/05 11:52:08 [debug] 16838#0: *205 http script value: "dafa"
2018/01/05 11:52:08 [debug] 16838#0: *205 http script set $a
2018/01/05 11:52:08 [debug] 16838#0: *205 test location: ~ "/waf_admin/templates/(css|js)"
2018/01/05 11:52:08 [debug] 16838#0: *205 test location: ~ "/"
2018/01/05 11:52:08 [debug] 16838#0: *205 using configuration "/"
2018/01/05 11:52:08 [debug] 16838#0: *205 http cl:-1 max:4194304
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 3
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 4
2018/01/05 11:52:08 [debug] 16838#0: *205 post rewrite phase: 5
2018/01/05 11:52:08 [debug] 16838#0: *205 generic phase: 6
2018/01/05 11:52:08 [debug] 16838#0: *205 generic phase: 7
2018/01/05 11:52:08 [debug] 16838#0: *205 access phase: 8
2018/01/05 11:52:08 [debug] 16838#0: *205 access phase: 9
2018/01/05 11:52:08 [debug] 16838#0: *205 post access phase: 10
2018/01/05 11:52:08 [debug] 16838#0: *205 try files phase: 11
2018/01/05 11:52:08 [debug] 16838#0: *205 content phase: 12


2018/01/05 11:52:08 [debug] 16838#0: *205 open index "/opt/openresty//nginx/html/index.html"
2018/01/05 11:52:08 [debug] 16838#0: *205 stat() "/opt/openresty//nginx/html/index.html" failed (2: No such file or directory)
ngx_index模块 check index.html	

2018/01/05 11:52:08 [debug] 16838#0: *205 http index check dir: "/opt/openresty//nginx/html"
2018/01/05 11:52:08 [debug] 16838#0: *205 open index "/opt/openresty//nginx/html/index.html1"
ngx_index模块 check index.html1


2018/01/05 11:52:08 [debug] 16838#0: *205 internal redirect: "/index.html1?"
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 0
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 1
2018/01/05 11:52:08 [debug] 16838#0: *205 http script value: "dafa"
2018/01/05 11:52:08 [debug] 16838#0: *205 http script set $a
2018/01/05 11:52:08 [debug] 16838#0: *205 test location: ~ "/waf_admin/templates/(css|js)"
2018/01/05 11:52:08 [debug] 16838#0: *205 test location: ~ "/"
2018/01/05 11:52:08 [debug] 16838#0: *205 using configuration "/"
2018/01/05 11:52:08 [debug] 16838#0: *205 http cl:-1 max:4194304
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 3
2018/01/05 11:52:08 [debug] 16838#0: *205 rewrite phase: 4
2018/01/05 11:52:08 [debug] 16838#0: *205 post rewrite phase: 5
2018/01/05 11:52:08 [debug] 16838#0: *205 generic phase: 6
2018/01/05 11:52:08 [debug] 16838#0: *205 generic phase: 7
2018/01/05 11:52:08 [debug] 16838#0: *205 access phase: 8
2018/01/05 11:52:08 [debug] 16838#0: *205 access phase: 9
2018/01/05 11:52:08 [debug] 16838#0: *205 post access phase: 10
2018/01/05 11:52:08 [debug] 16838#0: *205 try files phase: 11
2018/01/05 11:52:08 [debug] 16838#0: *205 content phase: 12
2018/01/05 11:52:08 [debug] 16838#0: *205 content phase: 13
2018/01/05 11:52:08 [debug] 16838#0: *205 content phase: 14
2018/01/05 11:52:08 [debug] 16838#0: *205 http filename: "/opt/openresty//nginx/html/index.html1"

ngx_indexngx_autoindex 模块都只会作用于那些 URI 以 / 结尾的请求,例如请求 GET /cats/,而对于不以 / 结尾的请求则会直接忽略,同时把处理权移交给 content 阶段的下一个模块。而 ngx_static 模块则刚好相反,直接忽略那些 URI 以 / 结尾的请求。

这里注意到url以/结尾。匹配location 最后走到content阶段,根据content阶段 ngx_index模块-->ngx_autoindex模块-->ngx_static模块的顺序。ngx_index的index依次查找指令附带的参数是否存在文件,如果存在就发起internal redirect,否则进行下一个参数check。查找位置为root 指令提供的,由内向外最近的root指令起作用。如果没有配置root指令,则使用$prefix/html目录。所有参数对应的文件都不存在时,进行ngx_autoindex 模块

请注意日志中的internal redirect这是由于index指令在check后,存在文件便会发起"内部rewrite" 这里的internal redirect和rewrite的last flag"内部rewrite"有不同的地方。这里的处理阶段回到server_rewrite阶段,此处我们有意的在server段的中添加set指令(server段的set指令工作在server_rewrite阶段),日志中的http script set $a可以看出处理阶段。值得注意的是debug日志中也有意的做了区别,使用的为internal redirect而不是像rewrite指令产生的rewriteen data日志。

try_files

Syntax:	try_files file ... uri;
try_files file ... =code;
Default:	—
Context:	server, location

注意这里和index的区别,最后为uri。 If none of the files were found, an internal redirect to the uri specified in the last parameter is made。

server {
    listen 8087;
    server_name 172.19.23.208;
    default_type text/html;

    set $a 'dafa';
    location ~ /waf_admin/templates/(css|js) {
        root  /opt/openresty/nginx/conf/;
    }

    location ~ / {
        try_files /index.htm /index.html /index.html1;  
    }
}

注意这里是用的是internal redirect 所以这里和index产生的"内部rewrite"效果一致,从日志中也可以看出来。

2018/01/05 17:18:13 [debug] 31757#0: *220 http process request line
2018/01/05 17:18:13 [debug] 31757#0: *220 http request line: "GET / HTTP/1.1"
2018/01/05 17:18:13 [debug] 31757#0: *220 http uri: "/"
2018/01/05 17:18:13 [debug] 31757#0: *220 http args: ""
2018/01/05 17:18:13 [debug] 31757#0: *220 http exten: ""
2018/01/05 17:18:13 [debug] 31757#0: *220 posix_memalign: 0000000000A67860:4096 [@16](https://my.oschina.net/u/2771800)
2018/01/05 17:18:13 [debug] 31757#0: *220 http process request header line
2018/01/05 17:18:13 [debug] 31757#0: *220 http header: "User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"
2018/01/05 17:18:13 [debug] 31757#0: *220 http header: "Host: 172.19.23.208:8087"
2018/01/05 17:18:13 [debug] 31757#0: *220 http header: "Accept: */*"
2018/01/05 17:18:13 [debug] 31757#0: *220 http header done
2018/01/05 17:18:13 [debug] 31757#0: *220 event timer del: 7: 1515143953722
2018/01/05 17:18:13 [debug] 31757#0: *220 rewrite phase: 0
2018/01/05 17:18:13 [debug] 31757#0: *220 rewrite phase: 1
2018/01/05 17:18:13 [debug] 31757#0: *220 http script value: "dafa"
2018/01/05 17:18:13 [debug] 31757#0: *220 http script set $a
2018/01/05 17:18:13 [debug] 31757#0: *220 test location: ~ "/waf_admin/templates/(css|js)"
2018/01/05 17:18:13 [debug] 31757#0: *220 test location: ~ "/"
2018/01/05 17:18:13 [debug] 31757#0: *220 using configuration "/"
2018/01/05 17:18:13 [debug] 31757#0: *220 http cl:-1 max:4194304
2018/01/05 17:18:13 [debug] 31757#0: *220 rewrite phase: 3
2018/01/05 17:18:13 [debug] 31757#0: *220 rewrite phase: 4
2018/01/05 17:18:13 [debug] 31757#0: *220 post rewrite phase: 5
2018/01/05 17:18:13 [debug] 31757#0: *220 generic phase: 6
2018/01/05 17:18:13 [debug] 31757#0: *220 generic phase: 7
2018/01/05 17:18:13 [debug] 31757#0: *220 access phase: 8
2018/01/05 17:18:13 [debug] 31757#0: *220 access phase: 9
2018/01/05 17:18:13 [debug] 31757#0: *220 post access phase: 10
2018/01/05 17:18:13 [debug] 31757#0: *220 try files phase: 11
2018/01/05 17:18:13 [debug] 31757#0: *220 trying to use file: "/index.htm" "/opt/openresty//nginx/html/index.htm"
2018/01/05 17:18:13 [debug] 31757#0: *220 trying to use file: "/index.html" "/opt/openresty//nginx/html/index.html"
2018/01/05 17:18:13 [debug] 31757#0: *220 trying to use file: "/index.html1" "/opt/openresty//nginx/html/index.html1"
2018/01/05 17:18:13 [debug] 31757#0: *220 internal redirect: "/index.html1?"
2018/01/05 17:18:13 [debug] 31757#0: *220 rewrite phase: 0
2018/01/05 17:18:13 [debug] 31757#0: *220 rewrite phase: 1
2018/01/05 17:18:13 [debug] 31757#0: *220 http script value: "dafa"
2018/01/05 17:18:13 [debug] 31757#0: *220 http script set $a
2018/01/05 17:18:13 [debug] 31757#0: *220 test location: ~ "/waf_admin/templates/(css|js)"
2018/01/05 17:18:13 [debug] 31757#0: *220 test location: ~ "/"
2018/01/05 17:18:13 [debug] 31757#0: *220 using configuration "/"
2018/01/05 17:18:13 [debug] 31757#0: *220 http cl:-1 max:4194304

这里注意,nginx internal redirect的次数限制为10次,超过后则会500错误,日志中会有rewrite or internal redirection cycle while internally redirecting to

proxy_pass

该指令属于ngx_http_proxy_module运行在content阶段。可以发起跳转。不过这里的跳转,可以理解为,新请求完全重新处理。

重写的影响

这里包括rewrite、 internal redirect、 proxy_pass产生新的请求。

复杂情况下,我们使用Openresty作为请求预处理容器。如果相关的访问控制lua代码引入到access_by_lua_file到http或者server块。由于内部处理通常会有多次跳转过程,如果这个过程包含了access_by_lua_file,便会使得lua代码执行两次,如果代码中包含计数的情况,处理结果会和预期不符。同理set_by_lua*rewrite_by_lua*content_by_lua*都会有这个潜在问题。同时需要注意的是在引用第三方module时通常会有一些与预期不符的结果。我们可以查看第三方module的phase来确定阶段,或者通过debug日志来确定是否由于处理阶段不同而导致结果差异。

展开阅读全文
打赏
0
0 收藏
分享
加载中
更多评论
打赏
0 评论
0 收藏
0
分享
返回顶部
顶部