Pinpointing the Python Code Paths with High Disk I/O (using OpenResty XRay)
ChangeLog 1.4.3
Yichun Zhang , 15 Dec 2013 (created 29 Oct 2013)Mainline Version 1.4.3.9 - 14 December 2013
- bugfix: the include path for LuaJIT C headers was still pointing
to
luajit-2.0
, which should have beenluajit-2.1
instead. thanks Tor Hveem for the report.
Mainline Version 1.4.3.7 - 14 December 2013
- upgraded LuaJIT to v2.1-20131211.
- see changes here: https://github.com/agentzh/luajit2/commits/v2.1-agentzh
- bundled Lua Resty Core Library 0.0.2.
- this library reimplements Lua Nginx Module's Lua API with LuaJIT FFI. see https://github.com/agentzh/lua-resty-core for more details.
- upgraded Lua Nginx Module to 0.9.3.
- feature: added a lot of pure C API (without using any Lua VM's C API) for FFI-based Lua API implementations like Lua Resty Core Library.
- feature: allow creating 0-delay timers upon worker process existing.
- feature: added new API function ngx.worker.exiting() for testing if the current worker process has started exiting.
- feature: ngx.re.find() now accepts the optional 5th argument "nth" to control which submatch capture's indexes are returned. thanks Lance Li for the feature request.
- feature: added new API for version numbers of both Nginx and Lua Nginx Module itself: ngx.config.nginx_version and ngx.config.ngx_lua_version. thanks smallfish for the patch.
- feature: added support for loading LuaJIT 2.1 bytecode files directly in
*_by_lua_file
configuration directives. - bugfix: ngx.req.set_header() did not completely override the existing request header with multiple values. thanks Aviram Cohen for the report.
- bugfix: modifying request headers in a subrequest could lead to assertion failures and crashes. thanks leaf corcoran for the report.
- bugfix: turning off lua_code_cache could lead to memory issues (segfaults and LuaJIT assertion failures) when Lua libraries using LuaJIT FFI were used. now we always create a clean separate Lua VM instance for every Nginx request served by us when the Lua code cache is disabled. thanks Ron Gomes for the report.
- bugfix: the linker option
-E
is not support in Cygwin's linker; we should test--export-all-symbols
at the same time. thanks Heero Zhang for the report. - bugfix: fixed the warnings from the Microsoft Visual C++ compiler. thanks Edwin Cleton for the report.
- optimize: optimized the implementation of ngx.headers_sent a bit.
- doc: added new section "Statically Linking Pure Lua Modules": https://github.com/chaoslawful/lua-nginx-module#statically-linking-pure-lua-modules
- doc: typo fixes from Zheng Ping.
- upgraded Headers More Nginx Module to 0.24.
- bugfix: more_set_input_headers did not completely override the existing request header with multiple values. thanks Aviram Cohen for the report.
- upgraded Set Misc Nginx Module to 0.23.
- feature: added new configuration directives set_formatted_gmt_time and set_formatted_local_time. thanks Trurl McByte for the patch.
- upgraded Memc Nginx Module to 0.14.
- feature: added new configuration directive memc_ignore_client_abort. thanks Eldar Zaitov for the patch.
- upgraded Rds Json Nginx Module to 0.13.
- bugfix: fixed the warnings from the Microsoft Visual C++ compiler. thanks Edwin Cleton for the report.
- upgraded Echo Nginx Module to 0.50.
- bugfix: fixed the warnings from the Microsoft Visual C++ compiler. thanks Edwin Cleton for the report.
- upgraded Array Var Nginx Module to 0.03.
- bugfix: fixed the warnings from the Microsoft Visual C++ compiler. thanks Edwin Cleton for the report.
- upgraded Redis Nginx Module module to 0.3.7.
- see changes here: http://mailman.nginx.org/pipermail/nginx/2013-December/041297.html
- feature: applied the larger_max_error_str patch to
the nginx core to allow error log messages up to 4096 bytes and to allow the
C macro
NGX_MAX_ERROR_STR
to be overridden from the outside. - feature: added new configure option
--with-pcre-conf-opt=OPTIONS
to the nginx core to allow custom PCRE ./configure build options. thanks Lance Li for the original patch.
Stable Version 1.4.3.6 - 20 November 2013
- bugfix: applied the official patch patch.2013.space.txt for the Nginx core to fix the security issue CVE-2013-4547.
Stable Version 1.4.3.4 - 12 November 2013
This release is essentially the same as the last mainline release, 1.4.3.3.
The following components are bundled in this release:
- LuaJIT-2.0.2
- array-var-nginx-module-0.03rc1
- auth-request-nginx-module-0.2
- drizzle-nginx-module-0.1.6
- echo-nginx-module-0.49
- encrypted-session-nginx-module-0.03
- form-input-nginx-module-0.07
- headers-more-nginx-module-0.23
- iconv-nginx-module-0.10
- lua-5.1.5
- lua-cjson-1.0.3
- lua-rds-parser-0.05
- lua-redis-parser-0.10
- lua-resty-dns-0.10
- lua-resty-lock-0.01
- lua-resty-memcached-0.12
- lua-resty-mysql-0.14
- lua-resty-redis-0.17
- lua-resty-string-0.08
- lua-resty-upload-0.09
- lua-resty-websocket-0.02
- memc-nginx-module-0.13
- nginx-1.4.3
- ngx_coolkit-0.2rc1
- ngx_devel_kit-0.2.19
- ngx_lua-0.9.2
- ngx_postgres-1.0rc3
- rds-csv-nginx-module-0.05
- rds-json-nginx-module-0.12
- redis-nginx-module-0.3.6
- redis2-nginx-module-0.10
- set-misc-nginx-module-0.22
- srcache-nginx-module-0.24
- xss-nginx-module-0.04
Mainline Version 1.4.3.3 - 6 November 2013
- upgraded Lua Nginx Module to 0.9.2.
- feature: added new API function ngx.re.find(), which is similar to ngx.re.match, but only returns the beginning index and end index (1-based) of the whole match, which is 30% ~ 40% faster than
ngx.re.match
for simplest regexes. - feature: added new API function ngx.config.prefix() to return the Nginx server "prefix" path.
- bugfix: reading ngx.header.HEADER could result in Lua string storage corruptions. thanks Dane Knecht for the report.
- bugfix: ngx.re.match: the "ctx" parameter table's "pos" field should start from 1 instead of 0.
- bugfix: fixed compilation errors with Nginx older than 1.0.0.
- bugfix: localizing the coroutine.* API functions in init_by_lua* for future use in contexts like content_by_lua* might hang the request. thanks James Hurst for the report.
- feature: added new API function ngx.re.find(), which is similar to ngx.re.match, but only returns the beginning index and end index (1-based) of the whole match, which is 30% ~ 40% faster than
- upgraded Srcache Nginx Module to 0.24.
- bugfix: fixed compilation errors with Nginx older than 0.9.2.
- bugfix: applied the cache_manager_exit patch to the Nginx core to fix an issue when the cache manager process is shutting down.
Mainline Version 1.4.3.1 - 29 October 2013
- upgraded the Nginx core to 1.4.3.
- see the changes here: http://nginx.org/en/CHANGES-1.4
- upgraded Lua Nginx Module to 0.9.1.
- feature: added the new configuration directive lua_use_default_type for controlling whether to send out a default
Content-Type
response header (as defined by the default_type directive). default on. thanks aviramc for the patch. - feature: now the raw request cosocket returned by ngx.req.socket(true) no longer requires the request body to be read already, which means that one can use this cosocket to read the raw request body data as well. thanks aviramc for the original patch.
- bugfix: when there were no existing
Cache-Control
response headers,ngx.header.cache_control = nil
would (incorrectly) create a newCache-Control
header with an empty value. thanks jinglong for the patch. - bugfix: the original letter-case of the header name was lost when creating the
Cache-Control
response header via the ngx.header.HEADER API. - bugfix: ngx.req.set_header("Host", value) would overwrite the value of $host with bad values. thanks aviramc for the patch.
- bugfix: use of ngx.exit() to abort pending subrequests in other "light threads" might lead to segfault or request hang when HTTP 1.0 full buffering is in effect.
- bugfix: removing a request header might lead to memory corruptions. thanks Bjørnar Ness for the report.
- bugfix: reading ngx.status might get different values than $status. thanks Kevin Burke for the report.
- bugfix: downstream write events might interfere with upstream cosockets that are slow to write to. thanks Aviram Cohen for the report.
- bugfix: the bookkeeping state for already-freed user threads might be incorrectly used by newly-created threads that were completely different, which could lead to bad results. thanks Sam Lawrence for the report.
- bugfix: calling ngx.thread.wait() on a user thread object that is already waited (i.e., already dead) would hang forever. thanks Sam Lawrence for the report.
- bugfix: the alert "zero size buf" could be logged when assigning an empty Lua string ("") to
ngx.arg[1]
in body_filter_by_lua*. - bugfix: subrequests initiated by ngx.location.capture* could trigger unnecessary response header sending actions in the subrequest because our capturing output header filter did not set
r->header_sent
. - bugfix: the Lua error message for the case that ngx.sleep() was used in log_by_lua* was not friendly. thanks Jiale Zhi for the report.
- bugfix: now ngx.req.socket(true) returns proper error when there is some other "light thread" reading the request body.
- bugfix: header_filter_by_lua, body_filter_by_lua, and ngx.location.capture* might not work properly with multiple "http {}" blocks in
nginx.conf
. thanks flygoast for the report. - optimize: made ngx.re.match and ngx.re.gmatch faster for LuaJIT 2.x when there is no submatch captures.
- optimize: pre-allocate space for the Lua tables in various places.
- doc: fixed the context for the lua_package_path and lua_package_cpath directives. thanks duhoobo for the report.
- feature: added the new configuration directive lua_use_default_type for controlling whether to send out a default
- upgraded Headers More Nginx Module to 0.23.
- bugfix: removing request headers via more_clear_input_headers might lead to memory corruptions.
- bugfix: more_set_input_headers might overwrite the value of the $host variable with bad values.
- bugfix: more_set_headers and more_clear_headers might not work when multiple "http {}" blocks were used in
nginx.conf
. - bugfix: eliminated use of C global variables during configuration phase because it might lead to issues when HUP reload failed.
- upgraded Srcache Nginx Module to 0.23.
- bugfix: this module might not work properly with multiple "http {}" blocks in
nginx.conf
. - bugfix: we might (incorrectly) return 500 in our output filters.
- bugfix: we did not set
r->header_sent
when we want to discard the header in our header filter.
- bugfix: this module might not work properly with multiple "http {}" blocks in
- upgraded Rds Json Nginx Module to 0.12.
- bugfix: in case of multiple "http {}" blocks in
nginx.conf
, our output filters might be disabled even when this module is configured properly. - bugix: we did not check the
NULL
pointer returned by an Nginx array element allocation.
- bugfix: in case of multiple "http {}" blocks in
- upgraded Rds Csv Nginx Module to 0.05.
- optimize: we now only register our output filters when this module is indeed used (the only exception is when multiple "http {}" blocks are used).
- upgraded Xss Nginx Module to 0.04.
- optimize: we now only register our output filters when this module is indeed used (the only exception is when multiple "http {}" blocks are used).
- upgraded Echo Nginx Module to 0.49.
- bugfix: echo_before_body and echo_after_body might now work properly when multiple "http {}" blocks were used in
nginx.conf
.
- bugfix: echo_before_body and echo_after_body might now work properly when multiple "http {}" blocks were used in
- upgraded Lua Resty Redis Library to 0.17.
- optimize: added an optional argument "n" to init_pipeline() as a hint for the number of pipelined commands.
- optimize: use LuaJIT 2.1's new table.new() primitive to pre-allocate space for Lua tables.
- upgraded Lua Resty Upload Library to 0.09.
- bugfix: removed use of the module() function to prevent bad side-effects.
- optimize: Removed use of lua tables and table.concat() for simple one-line Lua string concatenations.
- upgraded Lua Resty MySQL Library to 0.14.
- bugfix: avoided using Lua 5.1's module() function for defining our Lua modules because it has bad side effects.
- optimize: added an optional new argument "nrows" to the query() and read_result() methods, which can speed up things a bit.
- optimize: use LuaJIT v2.1's new table.new() API to optimize Lua table allocations. when table.new is missing, just fall back to the good old "{}" constructor. this gives 12% overall speed-up for a typical result set with 500 rows when LuaJIT 2.1 is used.
- optimize: eliminated use of table.insert() because it is slower than "tb[#tb + 1] = val".
- optimize: switched over to the multi-argument form of string.char().
- optimize: no longer use Lua tables and table.concat() to construct simple query strings.
- upgraded Lua Resty Web Socket Library to 0.02.
- optimize: use LuaJIT 2.1's table.new() to preallocate space for Lua tables, eliminating the overhead of Lua table rehash.
- feature: applied the proxy_host_port_vars patch to
the Nginx core to make
$proxy_host
and$proxy_port
accessible for dynamic languages like Lua and Perl. - bugfix: applied the gzip_flush_bug patch to the Nginx core to fix request hang caused by the ngx_gzip and ngx_gunzip modules when using ngx.flush(true), for example. Thanks Maxim Dounin for the review.
- bugfix: applied the cache_lock_hang_in_subreq patch to the Nginx core to fix the request hang when using proxy_cache_lock in subrequests and the cache lock timeout happens.
- bugfix: backported Maxim Dounin's patch to
fix an issue in the ngx_gzip module:
it did not clear
r->connection->buffered
when the pending data was already flushed out. this could hang Lua Nginx Module's ngx.flush(true) call, for example. See ChangeLog 1.4.2 for change log for OpenResty 1.4.2.x.