Pinpointing the Python Code Paths with High Disk I/O (using OpenResty XRay)
ChangeLog for 1.17.8.x
jiahao , 01 Nov 2020 (created 30 Jun 2020)Version 1.17.8.2 - 13 July 2020
- bugfix: builds with http_perl_module was broken. this regression had appeared in 1.17.8.1.
- change: we no longer remove a lot of files (docs, perl modules, tests, and etc.) in our tarball.
- upgraded lua-resty-limit-traffic to v0.07.
- bugfix: miss to uncommit the last limiter in resty.limit.traffic's list. Thanks Jiteng Wang for the patch.
Version 1.17.8.1 - 4 July 2020
upgraded the nginx core to 1.17.8.
- see the changes here: https://nginx.org/en/CHANGES
bugfix: nginx would crash when receiving SIGHUP in the single process mode. Thanks root for the patch.
bugfix: ngx_http_static_module: the 'Locatoin' response header value was not properly encoded by URI rules. Thanks lijunlong for the patch.
feature: passed C compiler option
-g
by default for statically linked openssl, pcre, and zlib libraries to enable debuginfo. Thanks lijunlong for the patch.feature: added support for OpenSSL 1.1.1 by upgrading the OpenSSL patches. Thanks spacewander for the patch.
feature: added a new
--with-luajit-ldflags=OPTS
option for specifying custom LuaJIT linker flags.feature: ensured all OpenSSL patches are now included in the release tarball. Thanks Thibault Charbonnier for the patch.
optimize: added an NGINX core patch to ensure unused listening fds are closed when
reuseport
is used. Thanks spacewander for the patch.optimize: removed many non-source files from the release tarball to reduce its final size. Thanks Thibault Charbonnier for the patch.
change: renamed the
ssl_pending_session
patch tossl_sess_cb_yield
for NGINX cores 1.17.1 and above. Thanks spacewander for the patch.change: removed the
gcc-maybe-uninitialized-warning
patch which is now obsolete. Thanks Thibault Charbonnier for the patch.change: we no longer maintain the NGINX
dtrace
patch.bugfix: support yielding in ssl_certificate_by_lua_* when
ssl_early_data
is on. Thanks spacewander for the patch.bugfix: ensured we apply the
init_cycle_pool_release
patch to NGINX cores >= 1.13.6 instead of 1.13.6 only. Thanks Thibault Charbonnier for the patch.tweak: updated the
--without-luajit-gc64
option to follow Mike Pall's commit which enables GC64 by default on x64 platforms. Thanks Thibault Charbonnier for the patch.tweak: updated the
./configure --help
usage text output for recent NGINX cores.win32/win64: upgraded PCRE to 8.44 and OpenSSL to 1.1.1d.
upgraded ngx_lua to 0.10.16.
feature:
ngx.req.set_uri()
: added the 'binary' optional boolean arg to allow arbitrary binary data in the unencoded URI. Thanks lijunlong for the patch.security:
ngx.req.set_header()
: now we always escape bytes in headernames and header values which are prohibited by RFC 7230. Thanks lijunlong for the patch.feature:
ngx.req.set_uri_args()
now automatically escapes control and whitespace characters if the query-string is provided directly. Thanks lijunlong for the patch.bugfix:
ngx.req.set_uri_args()
threw an exception with wrong argument info. Thanks lijunlong for the patch.bugfix:
set_by_lua_block
allowed more than one arg (in addition to the block). Thanks lijunlong for the patch.bugfix: prevented request smuggling in the
ngx.location.capture()
API. Thanks UltramanGaia for the report and Thibault Charbonnier for the patch.bugfix:
ngx.req.set_header()
: only override the input header once. Thanks spacewander for the patch.change: lua-resty-core is now mandatorily loaded, and the lua_load_resty_core directive is deprecated. Thanks Thibault Charbonnier for the patch.
change: given the above change, old CFunction APIs have been retired when newer FFI implementations are available via
resty.core
. Thanks Thibault Charbonnier for the patch.change: given the above changes, we now prevent compilation with PUC-Rio Lua; only LuaJIT 2.x is supported going forward. Thanks Thibault Charbonnier for the patch.
change: removed compatibility code for unsupported NGINX core versions (< 1.6.0). Thanks Thibault Charbonnier for the patch.
change: we now ignore
lua_regex_*
directives when NGINX is compiled without PCRE support, which allows for resty-cli to work without requiring PCRE. Thanks Thibault Charbonnier for the patch.change: removed extraneous error logging when
coroutine.resume()
throws runtime errors; this change ensures ngx_lua aligns with standard Lua behavior. Thanks Thibault Charbonnier for the patch.change:
coroutine.wrap()
now propagates runtime errors to the parent coroutine; this change ensures ngx_lua aligns with standard Lua behavior. Thanks Thibault Charbonnier for the patch.feature: ngx.pipe: added support for specifying environment variables in the
ngx_pipe.spawn()
FFI API. Thanks spacewander for the patch.feature: ngx.pipe: added support for specifying timeouts in the
ngx_pipe.spawn
FFI API. Thanks spacewander for the patch.feature: ngx.pipe: added support for calling
ngx_pipe.shutdown()
on a sub-process when a light thread is waiting on it. Thanks spacewander for the patch.feature: enabled the
ngx.thread
andngx.socket.udp
APIs in ssl_session_fetch_by_lua*. Thanks Tom Thorogood for the patch.feature: implemented the FFI interface for the
ngx_req.add_header
API. Thanks spacewander for the patch.feature: implemented the FFI interfaces for
ngx.crc32_short
andngx.crc32_long
. Thanks Thibault Charbonnier for the patch.feature: when a timer fails to run, we now log the location of its function's definition. Thanks spacewander for the patch.
bugfix: skip the ssl_session_store_by_lua* and ssl_session_fetch_by_lua* handlers when using TLS 1.3. Thanks spacewander for the patch.
bugfix: config: ensured LuaJIT FFI check and static PCRE builds succeed on all platforms. Thanks Thibault Charbonnier for the patch.
bugfix: config: fallback to
--undefined
option for ld 2.25 and below. Thanks Thibault Charbonnier for the patch.bugfix: config: ensured the
pcre_version
symbol is always preserved when PCRE is statically linked. Thanks Thibault Charbonnier for the patch.bugfix: config: ensured the
pcre_version
symbol is always preserved on Darwin platforms as well. Thanks Thibault Charbonnier for the patch.bugfix: config: fixed an issue preventing compiliation with dynamic modules; we now avoid specifying
-DLUA_DEFAULT_PATH
and-DLUA_DEFAULT_CPATH
viaCFLAGS
.bugfix: ensured set_by_lua_file directives containing NGINX variables re-computes their closure's code cache key. Thanks Thibault Charbonnier for the patch.
bugfix: fixed compilation with NGINX cores < 1.11.11. Thanks Thibault Charbonnier for the patch.
bugfix: error logs now set LuaJIT's currentline to
-1
if no currentline is available. Thanks spacewander for the patch.bugfix: ngx.pipe: ensured reading while a process died returns the "closed" error. Thanks Thibault Charbonnier for the patch.
bugfix: added missing arguments to an
ngx_log_error
call inngx_http_lua_pipe.c
. Thanks Thibault Charbonnier for the patch.optimize: improved code cache lookup performance by using
luaL_ref()
to avoid invoking the costlylj_str_new
for each Lua handler execution. Thanks Thibault Charbonnier for the patch.optimize: fixed the pre-allocated number of keys in the
ngx.socket.tcp
table. Thanks Thibault Charbonnier for the patch.optimize: removed declaration of the obsolete
ngx_http_lua_inject_logby_ngx_api
function. Thanks Thibault Charbonnier for the patch.refactor: simplified Lua chunk and file cache key generation. Thanks Thibault Charbonnier for the patch.
refactor: reduced cache key size by removing unnecessary chunkname component. Thanks Thibault Charbonnier for the patch.
misc:
ngx_http_lua_util.h
: removedngx_str_set
definition as it is always defined in NGINX 1.6.0+ (the minimum supported NGINX core version). Thanks Thibault Charbonnier for the patch.misc: removed dead code, guard non-OpenResty LuaJIT definitions, and fixed styling issues. Thanks Thibault Charbonnier for the patch.
misc: fixed some warnings from the clang static code analyzer.
style: fixed a minor alignment issue in
ngx_http_lua_ssl_certby.c
. Thanks Thibault Charbonnier for the patch.style: updated
nginx_version
guard macros and assume it is always defined. Thanks Thibault Charbonnier for the patch.
upgraded ngx_stream_lua to 0.0.8.
- feature: this module can now be compiled as a dynamic module with via the
--with-dynamic-module=PATH
option of./configure
. - bugfix: config: fixed an issue preventing compiliation with dynamic modules; we now avoid specifying
-DLUA_DEFAULT_PATH
and-DLUA_DEFAULT_CPATH
viaCFLAGS
. - Ported many features from ngx_http_lua 0.10.16. Thanks Thibault Charbonnier for the ports.
- change: lua-resty-core is now mandatorily loaded, and the lua_load_resty_core directive is deprecated.
- change: given the above change, old CFunction APIs have been retired when newer FFI implementations are available via
resty.core
. - change: given the above changes, we now prevent compilation with PUC-Rio Lua; only LuaJIT 2.x is supported going forward.
- change: removed compatibility code for unsupported NGINX core versions (< 1.6.0).
- change: removed extraneous error logging when
coroutine.resume()
throws runtime errors; this change ensures ngx_stream_lua aligns with standard Lua behavior. - change:
coroutine.wrap()
now propagates runtime errors to the parent coroutine; this change ensures ngx_stream_lua aligns with standard Lua behavior. - feature: when a timer fails to run, we now log the location of its function's definition.
- bugfix: error logs now set LuaJIT's currentline to
-1
if no currentline is available. - bugfix: fixed compilation with NGINX cores < 1.11.11.
- misc: fixed some warnings from the clang static code analyzer.
- Ported many features from ngx_http_lua 0.10.15. Thanks Thibault Charbonnier for the ports.
- feature: added the
pool_size
andbacklog
options to the tcpsock:connect() API in order to support backlog queuing in cosocket connection pools. - feature: implemented the tcpsock:receiveany() upstream cosocket API.
- feature: allowed sending boolean and nil values in cosockets.
- feature: api.h: exposed the
ngx_stream_lua_ffi_str_t
C data type for other NGINX C modules. - feature: errors are now logged when timers fail to run.
- bugfix: we now avoid tcpsock:setkeepalive() putting connections into the pool when NGINX is already shutting down.
- bugfix: inlined Lua code snippets in
nginx.conf
failed to use the Lua source checksum as part of the Lua code cache key. - bugfix: config: ensured LuaJIT FFI check and static PCRE builds succeed on all platforms.
- bugfix: config: fallback to
--undefined
option for ld 2.25 and below. - bugfix: config: ensured the
pcre_version
symbol is always preserved when PCRE is statically linked. - bugfix: config: ensured the
pcre_version
symbol is always preserved on Darwin platforms as well.
- feature: added the
- feature: this module can now be compiled as a dynamic module with via the
upgraded lua-resty-core to 0.1.18.
- bugfix:
ngx.req.get_headers()
might return an empty table without the metatable set. Thanks chengjie.zhou for the patch. - feature:
resty.core.uri
:ngx.escape_uri()
: add optional argumenttype
. Thanks lijunlong for the patch. - change: we now require ngx_lua v0.10.16 and ngx_stream_lua v0.0.8.
- change: updated to support the ngx_http_lua module without the now removed CFunction APIs. Thanks Thibault Charbonnier for the patch.
- feature: updated the
ngx_ssl.get_tls1_version_str()
API to recognize TLS 1.3 connections. Thanks Thibault Charbonnier for the patch. - feature: implemented the
ngx.req
module and thengx_req.add_header()
API. Thanks spacewander for the patch. - feature: added support specifying timeouts in the
ngx_pipe.spawn()
API. Thanks spacewander for the patch. - feature: added support for specifying environment variables in the
ngx_pipe.spawn()
API. Thanks spacewander for the patch. - feature: allowed for calling
ngx_pipe.shutdown()
on a sub-process when a light thread is waiting on it. Thanks spacewander for the patch. - feature: implemented
ngx.crc32_short()
andngx.crc32_long()
via FFI. Thanks Thibault Charbonnier for the patch. - feature: ensured loading the
ngx.re
module without PCRE produces a friendly error. Thanks Thibault Charbonnier for the patch. - feature: enabled the FFI-based API for all
ngx.worker.*
APIs in the stream subsystem. Thanks Thibault Charbonnier for the patch. - feature: enabled the FFI-based API for
ngx.var
in the stream subsystem. Thanks Thibault Charbonnier for the patch. - feature: enabled the FFI-based API for
ngx.ctx
and 'ngx.status' in the stream subsystem. Thanks Thibault Charbonnier for the patch. - feature: enabled the FFI-based API for
ngx.encode_base64()
andngx.decode_base64()
in the stream subsystem. Thanks Thibault Charbonnier for the patch. - feature: enabled the FFI-based API for
ngx.exit()
in the stream submodule. Thanks Thibault Charbonnier for the patch. - feature: enabled the FFI-based APIs for
ngx.escape_uri()
andngx.unescape_uri()
in the stream subsystem. Thanks Thibault Charbonnier for the patch. - feature: enabled the FFI-based APIs for
ngx.md5()
,ngx.md5_bin()
, andngx.sha1_bin()
in the stream subsystem. Thanks Thibault Charbonnier for the patch. - bugfix:
ngx.re
: fixed the error stacktrace level when missing PCRE support. Thanks Thibault Charbonnier for the patch. - bugfix: ensured
ngx.pipe
APIs with an invalidself
argument throw an error with the proper stack level. Thanks Thibault Charbonnier for the patch. - bugfix: updated
ssl.get_tls1_version_str()
to return the error"unknown version"
when the TLS version number is not recognized. Thanks Thibault Charbonnier for the patch. - optimize: load the
resty.core.misc
module last to avoid metatable lookups onngx
. Thanks Thibault Charbonnier for the patch. - optimize: made
ngx_resp.add_header()
early-exit whenheader_value
is an empty table. Thanks spacewander for the patch.
- bugfix:
upgraded LuaJIT to 2.1-20200102.
- feature: Increased the maximum number of allowed upvalues from 60 to 120.
- feature: Added initial s390x support.
- Move all register allocations out of the asm_href loop.
- aarch64: Fix register allocation issue for XLOAD.
- aarch64: Use the xzr register whenever possible.
- aarch64: Allocate LJ_TISNUM early.
- aarch64: Fixed crash with side traces under register pressure.
- ARM: Fix up condition codes for conditional arithmetic insn.
- ARM64: Added support for FNMADD and FNMSUB.
- Fixed
os.date
for timezone change awareness. jit.prngstate
: Return a sane value (0) forLUAJIT_DISABLE_JIT
.- thread.exdata: Build
recff_thread_exdata
only forLJ_HASFFI
. - Removed redundant
emit_check_ofs
. - doc: mentioned the increase in the maximum number of upvalues in the miscellaneous section.
- doc: reworded the description of the table.isempty API.
- imported Mike Pall's latest changes:
- Properly fix pointer diff in string.find().
- x64: Enable
LJ_GC64
mode by default. - FFI: Eliminate hardcoded string hashes.
- Fix interaction between profiler hooks and finalizers.
- Don't use STRREF for pointer diff in string.find().
- Fix
tonumber("-0")
. - Fix hash table chaining (again).
- Fix declarations of _BitScanForward/_BitScanReverse.
- Add stricter check for
print()
vs.tostring()
shortcut. - Prevent integer overflow while parsing long strings.
- Fix stack check when recording BC_VARG.
- FFI: Add missing write barrier on C library index update.
- FFI: Workaround for platform dlerror() returning NULL.
- OSX: Use
__thread
attribute. - OSX: Don't set a default
MACOSX_DEPLOYMENT_TARGET
. - Build MinGW import library, too.
- Fix MinGW make clean.
- Update Android and iOS build docs.
- Add note about the unsuitabilty of
math.random()
for crypto. - Update MSVC build script and docs.
- More recent MSVC is partially C99 compliant.
- Fix narrowing of conversions to U32.
- doc: readme.md: improve completeness and readability.
- Fix unsinking of 64 bit constants.
- Fix bytecode dump unpatching.
- Fix
debug.getinfo()
argument check. - MIPS: Fix delay slot hint.
- Fix TNEW load forwarding with instable types.
- ARM: Fix GCC 7
-Wimplicit-fallthrough
warnings. - ARM: Fix condition code check fusion.
- ARM64: Avoid side-effects of constant rematerialization.
upgraded lua-resty-lrucache to 0.10.
- feature: implemented a user flags attribute similar to that of the 'shdict' API. Thanks Thibault Charbonnier for the patch.
- feature: implemented the
cache:get_keys()
API. Thanks Thibault Charbonnier for the patch. - feature: implemented the
cache:count()
andcache:capacity()
APIs. Thanks Datong Sun for the patch. - feature: implemented a user
flags
attribute similar to that of thengx.shared
API. Thanks Thibault Charbonnier for the patch. - makefile: added a
lint
target to detect invalid test cases which we now use in Travis CI. Thanks Thibault Charbonnier for the patch. - doc: cleaned up outdated resources and grammatical improvements.
- doc: fixed a minor typo in the
cache:flush_all()
description. - doc: updated the
cache:set
usage section to reflect thettl
argument being optional.
upgraded lua-resty-string to 0.12.
- optimize: removed a duplicate string length lookup in
to_hex()
. Thanks Robert Paprocki for the patch.
- optimize: removed a duplicate string length lookup in
upgraded lua-resty-redis to 0.28.
- bugfix: handle mixture of
read_reply()
and other commands. Thanks spacewander for the patch. - feature: added new options
ssl
andssl_verify
to thered:connect()
API for connecting to Redis over TLS. Thanks Vinayak Hulawale for the patch. - feature: implemented the
red:set_timeouts()
API. Thanks zouyi for the patch.
- bugfix: handle mixture of
upgraded lua-resty-shell to 0.03.
- doc: clarified return values and behavior upon reaching a timeout threshold. Thanks Dejiang Zhu for the patch.
upgraded resty-cli to 0.25.
- security: restydoc-index: we did not quote doc file paths properly when interpolating them into shell commands, which was a security vulnerability. Thanks xlibor for the patch.
- feature: resty: added new
--user-runner
option. - feature: set the
ngx.config.is_console
field totrue
to detect the resty-cli environment from Lua scripts. - bugfix: we do not forward
SIGHUP
to NGINX processes anymore since resty-cli is usually run as non-daemon andSIGHUP
should be converted toSIGQUIT
.
upgraded ngx_echo to 0.62.
- bugfix: config: we now always use double quotes in
[...]
conditionals. - bugfix: config: avoided an error with NGINX 1.17.0 and above. Thanks Thibault Charbonnier for the patch.
- style: fixed a coding style issue found by
ngx-releng
.
- bugfix: config: we now always use double quotes in
upgraded ngx_srcache to 0.32.
- bugfix: config: we should always use double quotes in
[...]
conditionals. - bugfix: config: avoided an error with NGINX 1.17.0 and above. Thanks Thibault Charbonnier for the patch.
- style: fixed minor coding style issues found by
ngx-releng
. - doc: documented a tip to make memcached store objects bigger than 1MB.
- bugfix: config: we should always use double quotes in
upgraded lua-cjson to 2.1.0.8.
- feature: added an option to disable forward slash escaping. Thanks Jesper Lundgren for the patch.
upgraded lua-resty-memcached to v0.15.
- bugfix:
gets()
did not return socket send errors at the correct index. Thanks Justin Li for the patch.
- bugfix: