Pinpointing the Python Code Paths with High Disk I/O (using OpenResty XRay)
OpenResty 1.21.4.2 RC1 is out
Johnny Wang , 17 Apr 2023 (created 17 Apr 2023)We have just kicked out a new release candidate, OpenResty 1.21.4.2 RC1, for the community to test out.
Source package
https://openresty.org/download/openresty-1.21.4.2rc1.tar.gz
PGP for this source tar ball:
https://openresty.org/download/openresty-1.21.4.2rc1.tar.gz.asc
Windows binary packages
Win64 version:
https://openresty.org/download/openresty-1.21.4.2rc1-win64.zip
PGP for the Win64 zip file:
https://openresty.org/download/openresty-1.21.4.2rc1-win64.zip.asc
Win32 version:
https://openresty.org/download/openresty-1.21.4.2rc1-win32.zip
PGP for the Win32 zip file:
https://openresty.org/download/openresty-1.21.4.2rc1-win32.zip.asc
Version Highlights
- LuaJIT
- Avoid negation of signed integers in C that may hold INT*_MIN, since recent C compilers 'take advantage' of the undefined behavior, this completely changes the meaning of expressions like
(k == -k)
. - Correct fix for stack check when recording
BC_VARG
. - Fix trace join to
BC_JLOOP
originating fromBC_ITERN
. - Fix
math.floor()
andmath.ceil()
. - Many fixes imported from Mike Pall's upstream https://github.com/luajit/luajit repository.
- Avoid negation of signed integers in C that may hold INT*_MIN, since recent C compilers 'take advantage' of the undefined behavior, this completely changes the meaning of expressions like
- Implemented
monotonic_msec()
andmonotonic_time()
inresty.core.time
. - Apple Silicon FFI ABI limitation workaround.
- Reimplemented
tcpsock:sslhandshake
, coroutine wrapper, andngx.req.is_internal
with FFI. - SSL/TLS supports passphrase protected private key.
- Fixed a segmentation fault when get headers via
ngx.req.raw_header
with malformed requests. - Fixed potential null pointer dereference.
- lua-cjson module
empty_array
can not work on Apple because cjson did not compare light userdata address with masked address. Since we usedjson_lightudata_mask
when creating lightuserdata, same mask should be applied when comparing the return oftouserdata
.
- headers-more-nginx-module module
- Fixed crash while accessing uninitialized pointer.
- lua-resty-websocket module
- Added mtls client cert support.
- lua-resty-memcached module
- Implemented
init_pipeline()
,commit_pipeline()
, andcancel_pipeline()
.
- Implemented
Full Change logs
Complete change logs since the last (formal) release, 1.21.4.1:
upgraded lua-nginx-module to v0.10.24
- bugfix:
ngx_http_lua_ffi_worker_pids()
should also be supported on windows. Thanks lijunlong for the patch. - bugfix: comparison of integer expressions of different signedness. Thanks usrtax for the patch.
- feature: introduced new API
ngx_http_lua_get_lua_http10_buffering()
. Thanks Xiaobo Liu for the patch. - feature: add new APIs to manipulate the coctx. Thanks Xiaobo Liu for the patch.
- bugfix: the cleanup of the HTTP request will not be called for the request created by
ngx_http_lua_create_fake_request()
. Thanks lijunlong for the patch. - bugfix: cosocket did not exit when
worker_shutdown_timeout
handler is called. Thanks lijunlong for the patch. - feature: implement HTTP 3.0 support for
ngx.req.http_version()
. Thanks Yu.Zhu for the patch. - bugfix: fix receiveuntil rest bytes count. Thanks ZongRun for the patch.
- bugfix: add a timed recycling child process as a last resort. Thanks lijunlong for the patch.
- feature: add new FFI API
ngx_http_lua_ffi_msec
. Thanks lijunlong for the patch. - bugfix: did not wakeup coroutine when worker thread finished. Thanks kingluo for the patch.
- feature: add
ngx_http_lua_ffi_worker_pids
to get all workers pid map. Thanks attenuation for the patch. - bugfix:
run_worker_thread
arg is self-reference. Thanks fesily for the patch. - feature: introduced new API
tcpsock:bind()
. Thanks doujiang24 for the patch. - feature: add
shdict
APIs into worker thread. Thanks jinhua luo for the patch. - bugfix: set flags for Darwin arm64. Thanks lijunlong for the patch.
- bugfix: improved handling of multiple headers changed in nginx 1.23.0. Thanks Hiroaki Nakamura for the patch.
- change: increased the maximum size to 65536 for the udp datagram. Thanks lijunlong for the patch.
- optimize: destroy pipe proc when freeing the request. Thanks lijunlong for the patch.
- optimize: add error log when closing the pipe failed. Thanks lijunlong for the patch.
- bugfix: fix potential null pointer dereference found by Coverity. Thanks Ilya Shipitsin for the patch.
- optimize: fixed dead code found by Coverity. Thanks Ilya Shipitsin for the patch.
- feature: in
content_by_lua_file
, return 503 for file read error. Thanks jizhuozhi for the patch. - bugfix: Apple Silicon FFI ABI limitation workaround. Thanks Chrono for the patch.
- bugfix: failed to compile when nginx https is disabled. Thanks lijunlong for the patch.
- feature: add
server_rewrite_by_lua*
. Thanks xiaobiaozhao for the patch. - cosocket: add function
tcpsock:setclientcert
, reimplementedtcpsock:sslhandshake
with FFI. Thanks Datong Sun for the patch. - optimize: use
ngx_hash_t
to optimize the built-in header look-up process forngx.header.HEADER
. Thanks lijunlong for the patch. - feature: add FFI implementation for
ngx.arg
getter. Thanks 罗泽轩 for the patch. - bugfix: fixed size of the array when initialized in the
init_worker_by*
phase. Thanks Jiahao Wang for the patch. - bugfix: ambiguous error message 'connection in dubious state' when connection is closed. Thanks lijunlong for the patch.
- bugfix: passing the wrong chunkname argument to
luaL_loadbuffer
. Thanks lijunlong for the patch. - optimize: change lua chunkname to config filename and line number for
{init,header_filter,body_filter}_by_lua_block
and so on. Thanks lijunlong for the patch. - change: the error message should use the first line rather than the last line of the code block when loading lua code block fails. Thanks lijunlong for the patch.
- bugfix: segment fault when get header via
ngx.req.raw_header
with malformed requests. Thanks Gordon McKinney for the patch. - change: remove useless code for get old_cpath. Thanks Tinglong Yang for the patch.
- bugfix:
ngx.run_worker_thread
injected API into the wrong table. Thanks jinhua luo for the patch. - feature: add API to fetch raw nginx SSL pointer of the downstream request. Thanks James Callahan for the patch.
- feature: SSL/TLS: support for passphrase protected key. Thanks guanglinlv for the patch.
- feature: expose the 'Last-Modified' response header as
ngx.header["Last-Modified"]
. Thanks cbeckr for the patch. - bugfix: posted event handler was called after event memory was freed. Thanks lijunlong for the patch.
- optimize: don't calculate hash when clearing the request header. Thanks spacewander for the patch.
- feature: check the number of parameters for
ngx.thread.wait
. Thanks tan jinhua for the patch. - change: use
nil
instead offalse
whenlpush
&rpush
overflow. Thanks yang.yang for the patch. - feature: prevent calling
ngx.exit()
with invalid values. Thanks Thibault Charbonnier for the patch. - feature: added FFI-based function
ngx_http_lua_ffi_req_is_internal
. Thanks chronolaw for the patch. - feature: added http const
HTTP_NOT_IMPLEMENTED
. Thanks Landon Manning for the patch.
- bugfix:
upgraded stream-lua-nginx-module to v0.0.13
- bugfix:
ngx_stream_lua_ffi_worker_pids()
should also be supported on windows. Thanks lijunlong for the patch. - feature: introduced new API
ngx_stream_lua_ffi_monotonic_msec
. Thanks lijunlong for the patch. - feature: add
CONTEXT_YIELDABLE
constant to stream subsys. Thanks 罗泽轩 for the patch. - bugfix: wrong size for the pending timers. Thanks lijunlong for the patch.
- feature: add
ngx_stream_lua_ffi_worker_pids
to get all workers pid map. Thanks attenuation for the patch. - bugfix: Apple Silicon FFI ABI limitation workaround. Thanks Chrono for the patch.
- feature: SSL/TLS supports passphrase protected private key. Thanks guanglinlv for the patch.
- bugfix: posted event handler was called after event memory was freed. Thanks lijunlong for the patch.
- bugfix:
upgraded lua-resty-core to v0.1.26
- bugfix:
ngx.worker.pids()
should also be supported on windows. Thanks lijunlong for the patch. - optimize: avoid NYI in the
get_string_buf
function. Thanks Jiahao Wang for the patch. - feature: implemented
monotonic_msec()
andmonotonic_time()
inresty.core.time
. Thanks lijunlong for the patch. - bugfix: get
ngx_lua_version
before 'not ngx.config'. Thanks willmafh for the patch. - feature: add
ngx.worker.pids
to get all workers pid map. Thanks attenuation for the patch. - optimize: destroy pipe when freeing request. Thanks lijunlong for the patch.
- optimize: localize
pcall
function inbase.lua
to improve performance. Thanks Chrono for the patch. - feature: implemented
get_supported_versions()
inclienthello
phase. Thanks Attenuation for the patch. - bugfix: Apple Silicon FFI ABI limitation workaround. Thanks Chrono for the patch.
- bugfix: did not get the correct error message for when require module. Thanks lijunlong for the patch.
- feature: added support for
ssl_server_rewrite_by_lua
. Thanks lijunlong for the patch. - cosocket: add function
tcpsock:setclientcert
, reimplementedtcpsock:sslhandshake
with FFI. Thanks Datong Sun for the patch. - feature: provide a way to reuse table in
ngx.req.get_uri_args
. Thanks 罗泽轩 for the patch. - feature: add FFI implementation for
ngx.arg
getter. Thanks 罗泽轩 for the patch. - optimize: localize
bit.bor
inclienthello.lua
to improve performance. Thanks Chrono for the patch. - optimize: reimplemented the coroutine wrapper using the FFI API. Thanks lijunlong for the patch.
- feature: support passphrase protected private key. Thanks guanglinlv for the patch.
- feature: reimplemented
ngx.req.is_internal
with LuaJIT FFI. Thanks chronolaw for the patch. - optimize: removed the extra return. Thanks lijunlong for the patch.
- bugfix:
upgraded luajit2 to v2.1-20230410
- Improve error reporting on stack overflow.
- Avoid negation of signed integers in C that may hold INT*_MIN.
- Correct fix for stack check when recording BC_VARG.
- Disable FMA by default. Use -Ofma or jit.opt.start("+fma") to enable.
- FFI: Fix dangling reference to CType. Improve checks.
- ARM64: Fix code generation for IR_SLOAD with typecheck + conversion.
- Avoid assertion in case of stack overflow from stitched trace.
- Ensure correct stack top for OOM error message.
- ARM64: Fix IR_SLOAD assembly.
- Fix trace join to BC_JLOOP originating from BC_ITERN.
- bugfix: fix
math.floor()
andmath.ceil()
. Thanks Aditya Bisht for the patch. - Add -F option to override filename in jit.bcsave (luajit -b).
- Patch luajit.pc with INSTALL_INC, if customized.
- LJ_GC64: Fix
lua_concat()
. - Prevent use of RTLD_DEFAULT when NO_RTLD_DEFAULT is defined.
- Improve GC estimation for userdata with attached managed memory.
- Add missing GC steps to string buffer methods.
- x86/x64: Limit VLOAD fusion to simple cases.
- OSX/iOS/ARM64: Fix generation of Mach-O object files.
- Prevent trace start at BC_ITERL after compiled BC_ITERN.
- ARM64: Allow building with unwinding disabled.
- FFI: Fix sizeof expression in C parser for reference types.
- FFI: Fix ffi.alignof() for reference types.
- FFI: Allow ffi.metatype() for typedefs with attributes.
- OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file.
- LJ_GC64: Fix IR_VARG offset for fixed number of results.
- x86/x64: Fix math.ceil(-0.9) result sign.
- Make embedded bytecode readable and forward-compatible.
- Update console build instructions.
- Avoid zero-sized arrays in jit_State.
- Don't use jit_State during build with JIT disabled.
- DynASM/ARM64: Fix LSL/BFI* encoding with variable registers.
- Fix ITERN loop detection when hook checks are enabled.
- Prevent C compiler undefined-behavior optimization.
- Fix alias analysis for table length forwarding.
- Fix loop initialization in table.foreach().
- LJ_GC64: Fix HREFK optimization.
- Fix recording of __concat metamethod.
- Cleanup of system and architecture support docs.
upgraded resty-cli to v0.29
- change: save the original
ngx.exit
tongx.orig_exit
. Thanks lijunlong for the patch.
- change: save the original
upgraded lua-cjson to 2.1.0.11
- bugfix: empty_array can not work on Apple because cjson did not compare light userdata address with masked address. Thanks Datong Sun for the patch.
- bugfix: windows luarocks doesn't export
cjson.safe
. Thanks momoterraw for the patch.
upgraded headers-more-nginx-module to v0.34
- bugfix: nginx crash when accessing uninitialized pointer. Thanks lijunlong for the patch.
- bugfix: update handling of multiple headers changed in nginx 1.23.0. Thanks Hiroaki Nakamura for the patch.
- bugfix: fixed build error with nginx >= 1.23.0. Thanks somni for the patch.
upgraded lua-resty-upstream-healthcheck to v0.07
- change: improved healthcheck status for prometheus. Thanks Jonas Badstübner for the patch.
- bugfix:
opts.host
has not been assigned to ctx. Thanks yueziii for the patch. - feature: allowing check on different port. Thanks Franck Lombardi for the patch.
- feature: add HTTPS health check. Thanks Yannic Rieger for the patch.
- feature: add prometheus metrics format status. Thanks Yannic Bastian Rieger for the patch.
upgraded opm to v0.0.7
- change: web: show the package installation command in the package details page. Thanks guisheng zhou for the patch.
- bugfix: 'install_dir' should not be
/usr/local/openresty/site
but/usr/local/openresty/bin
. Thanks lijunlong for the patch. - feature: web: support github login and deferred deletion. Thanks xlibor for the patch.
upgraded ngx_devel_kit to v0.3.2
- change: fixed a typo in
ndk_upstream_list.c
. Thanks lijunlong for the patch.
- change: fixed a typo in
upgraded lua-resty-lock to v0.09
- optimize: return setmetatable is NYI. Thanks lijunlong for the patch.
upgraded srcache-nginx-module to v0.33
- bugfix: update handling of cache_control changed in nginx 1.23.0. Thanks Hiroaki Nakamura for the patch.
- change: add status code 307, 308 to the default value of
srcache_store_statuses
. Thanks Jérémy Lal for the patch.
upgraded lua-resty-websocket to v0.10
- feature: add mtls client cert support. Thanks Qi for the patch.
- optimize: localize some
ngx.*
API to improve performance. Thanks Chrono for the patch. - bugfix: default to port 443 for wss urls. Thanks John Regan for the patch.
upgraded lua-resty-lrucache to v0.13
- optimize: remove NYI call in lurcache. Thanks lijunlong for the patch.
upgraded lua-resty-upload to v0.11
- feature: add an option to preserve body. Thanks Suika for the patch.
upgraded lua-resty-memcached to v0.17
- optimize: return
setmetatable
is NYI which can not be jit compiled. Thanks lijunlong for the patch. - optimize: reuse the
cmd_tab
to minimize the gc. Thanks lijunlong for the patch. - feature: implemented
{init,commit,cancel}_pipeline()
. Thanks syz for the patch. - feature: add support for connecting over TLS. Thanks Alessandro Ghedini for the patch.
- optimize: return
upgraded echo-nginx-module to v0.63
- optimize: fix potential null pointer dereference found by Coverity. Thanks Ilya Shipitsin for the patch.
- bugfix: fix potential null pointer dereference found by Coverity. Thanks Ilya Shipitsin for the patch.
- bugfix: fix minor issue found by Coverity in
src/ngx_http_echo_subrequest.c
. Thanks Ilya Shipitsin for the patch.
upgraded drizzle-nginx-module to v0.1.12
- bugfix: fix potential null pointer dereference found by Coverity. Thanks Ilya Shipitsin for the patch.
Testing
We have run extensive testing on our Amazon EC2 test cluster and ensured that all the components (including the Nginx core) play well together. The latest test report can always be found here:
We also always run our OpenResty Edge commercial software based on the latest open source version of OpenResty in our own global CDN network (dubbed "mini CDN") powering our openresty.org and openresty.com websites. See https://openresty.com/ for more details.
Community Support
See the Community Page.
Commercial Support
Commercial technical support and real-time noninvasive online monitoring and profiling solution is provided through the official OpenResty XRay product.
Feedback
Feedback on this release is more than welcome. Feel free to create new GitHub issues or send emails to one of our mailing lists.