OpenResty

Changes

Mon, 10 Jun 2013 22:33:17 GMT

permalink

ChangeLog1002008

Mon, 10 Jun 2013 18:54:00 GMT

permalink

Stable Release 1.2.8.6 - 10 June 2013

upgraded LuaJIT to 2.0.2.
- changes: http://luajit.org/changes.html

The following components are bundled:

LuaJIT-2.0.2
array-var-nginx-module-0.03rc1
auth-request-nginx-module-0.2
drizzle-nginx-module-0.1.5
echo-nginx-module-0.45
encrypted-session-nginx-module-0.03
form-input-nginx-module-0.07
headers-more-nginx-module-0.20
iconv-nginx-module-0.10
lua-5.1.5
lua-cjson-1.0.3
lua-rds-parser-0.05
lua-redis-parser-0.10
lua-resty-dns-0.09
lua-resty-memcached-0.11
lua-resty-mysql-0.13
lua-resty-redis-0.15
lua-resty-string-0.08
lua-resty-upload-0.08
memc-nginx-module-0.13rc3
nginx-1.2.8
ngx_coolkit-0.2rc1
ngx_devel_kit-0.2.18
ngx_lua-0.8.2
ngx_postgres-1.0rc2
rds-csv-nginx-module-0.05rc2
rds-json-nginx-module-0.12rc10
redis-nginx-module-0.3.6
redis2-nginx-module-0.10
set-misc-nginx-module-0.22rc8
srcache-nginx-module-0.21
xss-nginx-module-0.03rc9

Devel Release 1.2.8.5 - 23 May 2013

upgraded LuaNginxModule to 0.8.2.
- feature: added ngx.HTTP_MKCOL, ngx.HTTP_COPY, ngx.HTTP_MOVE, and other WebDAV request method constants; also added corresponding support to ngx.req.set_method and ngx.location.capture. thanks Adallom Roy for the patch.
- feature: allow injecting new user Lua APIs (and overriding existing Lua APIs) in the "ngx" table.
- bugfix: ngx.req.set_body_file() always enabled Direct I/O which caused the alert message "fcntl(O_DIRECT) ... Invalid argument" in error logs on file systems lacking the Direct I/O support. thanks Matthieu Tourne for reporting this issue.
- bugfix: buffer corruption might happen in ngx.req.set_body_file() when Nginx upstream modules were used later because ngx.req.set_body_file() incorrectly set r->request_body->buf to the in-file buffer which could get reused by ngx_http_upstream for its own purposes.
- bugfix: no longer automatically turn underscores (_) to dashes (-) in header names for ngx.req.set_header and ngx.req.clear_header. thanks aviramc for the report.
- bugfix: segmentation fault might happen in nginx 1.4.x when calling ngx.req.set_header on the Cookie request headers because recent versions of Nginx no longer always initialize r->headers_in.cookies. thanks Rob W for reporting this issue.
- bugfix: fixed the C compiler warning "argument 'nret' might be clobbered by 'longjmp' or 'vfork'" when compiling with Ubuntu 13.04's gcc 4.7.3. thanks jacky and Rajeev's reports.
- bugfix: temporary memory leaks might happen when using ngx.escape_uri, ngx.unescape_uri, ngx.quote_sql_str, ngx.decode_base64, and ngx.encode_base64 in tight Lua loops because we allocated memory in nginx's request memory pool for these methods.
- optimize: ngx.escape_uri now runs faster when the input string contains no special bytes to be escaped.
- testing: added custom test scaffold t::TestNginxLua which subclasses Test::Nginx::Socket. it supports the environment TEST_NGINX_INIT_BY_LUA which can be used to add more custom Lua code to the value of the init_by_lua directive in the Nginx configuration.
upgraded SrcacheNginxModule to 0.21.
- bugfix: responses with a status code smaller than all the status codes specified in the srcache_store_statuses directive were not skipped as expected. thanks Lanshun Zhou for the patch.
feature: applied the invalid_referer_hash patch to the Nginx core to make the $invalid_referer variable accessible in embedded dynamic languages like Perl and Lua. thanks Fry-kun for requesting this.
updated the dtrace patch for the Nginx core.
- print out more info about the Nginx in-file bufs in the tapset function ngx_chain_dump.

Devel Release 1.2.8.3 - 13 May 2013

applied the official patch for the nginx core to address the recent nginx security vulnerability CVE-2013-2070.

Devel Release 1.2.8.1 - 26 April 2013

upgraded the Nginx core to 1.2.8.
- see http://nginx.org/en/CHANGES-1.2 for changes.
upgraded LuaNginxModule to 0.8.1.
- feature: implemented the new timer API: the ngx.timer.at Lua function and two configure directives lua_max_pending_timers and lua_max_running_timers. thanks Matthieu Tourne for requesting this feature.
- feature: added the "U" regex option to the ngx.re API to mean enabling the UTF-8 matching mode but disabling UTF-8 validity check on the subject strings. thanks Lance Li for the patch.
- bugfix: setting ngx.header.etag could not affect other things reading the ETag response header (like the etag directive introduced in Nginx 1.3.3+). thanks Brian Akins for the patch.
- bugfix: when lua_http10_buffering is on, for HTTP 1.0 requests, ngx.exit(N) would always trigger the Nginx's own error pages when N >= 300. thanks Matthieu Tourne for reporting this issue.
- bugfix: modifying the Cookie request headers via ngx.req.set_header or ngx.req.clear_header did not update the Nginx internal data structure, r->headers_in.cookies, at the same time, which might cause issues when reading variables $cookie_COOKIE, for example. thanks Matthieu Tourne for the patch.
- bugfix: modifying the Via request header with ngx.req.set_header or ngx.req.clear_header did not update the special field r->headers_in.via when the ngx_gzip module was enabled.
- bugfix: modifying the X-Real-IP request header with ngx.req.set_header or ngx.req.clear_header did not update the special field r->headers_in.x_real_ip when the ngx_realip module was enabled. thanks Matthieu Tourne for the patch.
- bugfix: modifying the Connection request header via ngx.req.set_header or ngx.req.clear_header did not update the special internal field in the Nginx core, r->headers_in.connection_type. Thanks Matthieu Tourne for the patch.
- bugfix: modifying the User-Agent request header via ngx.req.set_header or ngx.req.clear_header did not update those special internal flags in the Nginx core, like r->headers_in.msie6 and r->headers_in.opera. Thanks Matthieu Tourne for the patch.
- bugfix: fixed several places in the header API where we should return NGX_ERROR instead of NGX_HTTP_INTERNAL_SERVER_ERROR.
upgraded SrcacheNginxModule to 0.20.
- bugfix: use of C global variables at the configuration phase would cause troubles when HUP reload failed.
upgraded HeadersMoreNginxModule to 0.20.
- bugfix: modifying the Cookie request headers via more_set_input_headers or more_clear_input_headers did not update the Nginx internal data structure, r->headers_in.cookies, at the same time, which might cause issues when reading variable $cookie_COOKIE, for example.
- bugfix: modifying the Via request header via more_set_input_headers or more_clear_input_headers did not update the special internal field in the Nginx core, r->headers_in.via, when the ngx_gzip module was enabled.
- bugfix: modifying the X-Real-IP request header via more_set_input_headers or more_clear_input_headers did not update the special internal field in the Nginx core, r->headers_in.x_real_ip, when the ngx_realip module was enabled.
- bugfix: modifying the Connection request header via more_set_input_headers or more_clear_input_headers did not update the special internal field in the Nginx core, r->headers_in.connection_type.
- bugfix: modifying the User-Agent request header via more_set_input_headers or more_clear_input_headers did not update those special internal flags in the Nginx core, like r->headers_in.msie6 and r->headers_in.opera.
- bugfix: fixed places where we should return NGX_ERROR instead of NGX_HTTP_INTERNAL_SERVER_ERROR.
feature: always enable debuginfo in the bundled LuaJIT 2.0.1 build and Lua 5.1.5 build to support Nginx Systemtap Toolkit.
bugfix: no longer pass -O0 to gcc when the --with-debug configure option is specified because gcc often generates bogus DWARF info when optimization is turned off.

See ChangeLog1002007 for change log for OpenResty 1.2.7.x.

Download

Mon, 10 Jun 2013 18:52:00 GMT

permalink

You can download the latest development or stable release tarball below. After that, you can follow the instructions on the Installation page to build and install it.

Releases

Development release

Development releases are for developers and brave users to try out new bug fixes and new features. Development releases are also considered production-ready.

ngx_openresty-1.2.8.5.tar.gz 2.9MB PGP Changes - 23 May 2013

Stable release

Stable releases are for really conservative users.

ngx_openresty-1.2.8.6.tar.gz 2.9MB PGP Changes - 10 June 2013

Legacy releases

Sometimes, users may want to download old releases.

ngx_openresty-1.2.7.8.tar.gz 2.9MB PGP Changes - 13 May 2013
ngx_openresty-1.2.7.6.tar.gz 2.9MB PGP Changes - 17 April 2013
ngx_openresty-1.2.6.6.tar.gz 2.9MB PGP Changes - 17 February 2013
ngx_openresty-1.2.4.14.tar.gz 2.9MB PGP Changes - 23 December 2012
ngx_openresty-1.2.3.8.tar.gz 2.9MB PGP Changes - 8 October 2012
ngx_openresty-1.2.1.14.tar.gz 2.9MB PGP Changes - 14 August 2012
ngx_openresty-1.0.15.10.tar.gz 2.7MB PGP Changes - 13 June 2012
ngx_openresty-1.0.11.28.tar.gz 2.7MB PGP Changes - 25 March 2012
ngx_openresty-1.0.10.48.tar.gz 2.6MB PGP Changes - 1 February 2012
ngx_openresty-1.0.10.44.tar.gz 2.5MB PGP Changes - 16 January 2012
ngx_openresty-1.0.10.24.tar.gz 2.5MB PGP Changes - 11 December 2011
ngx_openresty-1.0.9.10.tar.gz 2.5MB PGP Changes - 16 November 2011
ngx_openresty-1.0.8.26.tar.gz 2.5MB PGP Changes - 3 November 2011
ngx_openresty-1.0.6.22.tar.gz 2.4MB PGP Changes - 7 October 2011

See Installation for installation instructions.

Presentations

Sun, 09 Jun 2013 20:01:00 GMT

permalink

Here goes a list of slides that I used in my talks regarding OpenResty.

These slides are powered by the slides.htm slide-making engine based on AJAX.

Note: Please use the arrow keys or pageup/pagedown keys on your keyboard to switch slides.

Year 2013

sregex: matching Perl 5 regexes on data streams

This talk was given at YAPC::NA 2013 in Austin, TX, USA.

Talk summary: http://www.yapcna.org/yn2013/talk/4762

View slides in your web browser: http://agentzh.org/misc/slides/yapc-na-2013-sregex/

Download the slides as PDF: http://agentzh.org/misc/slides/yapc-na-2013-sregex.pdf

Flame Graphs for online performance profiling

This talk was given at YAPC::NA 2013 in Austin, TX, USA.

Talk summary: http://www.yapcna.org/yn2013/talk/4579

View slides in your web browser: http://agentzh.org/misc/slides/yapc-na-2013-flame-graphs/

Download the slides as PDF: http://agentzh.org/misc/slides/yapc-na-2013-flame-graphs.pdf

Year 2012

ngx_openresty: an Nginx ecosystem glued by Lua

This talk was given at Tech-Club Technical Salon held in the Xiamen city.

View slides in your web browser: http://agentzh.org/misc/slides/ngx-openresty-ecosystem/

Download the slides as PDF: http://agentzh.org/misc/slides/ngx-openresty-ecosystem.pdf

Scripting libdrizzle with Lua inside Nginx

This presentation was given at Percona Live MySQL Conference 2012 held in Santa Clara, CA, USA.

View slides in your web browser: http://agentzh.org/misc/slides/libdrizzle-lua-nginx/#2

Download the slides as PDF: http://agentzh.org/misc/slides/libdrizzle-lua-nginx.pdf

Year 2011

Applications of ngx_openresty and perl in lz.taobao.com

This talk was given at Beijing Perl Workshop 2011.

Watch the video online: http://v.ku6.com/show/TY8Vre59guTE_C8o.html

View slides in your web browser: http://agentzh.org/misc/slides/perl-lz-apps/

Download the slides as PDF: http://agentzh.org/misc/slides/perl-lz-apps.pdf

Year 2010

Introduction to nginx.conf scripting

This talk was given at the Beijing Perl Workshop 2010 April meeting and the Beijing OpenParty 2010 June event.

View slides in your web browser: http://agentzh.org/misc/slides/nginx-conf-scripting/

Download the slides as PDF: http://agentzh.org/misc/slides/nginx-conf-scripting.pdf

Please note that ngx_eval module is no longer recommended because we're in more favor of ngx_lua nowadays.

Recent developments in nginx.conf scripting

This talk was given at the Beijing OpenParty 2010 June event.

View slides in your web browser: http://agentzh.org/misc/slides/recent-dev-nginx-conf/

Download the slides as PDF: http://agentzh.org/misc/slides/recent-dev-nginx-conf.pdf

The state of the art of nginx.conf scripting

This talk was given at the ECUG 2010 event.

Watch the (Chinese-speech) video online: http://v.ku6.com/show/D00rqtnRwKzJdIsB.html

View the (English) slides in your web browser: http://agentzh.org/misc/slides/nginx-state-of-the-art/

Download the slides as PDF: http://agentzh.org/misc/slides/nginx-state-of-the-art.pdf

DynamicRoutingBasedOnRedis

Sun, 09 Jun 2013 01:49:00 GMT

permalink

This sample demonstrates how to use Redis to route incoming requests to different HTTP backends based on the requests' User-Agent header.

This demo uses the components LuaNginxModule and LuaRestyRedisLibrary enabled by default in OpenResty.

Here's the complete code listing for our nginx.conf:

worker_processes  2;
error_log logs/error.log info;

events {
    worker_connections 1024;
}

http {
    server {
        listen 8080;

        location / {
            resolver 8.8.4.4;  # use Google's open DNS server

            set $target '';
            access_by_lua '
                local key = ngx.var.http_user_agent
                if not key then
                    ngx.log(ngx.ERR, "no user-agent found")
                    return ngx.exit(400)
                end

                local redis = require "resty.redis"
                local red = redis:new()

                red:set_timeout(1000) -- 1 second

                local ok, err = red:connect("127.0.0.1", 6379)
                if not ok then
                    ngx.log(ngx.ERR, "failed to connect to redis: ", err)
                    return ngx.exit(500)
                end

                local host, err = red:get(key)
                if not host then
                    ngx.log(ngx.ERR, "failed to get redis key: ", err)
                    return ngx.exit(500)
                end

                if host == ngx.null then
                    ngx.log(ngx.ERR, "no host found for key ", key)
                    return ngx.exit(400)
                end

                ngx.var.target = host
            ';

            proxy_pass http://$target;
        }
    }
}

And then let's start the redis server on the localhost:6379:

$ ./redis-server  # default port is 6379

and feed some keys into this using the redis-cli utility:

   $ ./redis-cli
   redis> set foo apache.org
   OK
   redis> set bar nginx.org
   OK

And then let's test our nginx app!

   $ curl --user-agent foo localhost:8080
   <apache.org home page goes here>

   $ curl --user-agent bar localhost:8080
   <nginx.org home page goes here>

To further tune the performance, one could enable the connection pool for the redis connections, as documented in LuaRestyRedisLibrary's README.

Before you benchmarking your interface defined here, please ensure that you've raised the error log level to warn or notice in your nginx.conf file, as in

error_log logs/error.log warn;

because flushing error log is a very expensive operation and can hurt performance a lot.

ChangeLog1002007

Mon, 13 May 2013 19:27:00 GMT

permalink

Stable Release 1.2.7.8 - 13 May 2013

applied the official patch for the nginx core to address the recent nginx security vulnerability CVE-2013-2070.

Stable Release 1.2.7.6 - 17 April 2013

upgraded LuaNginxModule to 0.7.21.
- bugfix: boolean values in an array table were rejected with the exception "attempt to use boolean as query arg value" while encoding a Lua (hash) table as URL arguments. thanks Calin Don for reporting this issue.
- bugfix: ngx.req.raw_header() would return an empty string value when the default header buffer (c->buffer) can hold the request line but not the whole header. thanks KDr2 for reporting this issue.
upgraded EncryptedSessionNginxModule to 0.03.
- refactor: fixed typos in the source code: replacing "3des" with "aes"; thanks Edgar Liu for reporting this issue.
upgraded IconvNginxModule to 0.10.
- bugfix: failed to build on Solaris with the bogus error message "ngx_devel_kit is required to build ngx_iconv; please put it before ngx_iconv".

The following components are bundled:

LuaJIT-2.0.1 (with hotfix #1)
array-var-nginx-module-0.03rc1
auth-request-nginx-module-0.2
drizzle-nginx-module-0.1.5
echo-nginx-module-0.45
encrypted-session-nginx-module-0.03
form-input-nginx-module-0.07
headers-more-nginx-module-0.19
iconv-nginx-module-0.10
lua-5.1.5
lua-cjson-1.0.3
lua-rds-parser-0.05
lua-redis-parser-0.10
lua-resty-dns-0.09
lua-resty-memcached-0.11
lua-resty-mysql-0.13
lua-resty-redis-0.15
lua-resty-string-0.08
lua-resty-upload-0.08
memc-nginx-module-0.13rc3
nginx-1.2.7
ngx_coolkit-0.2rc1
ngx_devel_kit-0.2.18
ngx_lua-0.7.21
ngx_postgres-1.0rc2
rds-csv-nginx-module-0.05rc2
rds-json-nginx-module-0.12rc10
redis-nginx-module-0.3.6
redis2-nginx-module-0.10
set-misc-nginx-module-0.22rc8
srcache-nginx-module-0.19
xss-nginx-module-0.03rc9

Devel Release 1.2.7.5 - 9 April 2013

upgraded EchoNginxModule to 0.45.
- bugfix: $echo_client_request_headers would return the first part of the request body when request body was read before reading this variable.
- bugfix: $echo_client_request_headers might not work properly in a subrequest.
upgraded DrizzleNginxModule to 0.1.5.
- bugfix: compilation errors occurred with nginx 1.3.15. thanks Karl Blessing for reporting this issue.
- docs: fixed a typo in the sample code for $drizzle_thread_id reported by 岚偑/yy秋叶.
- docs: documented the config syntax for db passwords with special chars in them.
upgraded LuaNginxModule to 0.7.20.
- feature: now we allow the "0" time argument in ngx.sleep().
- feature: ngx.location.capture and ngx.location.capture_multi now return a lua table with the boolean field "truncated", which indicates whether the subrequest response body is truncated.
- bugfix: request hung when rewrite cycled in ngx.req.set_uri(uri, true) instead of throwing out an error log message and a 500 page properly. thanks Calin Don for the report.
- bugfix: assignment to ngx.status did not take effect when the response status line had already been generated (by ngx_proxy or others). thanks eqiuno for reporting this issue.
- bugfix: ngx.req.raw_header() would return the first part of the request body when request body was read before the call. thanks Matthieu Tourne for reporting this issue.
- bugfix: ngx.req.raw_header() might not work properly in a subrequest.
- bugfix: we would override the subrequest response status code later when error happens.
- bugfix: the debug log message "lua set uri jump to <uri>" generated by ngx.req.set_uri(uri, true) was wrong for "<uri>" was the old URI.
upgraded LuaRestyMySQLLibrary to 0.13.
- bugfix: 64-bit integer values in the MySQL packets (like last insert ids) could not be properly parsed due to the lack of support for 64-bit integers in LuaJIT's standard "bit" module. thanks Azure Wang for the patch implementing a temporary workaround.
- docs: various typo fixes from Tor Hveem and doledoletree.
upgraded LuaRestyMemcachedLibrary to 0.11.
- feature: added new method "touch" for the new Memcached command "touch". thanks merlin for the patch.
updated the upstream_truncation patch for the Nginx core.
- bugfix: chunked upstream response bodies were treated as 502. thanks Andy Yuan for the report.
- bugfix: request response status was changed to 502 after response header was sent in case of data truncation.
- bugfix: the "last buf" (i.e., bufs with "last_buf" or "last_in_chain" set) should not be sent downstream in case of upstream data truncation.
updated the dtrace patch for the Nginx core.
- feature: made the stap function ngx_chain_dump() print out info about the "last_buf" and "last_in_chain" flags in bufs and removed the old "<eof>" notation in the output.

Devel Release 1.2.7.3 - 24 March 2013

upgraded LuaNginxModule to 0.7.18.
- feature: implemented ngx.req.http_version() that returns the HTTP version number for the current request. thanks Matthieu Tourne for requesting this.
- feature: implemented the ngx.req.raw_header() function for returning the original raw HTTP protocol header string received by Nginx. thanks Matthieu Tourne for requesting this.
- feature: added new methods safe_set and safe_add to ngx.shared.DICT objects, which never override existing unexpired items but immediately return nil and a "no memory" string message when running out of storage. thanks Matthieu Tourne for requesting this.
- feature: datagram Unix domain sockets created by ngx.socket.udp() can now receive data from the other endpoint via "autobind" on Linux. thanks Dirk Feytons for the patch.
- change: the ngx.re.match, ngx.re.gmatch, ngx.re.sub, and ngx.re.gsub functions used to throw Lua exceptions aggressively for all the error conditions; now they just return an additional Lua string describing the error for almost all common errors instead of throwing exceptions, including pcre compile-time and exec-time failures. thanks Matthieu Tourne for requesting this change.
- bugfix: use of ngx.req.socket() could make socket reading hang infinitely when the request did not take a request body at all (that is, when the Content-Length request header is missing). thanks Matthieu Tourne for reporting this issue.
- bugfix: when a non-table value was specified for the "args" option in the ngx.location.capture or ngx.location.capture_multi call, memory invalid access might happen, which resulted in garbage data at least. thanks Siddon Tang for reporting this issue.
- bugfix: when the Lua code using UDP/TCP cosockets + resolver was run in a subrequest, the subrequest could hang due to missing calls to ngx_http_run_posted_requests in the UDP/TCP cosocket resolver handler. thanks Lanshun Zhou for reporting this issue.
- bugfix: ngx.socket.udp: memory leaks or invalid memory accesses might happen when the DNS resolver failed to resolve.
- bugfix: rewrite_by_lua_no_postpone can only work globally and did not reject contexts like "server" and "location" configuration blocks. thanks Matthieu Tourne for reporting this issue.
- bugfix: (large) in-file request bodies could not be inherited correctly by multiple subrequests issued by ngx.location.capture. thanks Matthieu Tourne for reporting this issue.
- bugfix: ngx.req.get_headers(limit, true) would still return header names in the pure lower-case form when the "limit" argument was an integer. thanks Matthieu Tourne for reporting this issue.
- bugfix: ngx.re.match: when the "D" regular expression option was specified, an empty Lua table would always be created even when the named capture was actually empty. thanks Matthieu Tourne for reporting this issue.
- docs: made it explicit that redirecting to external domains is also supported in ngx.redirect(). thanks Ron Gomes for asking.
upgraded EchoNginxModule to 0.44.
- bugfix: $echo_client_request_headers was evaluated to only the last part of the request header when "large header buffers" were used.
- change: preserve the trailing CR LF at the end of the whole HTTP protocol header returned by $echo_client_request_headers.
upgraded Redis2NginxModule to 0.10.
- feature: allow use of the request body data in Nginx variables for main requests by always reading the request body automatically; we used to always discard the request body just like the standard ngx_memcached module. thanks Ristona Hua for sharing this usage.
- docs: updated the docs for the limitations on Redis pub/sub. thanks LazyZhu for pointing out the potential confusions.
- docs: now we recommend LuaRestyRedisLibrary instead when being used with LuaNginxModule.
upgraded LuaRestyUploadLibrary to 0.08.
- bugfix: when multiple Content-Type request headers were given, a Lua exception would be thrown; now we just pick up the first one.
- docs: better error handling in the code sample. thanks wgm.china for the report.
feature: applied the variables_in_redis_pass patch to RedisNginxModule 0.3.6 to allow use of Nginx variables in the redis_pass directive. thanks Diptamay Sanyal for requesting this feature.
bugfix: applied Lanshun Zhou's run_posted_requests_in_resolver patch to the Nginx core: http://mailman.nginx.org/pipermail/nginx-devel/2013-March/003476.html
bugfix: applied the official hotfix #1 patch for the bundled LuaJIT 2.0.1.

Devel Release 1.2.7.1 - 22 February 2013

upgraded the Nginx core to 1.2.7.
- see http://nginx.org/en/CHANGES-1.2 for changes.
upgraded LuaJIT 2.0 to 2.0.1.
- see http://luajit.org/changes.html for changes.
upgraded LuaNginxModule to 0.7.16.
- optimize: removed the unsed size field and related computatins from the script engine for the ngx.re API.
- optimize: saved a little memory in the script engine for the ngx.re API.

See ChangeLog1002006 for change log for OpenResty 1.2.6.x.

About

Wed, 08 May 2013 06:25:00 GMT

permalink

This bundle is mainly maintained by Yichun Zhang (agentzh). It was originally sponsored by Taobao.com before the year 2011 and it has been mainly supported by CloudFlare since then.

Because most of the Nginx modules are developed by the bundle maintainers, it can be ensured
that all these modules are played well together.

The bundled software components are copyrighted by the respective copyright holders.

This web site is a TiddlyWiki page that is under the version control of a repository on GitHub:

http://github.com/agentzh/openresty.org

Feel free to fork it or just contact us to obtain a git commit bit if you want to edit and improve this site.

Installation

Tue, 07 May 2013 22:35:00 GMT

permalink
If you haven't downloaded the OpenResty source code tarball, please go to the Download page first.

Basically, building and installing OpenResty is as simple as

tar xzvf ngx_openresty-VERSION.tar.gz
cd ngx_openresty-VERSION/
./configure --with-luajit
make
make install

where VERSION should be replaced by a concrete version number of OpenResty, like 0.8.54.6.
If you have problems while building or want finer control over the building process, please read on.

Prerequisites

You should have perl 5.6.1+, libreadline, libpcre, libssl installed into your system. For Linux, you should also ensure that ldconfig is in your PATH environment.

Debian and Ubuntu users

You're recommended to install the following packages using apt-get:

apt-get install libreadline-dev libncurses5-dev libpcre3-dev libssl-dev perl make

Fedora and RedHat users

You're recommended to install the following packages using yum:

yum install readline-devel pcre-devel openssl-devel

Mac OS X (Darwin) users

You're recommended to install prerequisites PCRE using some package management tool, like Homebrew:

brew install pcre

Alternatively you can install PCRE from source all by yourself :)

After installing PCRE, you may need to specify the paths for PCRE's headers and libraries to your C compiler and linker:

$ ./configure \
             --with-cc-opt="-I/usr/local/Cellar/pcre/8.21/include" \
             --with-ld-opt="-L/usr/local/Cellar/pcre/8.21/lib" \
             ...

FreeBSD users

You need to install the following ports:

devel/gmake
security/openssl
devel/pcre

Solaris 11 users

You need to install the following packages from the official repository:

gcc-3
SUNWlibm

Usually it's just as simple as

pfexec pkg install gcc-3 SUNWlibm

Build OpenResty

Download

download the latest ngx_openresty tarball can be fetched from the Download page and unpack it like this:

tar xzvf ngx_openresty-VERSION.tar.gz

where VERSION should be replaced by real version numbers like 0.8.54.6.

./configure

Then enter the ngx_openresty-VERSION/ directory, and type the following command to configure:

./configure --with-luajit

By default, --prefix=/usr/local/openresty is assumed. The --with-luajit option is strongly recommended because LuaJIT 2.0 can give a nice performance boost in terms of both CPU time and memory footprint and its VM is also fully resume-able, unlike the default standard Lua 5.1 interpreter. Enabling LuaJIT 2.0 should work on Linux, Mac OS X, FreeBSD, Solaris, and others. You should only disable LuaJIT 2.0 when your platform does not support LuaJIT.

You can specify various options, as in

./configure --prefix=/opt/openresty \
            --with-luajit \
            --without-http_redis2_module \
            --with-http_iconv_module \
            --with-http_postgres_module \
            -j2

All of the Nginx configure file options can be used here, including --add-module=PATH. Try ./configure --help to see more options available.

Errors in running the ./configure script can be found in the file build/nginx-VERSION/objs/autoconf.err where VERSION should be replaced by a concrete version number of OpenResty, like 0.8.54.6.

Notes for Solaris users

For Solaris, it's common to install libraries like OpenSSL to /lib, so when it complaints about missing OpenSSL and you have indeed already installed it, specify the --with-ld-opt='-L/lib' option.

make

Now you can compile everything up using the command

make

If your machine has multiple cores and your make supports the jobserver feature, you can compile things in parallel like this:

make -j2

assuming you have 2 CPU cores.

make install

If all the previous steps go without problems, you can install OpenResty into your system by typing the command

make install

On Linux, it often requires sudo to gain root access.

Resources

Fri, 15 Mar 2013 19:07:00 GMT

permalink

English Articles

Scaling TextRazor in the Cloud with Nginx and Lua, by Toby.

http://www.textrazor.com/blog/2013/03/scaling-textrazor-in-the-cloud-with-nginx-and-lua.html

Augmenting APIs with Nginx and Lua, by Raimon Grau

http://3scale.github.com/2013/01/09/augment-your-api-without-touching-it/

Pushing Nginx to its limit with Lua, by Matthieu Tourne

http://blog.cloudflare.com/pushing-nginx-to-its-limit-with-lua

Yak Shaving: Adding OAuth Support to Nginx via Lua, by Jose Diaz-Gonzalez

http://seatgeek.com/blog/dev/oauth-support-for-nginx-with-lua

My beautiful dark twisted reverse-proxy LRU cache, by Mike Ferrier

http://mikeferrier.com/2011/05/14/my-beautiful-dark-twisted-reverse-proxy-LRU-cache/

Day 41 - Setting up ngx_openresty (WAS: testing Test::Nginx) , by Antoine Bonavita

http://www.nginx-discovery.com/2011/03/day-41-setting-up-ngxopenresty-was.html

Chinese Articles

An introduction to OpenResty, by Wang Dong

http://wdicc.com/intro-openresty/

Nginx 3rd-Party Module Experiments Journal, by smallfish

http://chenxiaoyu.org/2011/10/30/nginx-modules.html

Recommending OpenResty - An Nginx Version with Fully Capable Addons, by Wendal Chen

http://wendal.net/338.html

Constructing Efficient and Transparent Caching Mechanism with MemcNginxModule and SrcacheNginxModule, by Zhang Yang

http://www.codinglabs.org/html/nginx-memc-and-srcache.html

The Nginx Session Module, by smallfish

http://chenxiaoyu.org/2011/11/09/nginx-session.html

Nginx-Lua HTTP 401 Basic Authentication, by smallfish

http://chenxiaoyu.org/2012/02/08/nginx-lua-401-auth.html

Using ngx_openresty to create an AJAX-style pager for data listing, by lhmwzy

http://blog.163.com/lhmwzy@126/blog/static/64215736201212384413704/

Nginx Variable Tutorials, by agentzh

http://blog.sina.com.cn/openresty

Nginx & Lua, by laowang

http://huoding.com/2012/08/31/156

Implementing a simple server-push solution, by laowang

http://huoding.com/2012/09/28/174

MarkupPostBody

Thu, 07 Mar 2013 06:42:00 GMT

<p/><p/>
<script type="text/javascript">

var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-24724965-1']);
_gaq.push(['_setDomainName', 'openresty.org']);
_gaq.push(['_trackPageview']);

var ga_f = function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
};
setTimeout(ga_f, 200);

</script>

ChangeLog1002006

Mon, 18 Feb 2013 03:06:00 GMT

permalink

Devel Release 1.2.6.6 - 17 February 2013

upgraded LuaNginxModule to 0.7.15.
- bugfix: the original Lua VM error messages might get lost in case of Lua code crashes when user coroutines were used. thanks Dirk Feytons for the report.
- diagnose: added more info about r->main->count to the debugging logs.
- style: massive coding style fixes according to the Nginx coding style.

The following components are bundled:

LuaJIT-2.0.0
array-var-nginx-module-0.03rc1
auth-request-nginx-module-0.2
drizzle-nginx-module-0.1.4
echo-nginx-module-0.42
encrypted-session-nginx-module-0.02
form-input-nginx-module-0.07
headers-more-nginx-module-0.19
iconv-nginx-module-0.10rc7
lua-5.1.5
lua-cjson-1.0.3
lua-rds-parser-0.05
lua-redis-parser-0.10
lua-resty-dns-0.09
lua-resty-memcached-0.10
lua-resty-mysql-0.12
lua-resty-redis-0.15
lua-resty-string-0.08
lua-resty-upload-0.07
memc-nginx-module-0.13rc3
nginx-1.2.6
ngx_coolkit-0.2rc1
ngx_devel_kit-0.2.18
ngx_lua-0.7.15
ngx_postgres-1.0rc2
rds-csv-nginx-module-0.05rc2
rds-json-nginx-module-0.12rc10
redis-nginx-module-0.3.6
redis2-nginx-module-0.09
set-misc-nginx-module-0.22rc8
srcache-nginx-module-0.19
xss-nginx-module-0.03rc9

Devel Release 1.2.6.5 - 8 February 2013

upgraded SrcacheNginxModule to 0.19.
- bugfix: HEAD and conditional GET requests would still fall back to content handler execution (leading to backend accesses) even in case of a cache hit. thanks Wang Lichao for reporting this issue.
- style: massive coding style fixes.
upgraded LuaRestyUploadLibrary to 0.07.
- bugfix: the boundary string could not be parsed if no space was present before the boundary=xxx parameter in the Content-Type request header. thanks chenshu for reporting this issue.

Devel Release 1.2.6.3 - 3 February 2013

upgraded LuaNginxModule to 0.7.14.
- feature: implemented named subpattern support in ngx.re.match, ngx.re.gmatch, ngx.re.sub, and ngx.re.gsub; also added new regex option D to allow duplicate named subpattern names. thanks Ray Bejjani for the patch.
- feature: implemented the J regex option for the PCRE Javascript compatible mode in the ngx.re API. thanks lhmwzy for requesting this.
- feature: setting ngx.header.HEADER after sending out the response headers now only produced an error message in the Nginx error logs and does not throw out a Lua exception. this should be handy for Lua development. thanks Matthieu Tourne for requesting this.
- feature: automatic Lua 5.1 interpreter detection on OpenBSD 5.2. thanks Ilya Shipitsin for the patch.
- refactor: when the Nginx core fails to send the "100 Continue" response in case of the "Expect: 100-continue" request header (or just running out of memory), ngx.req.read_body() will no longer throw out a "failed to read request body" Lua error but will just terminate the current request and returns the 500 error page immediately, just as what the Nginx core currently does in this case.
- bugfix: because of the recent API behaviour changes in nginx 1.2.6+ and 1.3.9+, the "http request count is zero" alert might happen when ngx.req.read_body() was called to read the request body and Nginx failed to send out the "100 Continue" response (like client connection early abortion and etc). thanks stonehuzhan for reporting this issue.
- bugfix: setting the "eof" argument (i.e., ngx.arg[2]) in body_filter_by_lua* for a subrequest could truncate the main request's response data stream.
- bugfix: in body_filter_by_lua*, the "eof" argument (i.e., ngx.arg[2]) was never set in Nginx subrequests.
- bugfix: for nginx 1.3.9+ compatibility, we return an error while using ngx.req.socket() to read the chunked request body (for now), because chunked support in the downstream cosocket API is still a TODO.
- bugfix: for nginx 1.3.9+ compatibility, rewrite_by_lua* or access_by_lua* handlers might hang if the request body was read there, because the Nginx core now overwrites r->write_event_handler to ngx_http_request_empty_handler in its ngx_http_read_client_request_body API.
- bugfix: for nginx 1.3.9+ compatibility, we now throw an error in ngx.req.init_body(), ngx.req.set_body_data(), and ngx.req.set_body_file() when calling them without calling ngx.req.read_body() or after calling ngx.req.discard_body().
- bugfix: a compilation error would happen when building with an Nginx core patched by the SPDY patch 58_1.3.11 because the patch had removed a request field from the Nginx core. thanks Chris Lea for reporting this.
- bugfix: we did not get the request reference counter (i.e., r->main->count) right when lua_need_request_body was turned on and nginx versions older than 1.2.6 or 1.2.9 were used.
- optimize: we no longer traverse the captured body chain everytime we append a new link to it in ngx.location.capture and ngx.location.capture_multi.
- docs: documented the ngx.quote_sql_str API.
upgraded SrcacheNginxModule to 0.18.
- bugfix: we might serve a truncated srcache_fetch subrequest's response body as the cached response.
upgraded EchoNginxModule to 0.42.
- feature: the echo_after_body directive is now enabled in Nginx subrequests (again).
- bugfix: we did not set the "last_in_chain" buffer flag when echo_after_body was used in subrequests.
upgraded FormInputNginxModule to 0.07.
- bugfix: Nginx might hang when it failed to send the "100 Continue" response for Nginx versions older than 1.2.6 (and those older than 1.3.9 in the 1.3.x series).
upgraded NginxDevelKit ot 0.2.18.
- bugfix: various fixes for C89 compliance. also stripped some line-trailing spaces.
- bugfix: guard macros were missing in the ndk_set_var.h header file.
- bugfix: the ndk_string submodule failed to compile with gcc 4.6. thanks Jon Kolb for the patch.
- bugfix: the ndk_set_var example did not use the new way in its config file. thanks Amos Wenger for the patch.
- docs: fixes in README to reflect recent changes. thanks Amos Wenger for the patch.
applied Ruslan Ermilov's resolver_wev_handler_segfault_with_poll patch to the Nginx core bundled. see the related nginx-devel thread for details.
excluded the allow_request_body_updating patch from the Nginx core bundled.

Devel Release 1.2.6.1 - 4 January 2013

upgraded the Nginx core to 1.2.6.
- see http://nginx.org/en/CHANGES-1.2 for changes.
upgraded LuaNginxModule to 0.7.13.
- bugfix: ngx.decode_args() might result in Lua string storage corruption. thanks Xu Jian for the report and Kindy Lin for the patch.
- bugfix: using a key with underscores in ngx.header.KEY resulted in Lua string storage corruption. thanks rkearsley for reporting this issue.
- bugfix: accessing ngx.var.VARIABLE allocated temporary memory buffers in the request memory pool, which could lead to unnecessarily large memory footprint; now it allocates such buffers via Lua GC.
- feature: automatically detect LuaJIT 2.0 on FreeBSD by default. thanks rkearsley for the patch.
- docs: explained why local foo = require "foo" is required for loading a Lua module. thanks rkearsley for asking.
- docs: fixed a typo in the code sample for tcpsock:receiveuntil(). thanks Yecheng Fu for the patch.
- docs: fixed a typo in the Lua code sample for ngx.re.gmatch (we forgot to add do there). thanks Guo Yin for reporting this issue.
upgraded LuaRestyUploadLibrary to 0.06.
- optimize: use the pure lower-case form of the key content-type to index the headers table returned by ngx.req.get_headers() so as to avoid the overhead of calling the __index metamethod.
upgraded SrcacheNginxModule to 0.17.
- bugfix: srcache_store would emit the misleading error message "srcache_store: skipped because response body truncated: N > 0" for HEAD requests (because a HEAD request's response never carries a body); now it just skips such responses silently. thanks Yang Jin for reporting this issue.
bugfix: when relative paths were used in --with-zlib=DIR, --with-libatomic=DIR, --with-md5=DIR, and --with-sha1=DIR, the build system of Nginx could not find DIR at all. thanks LazyZhu for reporting this issue.

See ChangeLog1002004 for change log for OpenResty 1.2.4.x.

UsingLuaRocks

Fri, 08 Feb 2013 10:44:00 GMT

permalink

This sample demonstrates usage of LuaRocks with OpenResty. It's been tested on Linux and Mac OS X, with the standard Lua interpreter or with LuaJIT.

LuaRocks is a deployment and management system for Lua modules. LuaRocks allows one to install Lua modules as self-contained packages called "rocks", which also contain version dependency information.

We assume that you have installed OpenResty into the default location, i.e., /usr/local/openresty. You can adjust the paths in this sample according to the actual installation prefix of your OpenResty installation. If you haven't installed OpenResty yet, check out the Download and Installation pages.

Install LuaRocks

First of all, let's install LuaRocks:

Download the latest LuaRocks tarball from http://www.luarocks.org/en/Download. As of this writing, the latest version is 2.0.12, and we'll use this version throughout this sample.

wget http://luarocks.org/releases/luarocks-2.0.12.tar.gz
tar -xzvf luarocks-2.0.12.tar.gz
cd luarocks-2.0.12/
./configure --prefix=/usr/local/openresty/luajit \
    --with-lua=/usr/local/openresty/luajit/ \
    --lua-suffix=jit \
    --with-lua-include=/usr/local/openresty/luajit/include/luajit-2.0
make
sudo make install

Install the Lua MD5 library with LuaRocks

In this sample, we'll use the Lua MD5 library to serve as an example, so let's install it with LuaRocks:

sudo luarocks install md5

Configuring our OpenResty application

Let's change the current directory to /usr/local/openresty/nginx/:

cd /usr/local/openresty/nginx/

Next, edit the conf/nginx.conf file to the following contents with your favorite text editor (like vim or emacs):

worker_processes  1;   # we could enlarge this setting on a multi-core machine
error_log  logs/error.log warn;

events {
    worker_connections  1024;
}

http {
    lua_package_path 'conf/?.lua;;';

    server {
        listen       80;
        server_name  localhost;

        location = /luarocks {
            content_by_lua '
                local foo = require("foo")
                foo.say("hello, luarocks!")
            ';
        }
    }
}

Finally, create the following two Lua module files conf/foo.lua

-- conf/foo.lua

module("foo", package.seeall)

local bar = require "bar"

ngx.say("bar loaded")

function say (var)
    bar.say(var)
end

and conf/bar.lua

-- conf/bar.lua

module("bar", package.seeall)

local rocks = require "luarocks.loader"
local md5 = require "md5"

ngx.say("rocks and md5 loaded")

function say (a)
    ngx.say(md5.sumhexa(a))
end

Start the Nginx server

Now we start the Nginx server with our app:

ulimit -n1024   # increase the maximal fd count limit per process
./sbin/nginx

If you have already started the Nginx server, then stop it before starting it:

./sbin/nginx -s stop

Test our app

Now we can test our app via the curl utility or any HTTP compliant clients like a web browser:

curl http://localhost/luarocks

we could get the following outputs at the first run:

rocks and md5 loaded
bar loaded
85e73df5c41378f830c031b81e4453d2

then at the second run:

85e73df5c41378f830c031b81e4453d2

The output changed because LuaNginxModule by default caches already loaded Lua modules and those outputing code run at Lua module loading time will no longer be run.

Now let's do some benchmark:

ab -c10 -n50000 http://127.0.0.1/luarocks

On my Thinkpad T400 laptop (Core2Duo T9600 CPU), it yields

Server Software:        ngx_openresty/1.0.4.2rc10
Server Hostname:        localhost
Server Port:            80

Document Path:          /luarocks
Document Length:        33 bytes

Concurrency Level:      10
Time taken for tests:   3.052 seconds
Complete requests:      50000
Failed requests:        0
Write errors:           0
Total transferred:      9400188 bytes
HTML transferred:       1650033 bytes
Requests per second:    16380.48 [#/sec] (mean)
Time per request:       0.610 [ms] (mean)
Time per request:       0.061 [ms] (mean, across all concurrent requests)
Transfer rate:          3007.41 [Kbytes/sec] received

Note that the throughput is achieved by a single nginx worker process. While doing such benchmark on your own, just be careful about error log level settings in your nginx.conf and not to run out of dynamic port range on your local machine, or it'll be significantly slow after a short of period of time.

Known issues

Pior to OpenResty 1.0.4.2rc10, it's known that turning lua_code_cache on will cause LuaRocks atop LuaNginxModule to throw out the following exception in error.log:

lua handler aborted: runtime error: stack overflow

If you're using any version of OpenResty before 1.0.4.2rc10, please consider upgrading.

StyleSheet

Wed, 23 Jan 2013 04:49:00 GMT

body {
font-size: 1.2em;
}

ChangeLog1002004

Sun, 23 Dec 2012 22:52:00 GMT

permalink

Stable Release 1.2.4.14 - 23 December 2012

upgraded LuaNginxModule to 0.7.9.
- bugfix: assignment to ngx.status would always be overridden by the later ngx.exit() calls for HTTP 1.0 requests if lua_http10_buffering is on (the default setting). thanks chenshu for reporting this issue.
- bugfix: there was a typo in the error message when accessing an Nginx variable that has not been defined.
- docs: documented the request body automatic inheritance behaviour in ngx.location.capture.
- docs: fixed incorrect dates shown in the code samples for ngx.http_time and ngx.parse_http_time. thanks Gosuke Miyashita for the patch.
upgraded LuaRestyUploadLibrary to 0.05.
- bugfix: unexpected runtime exceptions would be thrown when resty.upload met a in-part header field line or a terminating boundary line that was too long. this bug had appeared in LuaRestyUploadLibrary 0.04 and OpenResty 1.2.4.7.
- bugfix: resty.upload could not parse Content-Type request header values like boundary="simple boundary", that is, with double quotes around the boundary value.
- optimize: marked internal auxiliary functions as Lua local functions.

The following components are bundled:

LuaJIT-2.0.0
array-var-nginx-module-0.03rc1
auth-request-nginx-module-0.2
drizzle-nginx-module-0.1.4
echo-nginx-module-0.41
encrypted-session-nginx-module-0.02
form-input-nginx-module-0.07rc5
headers-more-nginx-module-0.19
iconv-nginx-module-0.10rc7
lua-5.1.5
lua-cjson-1.0.3
lua-rds-parser-0.05
lua-redis-parser-0.10
lua-resty-dns-0.09
lua-resty-memcached-0.10
lua-resty-mysql-0.12
lua-resty-redis-0.15
lua-resty-string-0.08
lua-resty-upload-0.05
memc-nginx-module-0.13rc3
nginx-1.2.4
ngx_coolkit-0.2rc1
ngx_devel_kit-0.2.17
ngx_lua-0.7.9
ngx_postgres-1.0rc2
rds-csv-nginx-module-0.05rc2
rds-json-nginx-module-0.12rc10
redis-nginx-module-0.3.6
redis2-nginx-module-0.09
set-misc-nginx-module-0.22rc8
srcache-nginx-module-0.16
xss-nginx-module-0.03rc9

Devel Release 1.2.4.13 - 11 December 2012

upgraded LuaNginxModule to 0.7.8.
- bugfix: ngx.req.set_body_file() might lead to memory issues because it directly used the storage of Lua strings allocated by the Lua GC (we should have allocated a new memory block on the Nginx side and copy the string data over).
upgraded LuaRestyMySQLLibrary to 0.12.
- feature: convert the MySQL newdecimal typed fields to Lua numbers by default as requested by shedar.
- optimize: marked the internal Lua function _recv_packet as a local function.

Devel Release 1.2.4.11 - 8 December 2012

upgraded LuaNginxModule to 0.7.7.
- feature: added ngx.req.start_time() to return the request starting time in seconds (the milliseconds part is the decimal part just as in ngx.now). thanks Matthieu Tourne for the patch.
- feature: setting ngx.status or calling ngx.exit(N) (where N >= 300) after sending out response headers no longer yields a Lua exception but only leaves an error message in the error.log file, which is useful for Lua land debugging. thanks Matthieu Tourne for requesting this.
- feature: the user can now call ngx.exit(444) to abort pending subrequests in other "light threads" from within a "light thread".
- feature: added new dtrace static probe http-lua-user-thread-wait.
- bugfix: ngx.location.capture and ngx.location.capture_multi might hang infinitely because the parent request might not be waken up right after the first time the post_subrequest callback was called.
- bugfix: the "light thread" object created by ngx.thread.spawn() or ngx.on_abort() might be prematurely collected by the Lua GC because we did not correctly register its coroutine object into the Lua registry table. this bug may crash the Lua VM and Nginx workers under load. thanks Zhu Dejiang for reporting this issue.
- bugfix: ngx.thread.wait() might hang infinitely when more than 4 user "light threads" are created in the same request handler due to the incorrect use of ngx_array_t for ngx_list_t. thanks Junwei Shi for reporting this issue.
- bugfix: when a user coroutine or user "light thread" dies with an error, our Lua backtrace dumper written in C may access one of its dead parent threads (if any) which could lead to segmentation faults.
- bugfix: ngx.exit(N) incorrectly threw out Lua exceptions when N was 408, 499, or 444 and the response header was already sent. thanks Kindy Lin for reporting this issue.
- bugfix: when the user callback function registered by ngx.on_abort() discarded the client abort event, the request would be aborted by force when the next client abort event happened.
- bugfix: an English typo in the error message for init_by_lua*.
applied slab_alloc_no_memory_as_info.patch to lower the log level of the error message "ngx_slab_alloc() failed: no memory" from "crit" to "info".
bugfix: the upstream_pipelining patch introduced a regression that when upstream_next is in action, Nginx might hang. thanks Kindy Lin for reporting this issue.
bugfix: include the latest changes in the LuaJIT 2.0 git repository (up to git commit 2ad9834d).

Devel Release 1.2.4.9 - 20 November 2012

upgraded LuaJIT to 2.0.0 final.
- change logs: http://luajit.org/changes.html
upgraded LuaNginxModule to 0.7.5.
- bugfix: ngx.req.clear_header() would result in memory invalid reads when removing the 21st, 41st, 61st (and etc) request headers. thanks Umesh Sirsiwal for reporting this issue.
- bugfix: ngx.log() would truncate the log messages which have null characters (\0) in it. thanks Wang Xi for reporting this issue.
- docs: eliminated the use of package.seeall in code samples and also explicitly discouraged the use of it.
- docs: documented the special case that client closes the connection before ngx.req.socket() finishes reading the whole body.
upgraded HeadersMoreNginxModule to 0.19.
- bugfix: more_clear_input_headers would result in memory invalid reads when removing the 21st, 41st, 61st (and etc.) request headers. thanks Umesh Sirsiwal for reporting this issue.
- docs: fixed an issue in the sample code that tried to clear Transfer-Encoding which cannot actually be cleared. thanks koukou73gr.
upgraded LuaRestyStringLibrary to 0.08.
- bugfix: the new() method in the resty.aes module might use a random key when the method option is omitted in the hash table argument. thanks wsser for the patch.
- feature: we now return a second string describing the error when either iv or key is bad.
bugfix: ./configure --with-pcre=PATH did not accept relative paths as PATH. thanks smallfish for reporting this issue.

Devel Release 1.2.4.7 - 11 November 2012

upgraded LuaJIT to 2.0.0rc3.
upgraded LuaNginxModule to 0.7.4.
- feature: added new directive lua_check_client_abort (default to off) for monitoring and processing the event that the client closes the (downstream) connection prematurely. thanks Zhu Dejiang for request this feature.
- feature: added new Lua API ngx.on_abort() which is used to register user Lua function callback for the event that the client closes the (downstream) connection prematurely. thanks Matthieu Tourne for suggesting this feature.
- feature: ngx.exit(N) can now abort pending subrequests when N = 408 (request time out) or N = 499 (client closed request) or N = -1 (error).
- bugfix: The TCP/stream cosocket's connect() method could not detect errors like "connection refused" when kqueue was used (on FreeBSD or Mac OS X, for example). thanks smallfish for reporting this issue.
- bugfix: reading operations on ngx.req.socket() did not return any errors when the request body got truncated; now we return the "client aborted" error.
upgraded LuaRestyDNSLibrary to 0.09.
- refactor: avoided using package.seeall in Lua module definitions, which improves performance and also prevents subtle bad side-effects.
- bugfix: a debugging output might be sent to stdout unexpectedly in some code path.
upgraded LuaRestyMemcachedLibrary to 0.10.
- refactor: avoided using package.seeall in Lua module definitions, which improves performance and also prevents subtle bad side-effects.
- docs: fixed typos in README. thanks cyberty for the patch.
upgraded LuaRestyRedisLibrary to 0.15.
- refactor: avoided using package.seeall in Lua module definitions, which improves performance and also prevents subtle bad side-effects.
- optimize: avoided using ipairs() which is slower than plain for i=1,N loops.
upgraded LuaRestyMySQLLibrary to 0.11.
- refactor: avoided using package.seeall in Lua module definitions, which improves performance and also prevents subtle bad side-effects.
- feature: now the new() method will return a string describing the error as the second return value in case of failures.
upgraded LuaRestyUploadLibrary to 0.04.
- refactor: avoided using package.seeall in Lua module definitions, which improves performance and also prevents subtle bad side-effects.
upgraded LuaRestyStringLibrary to 0.07.
- refactor: avoided using package.seeall in Lua module definitions, which improves performance and also prevents subtle bad side-effects.
- docs: typo-fixes in the code samples from Bearnard Hibbins.
bugfix: nginx upstream modules could not detect the "connection refused" error in time if kqueue was used; now we apply the upstream_test_connect_kqueue patch for the Nginx core.

Devel Release 1.2.4.5 - 30 October 2012

applied the official hotfix #1 patch to LuaJIT 2.0.0 beta11.
- see details here: http://www.freelists.org/post/luajit/Hotfix1-for-LuaJIT200beta11
upgraded LuaNginxModule to 0.7.3.
- feature: added the get_keys method for the shared memory dictionaries for fetching all the (or the specified number of) keys (default to 1024 keys). thanks Brian Akins for the patch.

Devel Release 1.2.4.3 - 17 October 2012

upgraded LuaJIT to 2.0.0 beta11.
- made LuaRestyRedisLibrary 27% faster, LuaRestyMemcachedLibrary 22% faster, and LuaRestyMySQLLibrary 15% faster, all for simple test cases loaded by ab, tested on Linux x86_64.
- all Lua APIs involved with I/O in LuaNginxModule are faster in general.
- complete change log: http://luajit.org/changes.html
upgraded LuaRestyMemcachedLibrary to 0.09.
- optimize: we now use Lua's own table.concat() to do string concatenation for all the memcached requests instead of relying on the cosocket API (on the C level) because calling the Lua C API is much slower especially when LuaJIT is in use. now for simple test cases loaded by ab -k -c10, we get 11.3% overall performance boost.
upgraded LuaNginxModule to 0.7.2.
- feature: now we can automatically detect the vendor-provided LuaJIT-2.0 package on Gentoo. thanks Il'ya V. Yesin for the patch. it is still recommended, however, to explicitly set the environments LUAJIT_INC and LUAJIT_LIB.

Devel Release 1.2.4.1 - 14 October 2012

upgraded the Nginx core to 1.2.4.
- see http://nginx.org/en/CHANGES-1.2 for changes.
upgraded LuaNginxModule to 0.7.1.
- feature: implemented the "light threads" API, which allows asynchronous concurrent processing within a single Nginx request handler, based on automatically-scheduled Lua coroutines. thanks Lee Holloway for requesting this feature.
  - http://wiki.nginx.org/HttpLuaModule#ngx.thread.spawn
  - http://wiki.nginx.org/HttpLuaModule#ngx.thread.wait
- bugfix: ngx.re.gsub() might throw out the exception attempt to call a string value when the replace argument was a Lua function and the subject string was large. thanks Zhu Maohai for reporting this issue.
- bugfix: older gcc versions might issue warnings like variable 'nrets' might be clobbered by 'longjmp' or 'vfork', like gcc 3.4.3 (for Solaris 11) and gcc 4.1.2 (for Red Hat Linux). thanks Wenhua Zhang for reporting this issue.
- docs: added a warning for ngx.var.VARIABLE that memory is allocated in the per-request memory pool. thanks lilydjwg.
- docs: made it clear why return is recommended to be used with ngx.exit(). thanks Antoine.
- docs: massive wording improvements from Dayo.
now we add SrcacheNginxModule before both LuaNginxModule and HeadersMoreNginxModule so that the former's output filter runs after those of the latter.

See ChangeLog1002003 for change log for OpenResty 1.2.3.x.

ChangeLog1002003

Mon, 08 Oct 2012 21:56:00 GMT

permalink

Stable Release 1.2.3.8 - 8 October 2012

This release is essentially the same as the development version 1.2.3.7.

The following components are bundled:

LuaJIT-2.0.0-beta10
array-var-nginx-module-0.03rc1
auth-request-nginx-module-0.2
drizzle-nginx-module-0.1.4
echo-nginx-module-0.41
encrypted-session-nginx-module-0.02
form-input-nginx-module-0.07rc5
headers-more-nginx-module-0.18
iconv-nginx-module-0.10rc7
lua-5.1.5
lua-cjson-1.0.3
lua-rds-parser-0.05
lua-redis-parser-0.10
lua-resty-dns-0.08
lua-resty-memcached-0.08
lua-resty-mysql-0.10
lua-resty-redis-0.14
lua-resty-string-0.06
lua-resty-upload-0.03
memc-nginx-module-0.13rc3
nginx-1.2.3
ngx_coolkit-0.2rc1
ngx_devel_kit-0.2.17
ngx_lua-0.6.10
ngx_postgres-1.0rc2
rds-csv-nginx-module-0.05rc2
rds-json-nginx-module-0.12rc10
redis-nginx-module-0.3.6
redis2-nginx-module-0.09
set-misc-nginx-module-0.22rc8
srcache-nginx-module-0.16
xss-nginx-module-0.03rc9

Devel Release 1.2.3.7 - 6 October 2012

upgraded LuaNginxModule to 0.6.10.
- feature: now ngx.req.get_headers() returns a Lua table with keys in the all-lower-case form by default. thanks James Hurst and Matthieu Tourne for the feature request.
- feature: now ngx.req.get_headers() adds an __index metamethod to the resulting Lua table by default, which will automatically normalize the lookup key by converting upper-case letters and underscores in case of a lookup miss. thanks James Hurst and Matthieu Tourne for suggesting this feature.
- feature: now ngx.req.get_headers() accepts a second (optional) argument, raw, for controlling whether to return the original form of the header names (that is, the original letter-case).
- feature: added public C API functions ngx_http_shared_dict_get and ngx_http_lua_find_zone to allow other Nginx C modules or a patched Nginx core to directly access the shared memory dictionaries created by LuaNginxModule. thanks Piotr Sikora for requesting this feature.
- bugfix: fixed a compilation warning in the TCP/stream cosocket codebase when using (at least) gcc 3.4.6 for MIPS. thanks Dirk Feytons for reporting this as GitHub issue #162.

Devel Release 1.2.3.5 - 1 October 2012

upgraded LuaNginxModule to 0.6.8.
- bugfix: ngx.re.gmatch might loop infinitely when the pattern matches an empty string. thanks Lance Li and xingxing for tracking this issue down.
- bugfix: pattern matching an empty substring in ngx.re.gmatch did not match at the end of the subject string.
- bugfix: ngx.re.gsub might enter infinite loops because it could not handle patterns matching empty strings properly.
- bugfix: ngx.re.gsub incorrectly skipped matching altogether when the subject string was empty.

Devel Release 1.2.3.3 - 26 September 2012

upgraded LuaNginxModule to 0.6.7.
- feature: implemented the shdict:flush_expired(max_count?) method for flushing out and removing expired items up to max_count (or unlimited when max_count == 0). thanks Brian Akins for the patch.
- optimize: tcpsock:send() now calls c->send() instead of ngx_output_chain(), which gives about 4% ~ 5% performance boost for a simple test case accessing Redis for several times.
- optimize: we now skip processing in the default write event handler when the write event is not ready.
- refactor: the I/O scheduler has been rewritten to keep track of the coroutine associated with each (non-blocking) I/O operation automatically, which paves a way to the upcoming ngx.thread API (aka the "lightweight thread" API).
- refactor: now we use a new Nginx subrequest model that bypasses ngx_http_postpone_filter_module completely, which paves a way for arbitrary order of outputting among subrequests and their parents when the ngx.thread API lands in the near future.
- bugfix: the "http finalize non-active request" alerts might happen when subrequests were used. thanks Lance Li for reporting this issue.
- bugfix: reset the subrequest status code when the ngx_http_upstream request in the subrequest fails due to timeout errors or premature connection close and etc. this fix also requires the nonbuffered-upstream-truncation patch for the Nginx core to cancel a limitation in ngx_http_upstream.
- bugfix: fixed the gcc error -Werror=strict-aliasing in the Lua/LuaJIT bytecode loader when -O2 or above is used. thanks jinglong for the patch.
- bugfix: the main request might be prematurely terminated if a subrequest issued by ngx.location.capture (or its friends) was finalized with error codes.
- bugfix: the Nginx built-in resolver might not be destroyed in time when it was used by ngx.socket.tcp and ngx.socket.udp.
- bugfix: coroutine.status() returned suspended for normal coroutines.
- bugfix: coroutine.resume() did not return an error immediately when operating on normal coroutines.
- bugfix: when the entry coroutine was yielded by coroutine.yield() then after it was resumed automatically its status would still be suspended.
- bugfix: the write event timer might not be removed in time in ngx.flush(true) when ngx_handle_write_event failed.
- bugfix: always remove the read event timer during downstream cosocket finalization if it is not removed yet.
- bugfix: ngx.flush(true) might not return immediately when it should.
- bugfix: the resume_handler field of the subrequest ctx was not properly initialized.
- feature: added new dtrace static probes http-lua-user-coroutine-yield and http-lua-entry-coroutine-yield.
- docs: fixed the documentation for ngx.req.set_header and made it clear that the modified request headers will be inherited by the subrequests by default. thanks James Hurst for reporting this issue.
- docs: documented the trick for doing background asynchronous jobs by using ngx.eof() + keepalive_timeout 0. thanks Lance Li for the suggestion.
upgraded LuaRedisParserLibrary to 0.10.
- bugfix: Lua stack overflow would happen when too many Redis arguments were passed into the build_query method. thanks Guo Yin for reporting this issue.
upgraded LuaRestyDNSLibrary to 0.08.
- feature: added new method tcp_query to enforce pure TCP transportation for the DNS queries.
- feature: added support for TCP retries when the UDP reply gets truncated.
- feature: added support for PTR queries and records.
- feature: added support for TXT queries and records.
- feature: added support for NS queries and records.
- bugfix: the udp resolver did not discard DNS replies with unmatched IDs for 128 times as originally designed.
upgraded LuaRestyRedisLibrary to 0.14.
- optimize: now we do the string concatenation for Redis queries on the Lua land instead of on the C land, which gives 6% ~ 7% over-all performance boost when using LuaJIT 2.0 beta10.
- docs: fixed a typo in the sample code. thanks xingxing for reporting it.
upgraded LuaRestyMemcachedLibrary to 0.08.
- feature: added new option key_transform to the new method to allow the user to override the default escaping and unescaping methods for Memcached keys. thanks Matthieu Tourne for the patch.
- bugfix: now the new method will return a string describing the error as the second return value in case of failures.
- docs: added more documentation for the set_keepalive method.
- docs: documented that this library cannot be used in those contexts where the LuaNginxModule cosocket API is unavailable.
- docs: documented that storing the object instance into Lua module-level variables will result in failures for concurrent requests.
upgraded SrcacheNginxModule to 0.16.
- bugfix: srcache_fetch would use truncated responses from MemcNginxModule or other upstream modules. this usually happened when the upstream read timer was expired or the upstream prematurely closed the connection. this fix also requires the nonbuffered-upsteram-truncation patch to cancel a limitation in the Nginx core. thanks Bryan Alger for reporting the issue.
- bugfix: the main request response was not discarded by srcache_store when there was an error in the last minute (like a read-timeout error or premature connection close happens when ngx_http_upstream reads the upstream response body). this fix also requires the nonbuffered-upstream-truncation patch for the Nginx core to cancel a limitation in ngx_http_upstream.
- bugfix: the main request might prematurely terminate if the srcache_store subrequest was finalized with error codes.
upgraded Redis2NginxModule to 0.09.
- bugfix: directives redis2_query, redis2_literal_raw_query, and redis2_raw_queries could not be inherited automatically by the location if blocks, resulting in the "no redis2 query specified or the query is empty" error. thanks Tomasz Prus for the patch.
feature: updated the dtrace patch to add new static probe create-pool-done.
feature: updated the dtrace patch to include new tapset functions ngx_indent, ngx_http_subreq_depth, and ngx_http_req_parent.
bugfix: added the nonbuffered-upstream-truncation patch for the Nginx core to make ngx_http_upstream provide a way in the context of a subrequest to signal the parent of errors when upstream data truncation happens. thanks Bryan Alger for reporting this issue. (This is a temporary solution and I'll work on a new patch as per Maxim Dounin's suggestions.)
bugfix: applied the channel-uninit-params patch for the Nginx core to fix Valgrind/Memcheck warnings about unitialized bytes in the parameters of sendmsg.
feature: updated the allow_request_body_updating patch to define the HAVE_ALLOW_REQUEST_BODY_UPDATING_PATCH macro.

Devel Release 1.2.3.1 - 22 August 2012

upgraded the Nginx core to 1.2.3.
- see http://nginx.org/en/CHANGES-1.2 for changes.
upgraded LuaNginxModule to 0.6.2.
- feature: (re)implemented the standard Lua coroutine API, which means that the user is now free to create and run their own coroutines within the boilerplate coroutine created automatically by LuaNginxModule. thanks chaoslawful and jinglong for the design and implementation.
- feature: added new dtrace static probes for the user coroutine mechanism: http-lua-coroutine-create and http-lua-coroutine-resume.
- feature: added new dtrace static probes for the cosocket mechanism: http-lua-socket-tcp-send-start, http-lua-socket-tcp-receive-done, and http-lua-socket-tcp-setkeepalive-buf-unread.
- bugfix: the send timeout timer for downstream output was not deleted in time in our write event handler, which might result in request abortion for long running requests. thanks Demiao Lin (ldmiao) for reporting this issue.
- bugfix: tcpsock:send() might send garbage if it was not the first call: we did not properly initialize the chain writer ctx for every send() call. thanks Zhu Dejiang for reporting this issue.
- bugfix: the ngx_http_lua_probe.h header file was not listed in the NGX_ADDON_DEPS list in the config file.
- optimize: removed unnecessary code that was for the old coroutine abortion mechanism based on Lua exceptions. we no longer need that at all because we have switched to using coroutine yield to abort the current coroutine for ngx.exec, ngx.exit, ngx.redirect, and ngx.req.set_uri(uri, true).
upgraded LuaRestyDNSLibrary to 0.06.
- feature: added support for MX type resource records.
- feature: unrecognized types of resource records will return their raw resource data (RDATA) as the rdata Lua table field.
upgraded LuaRestyRedisLibrary to 0.13.
- feature: added new method read_reply, mostly for using the Redis Pub/Sub API.
- feature: added new class method add_commands to allow adding support for new Redis commands on-the-fly. thanks Praveen Saxena for requesting this feature.
- docs: added a code sample for using the Redis transactions.
upgraded DrizzleNginxModule to 0.1.4.
- bugfix: the open socket #N left in connection alerts would appear in the nginx error log file when the MySQL/Drizzle connection pool was used and the worker process was shutting down.
upgraded PostgresNginxModule to 1.0rc2.
- bugfix: the open socket #N left in connection alerts would appear in the nginx error log file when the PostgreSQL connection pool was used and the worker process was shutting down.
- bugfix: removed the useless http-cache related code from ngx_postgres_upstream_finalize_request to suppress clang warnings.
added more dtrace static probes to the Nginx core: timer-add, timer-del, and timer-expire.
added more systemtap tapset functions: ngx_chain_next, ngx_chain_writer_ctx_out, ngx_chain_dump, and ngx_iovec_dump.

See ChangeLog1002001 for change log for OpenResty 1.2.1.x.

LuaNginxModule

Thu, 13 Sep 2012 18:37:00 GMT

permalink

This module embeds the Lua interpreter or LuaJIT 2.0 into the nginx core and integrates the powerful Lua threads (aka Lua coroutines) into the Nginx event model by means of Nginx subrequests.

Unlike Apache's mod_lua and Lighttpd's mod_magnet, Lua code executed using this module can be 100% non-blocking on network traffic as long as the Nginx API for Lua provided by this module is used to handle requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services.

At least the following Lua libraries and Nginx modules can be used with this LuaNginxModule:

Almost all the Nginx modules can be used with this LuaNginxModule by means of ngx.location.capture or ngx.location.capture_multi but it is recommended to use those lua-resty-* libraries instead of creating subrequests to access the Nginx upstream modules because the former is usually much more flexible and memory-efficient.

The Lua interpreter or LuaJIT instance is shared across all the requests in a single nginx worker process but request contexts are segregated using lightweight Lua coroutines. Loaded Lua modules persist in the nginx worker process level resulting in a small memory footprint in Lua even when under heavy loads.

Documentation: http://wiki.nginx.org/HttpLuaModule

Project page: http://github.com/chaoslawful/lua-nginx-module

DrizzleNginxModule

Thu, 13 Sep 2012 18:34:00 GMT

permalink

This is an nginx upstream module that talks to MySQL and/or Drizzle database servers by libdrizzle.

This ngx_drizzle module is not enabled by default. You should specify the --with-http_drizzle_module option while configuring OpenResty.

The libdrizzle C library is no longer bundled by OpenResty. You need to download the drizzle server tarball from https://launchpad.net/drizzle.

Latest drizzle7 release does not support building libdrizzle 1.0 separately and requires a lot of external dependencies like Boost and Protobuf which are painful to install. The last version supporting building libdrizzle 1.0 separately is 2011.07.21. You can download it from here:

http://openresty.org/download/drizzle7-2011.07.21.tar.gz

When you get the drizzle7 2011.07.21 release tarball, you can install libdrizzle-1.0 like this:

tar xzvf drizzle7-2011.07.21.tar.gz
cd drizzle7-2011.07.21/
./configure --without-server
make libdrizzle-1.0
make install-libdrizzle-1.0

Please ensure that you have the python command point to a python2 interpreter. It's known that on recent Arch Linux distribution, python is linked to python3 by default, and while running make libdrizzle-1.0 will yield the following error:

  File "config/pandora-plugin", line 185
    print "Dependency loop detected with %s" % plugin['name']
                                           ^
SyntaxError: invalid syntax
make: *** [.plugin.scan] Error 1

You can fix this by pointing python temporarily to python2.

When you install the libdrizzle-1.0 library to a custom path prefix, you can specify the libdrizzle prefix to OpenResty like this:

cd /path/to/ngx_openresty-VERSION/
./configure --with-libdrizzle=/path/to/drizzle --with-http_drizzle_module

Documentation page: http://wiki.nginx.org/HttpDrizzleModule

Project page: https://github.com/chaoslawful/drizzle-nginx-module

When used in conjunction with LuaNginxModule, it is recommended to use LuaRestyMySQLLibrary instead of this one, since it is more flexible and more memory-efficient.

FastCGINginxModule

Thu, 13 Sep 2012 18:29:00 GMT

This is a standard Nginx module for talking to the remote FastCGI servers: http://wiki.nginx.org/HttpFastcgiModule

ProxyNginxModule

Thu, 13 Sep 2012 18:28:00 GMT

This is a standard Nginx module for talking to remote HTTP or HTTPS servers: http://wiki.nginx.org/HttpProxyModule

MemcNginxModule

Thu, 13 Sep 2012 17:56:00 GMT

permalink

This module extends the standard memcached module to support almost the whole memcached ascii protocol.

It allows you to define a custom REST interface to your memcached servers or access memcached in a very efficient way from within the nginx server by means of subrequests or independent fake requests.

Documentation: http://wiki.nginx.org/HttpMemcModule

Project page: http://github.com/agentzh/memc-nginx-module

When used in conjunction with LuaNginxModule, it is recommended to use LuaRestyMemcachedLibrary instead of this one, since it is more flexible and more memory-efficient.