Scripting Nginx with Lua

James Hurst

Scripting Nginx with Lua

(Introducing OpenResty)

James Hurst @pintsized | github.com/pintsized

Credits

Nginx

Nginx

nginx.org
  • Popular high performance HTTP server.
  • Event based (asynchronous) architecture.
  • Low and predictable memory footprint.
  • Declarative configuration language.
  • Extendable with modules written in C.

Introducing OpenResty

openresty.org
  • OpenResty = Nginx + a collection of modules.
  • In production for large e-commerce sites in China (Alibaba Group).
  • Non-blocking I/O using Nginx modules, including:
    • Memcached
    • Redis
    • MySQL / Drizzle
    • PostgreSQL
  • Includes a Lua C module and a small library of Lua "resty" modules.
  • Bundles "cjson" for fast encoding / decoding between Lua and JSON.

Nginx configuration language

http {
    server {
        location /hello {
            set_unescape_uri $name $arg_name;
            set_if_empty $name "Anonymous";
            echo "Hello, $name!";
        }
    }
}


$> curl http://localhost/hello

Hello, Anonymous!

$> curl http://localhost/hello?name=James

Hello, James!

Non-blocking I/O with Nginx modules

location /memcached {
    set $memc_cmd $arg_cmd;
    set $memc_key $arg_key;
    set $memc_value $arg_val;
    set $memc_exptime $arg_exptime;
    memc_pass 127.0.0.1:11211;
}


$> curl http://localhost/memcached?cmd=set&key=foo&val=bar&exptime=3600

STORED

Lua

Lua Nginx Module

github.com/chaoslawful/lua-nginx-module
  • Introduces directives for running Lua chunks during Nginx phases:
    1. Rewrite phase.
    2. Access phase.
    3. Content phase.
    4. Log phase.
  • Exposes the Nginx environment to Lua via an API.
  • Is fast, as you'd expect.
  • Is even faster when compiled with --with-luajit.

Hello, Lua!

location /hellolua {
    default_type 'text/plain';

    content_by_lua '        local name = ngx.var.arg_name or "Anonymous"
        ngx.say("Hello, ", name, "!")    ';
}


$> curl http://localhost/hellolua?name=Lua

Hello, Lua!

Load external Lua modules

location /lua {
    rewrite_by_lua_file /path/to/rewrite.lua;
    access_by_lua_file /path/to/access.lua;
    content_by_lua_file /path/to/content.lua;
}
                

Modules are loaded once, on the first request, unless lua_code_cache is set to off (useful during development).

Lua Nginx Module API

  • Uses Lua coroutines to integrate with the Nginx reactor.
  • Provides a synchronous yet non-blocking API to Nginx features:
    • Access the HTTP request.
    • Compose a HTTP response.
    • Perform sub-requests (to other "location" blocks).
    • Connect (non-blocking) to other network services.

Sub-requests

Sub-requests with "ngx.location.capture"

  • A HTTP-like interface for issuing sub-requests.
  • Not actually HTTP, handled efficiently at the C level.
  • Synchronous API, yet non-blocking to Nginx (no deep callbacks mess).
  • Returns a table containing status, header and body fields.
  • Capture multiple concurrent subrequests with ngx.location.capture_multi()

Sub-requests with "ngx.location.capture" contd.

location / {
    content_by_lua '        local res = ngx.location.capture("/sub")
        if res.status >= 500 then 
            ngx.exit(res.status) 
        end
        ngx.status = res.status
        ngx.say(res.body)    ';
}

location /sub {
    echo "Hello, Sub-Request!";
}

                

Non-blocking I/O with sub-requests

location / {
    content_by_lua '        local res = ngx.location.capture("/memcached",
            { args = { cmd = "incr", key = ngx.var.uri } }
        )    ';
}

location /memcached {
    set $memc_cmd $arg_cmd;
    set $memc_key $arg_key;
    memc_pass 127.0.0.1:11211;
}

                

Sub-request use cases

  • Reusing non-Lua configurations in the Lua space.
  • I/O via 3rd party modules (databases etc).
  • Proxying to upstream applications with the Nginx proxy module.

Cosocket

Non-blocking network I/O with "cosocket"

  • More recent addition to the API.
  • Send and receive on TCP or Unix domain sockets.
  • API compatible with LuaSocket, yet non-blocking to Nginx.
  • Has a keepalive mechanism to avoid connect/close for each request.

Non-blocking network I/O with "cosocket" contd.

location /memcached {
    content_by_lua '        local sock = ngx.socket.connect("127.0.0.1", 11211)
        sock:send("SET foo bar 3600\r\n")
        local line = sock:receive()
        if line then
            ngx.say(line)
        end
        sock:setkeepalive()    ';
}

$> curl http://localhost/memcached

STORED

Shared Data

Per-request data with "ngx.ctx"

A Lua table to store data with a lifetime identical to the current request.

location /ctx {
    access_by_lua '        ngx.ctx.userid = 12345    ';
    content_by_lua '        ngx.say(ngx.ctx.userid)    ';
}
$> curl http://localhost/ctx

12345

Shared data with "ngx.shared.DICT"

A dictionary API for storing global data across all requests.

http {
    lua_shared_dict stats 10m;
    server {
        location / {
            content_by_lua '                ngx.shared.stats:incr("hits", 1)
                ngx.say(ngx.shared.stats:get("hits"))            ';
        }
    }
}

lua-resty-*

lua-resty-* Modules

Non-blocking I/O with "lua-resty-memcached"

location /memcached {
    content_by_lua '        local memcached = require "resty.memcached"
        local memc = memcached:new()
        local ok, err = memc:connect("127.0.0.1", 11211)

        local ok, err = memc:set("foo", "bar", 3600)
        if ok then
            ngx.say("STORED")
        end

        memc:set_keepalive()    ';
}
                

Testing

Running tests

  • The combination of Lua code, modules, and Nginx configuration make it hard to test everything in place.
  • Test::Nginx::Socket is a Perl module in CPAN.
  • Written by the prolific agentzh!
  • Allows config including Lua code to be supplied and run, with tests against the HTTP response.
  • Using MOCKEAGAIN it's possible to simulate poor network performance.

Example test

use Test::Nginx::Socket;
plan tests => $Test::Nginx::Socket::RepeatEach * 2 * blocks();
run_tests();

__DATA__
=== TEST 1: sanity
--- config
    location /hello {
        content_by_lua 'ngx.say("Hello, Lua!")';
    }
--- request
    GET /hello
--- response_body
Hello, Lua!

                

Summary

Summary

  • The Nginx architecture is excellent for highly scalable applications.
  • Nginx can do a variety of things thanks to module extensions, and one can resuse those extensions by issuing sub-requests in Lua.
  • lua-nginx-module makes use of the evented architecture in Nginx, providing a powerful and performant programming environment.
  • It's possible to do 100% non-blocking I/O with readable code.
  • It's a relatively low-level tool set.

Use cases?

  • You may already be using Nginx in front of your existing applications (Rails, Django, Node.js etc).
  • OpenResty empowers you to do (fast) work inside the same server:
    • Caching, see github.com/pintsized/ledge for an example.
    • A/B testing.
    • Improving content delivery (combining CSS on the fly, etc.)
  • RESTful API services.
  • API translation and authentication services.

Building full-blown web applications?

  • Currently lacking higher level abstractions.
  • Lua is already a good templating language.
  • Lua modules for building higher levels apps aren't too hard to imagine.
  • Something like github.com/pintsized/lua-resty-rack could be a step towards a convention for "resty apps". Feedback welcome!
  • Do we need another MVC framework?

Questions?