Webmachine streamed bodies
Author: Basho Technologies
Keywords: webmachine http rest web

# webmachine

- Home
- Source Code
- Contact

### Webmachine streamed bodies

Webmachine allows the resource developer to handle request and response bodies as either whole units (binary or iolist) to be handed around at once, or else to choose to "stream" the body.

The body-handling functions are:

- `wrq:req_body/1`
- `wrq:stream_req_body/2`
- `wrq:set_resp_body/2`

The last of these, `wrq:set_resp_body/2`, is also called implicitly with the return value of any content-producing function such as `to_html/2`.

The first of these (`req_body`) is the simplest. It will provide the whole incoming request body as a binary. (Unless the body is too large, as set by `wrq:set_max_recv_body/2` or defaulting to 50M) For the majority of resources, this is the easiest way to handle the incoming request body.

If a resource wants to handle the incoming request body a hunk at a time, it may call `wrq:stream_req_body/2` instead. Instead of a binary, this produces a `StreamBody` structure.

(It is an error to call both `wrq:req_body/1` and `wrq:stream_req_body/2` in the execution of a single resource.)

A `StreamBody` is a pair of the form `{Data,Next}` where `Data` is a binary and `Next` is either the atom `done` signifying the end of the body or else a 0-arity function that, when called, will produce the "next" `StreamBody` structure.

The integer parameter to `wrq:stream_req_body/2` indicates the maximum size in bytes of any `Hunk` from the resulting `StreamBody`.

When a resource provides a body to be sent in the response, it should use `wrq:set_resp_body/2`. The parameter to this function may be either an iolist, representing the entire body, or else a pair of the form `{stream, StreamBody}`.

An example may make the usage of this API clearer. A complete and working resource module using this API in both directions:

    -module(mywebdemo_resource).
    -export([init/1, allowed_methods/2, process_post/2]).

    -include_lib("webmachine/include/webmachine.hrl").

    init([]) -> {ok, undefined}.

    allowed_methods(ReqData, State) -> {['POST'], ReqData, State}.

    process_post(ReqData, State) ->
        Body = get_streamed_body(wrq:stream_req_body(ReqData, 3), []),
        {true, wrq:set_resp_body({stream, send_streamed_body(Body,4)},ReqData), State}.

    send_streamed_body(Body, Max) ->
        HunkLen=8*Max,
        case Body of
            <> ->
                io:format("SENT ~p~n",[<>]),
                {<>, fun() -> send_streamed_body(Rest,Max) end};
            _ ->
                io:format("SENT ~p~n",[Body]),
                {Body, done}
        end.

    get_streamed_body({Hunk,done},Acc) ->
        io:format("RECEIVED ~p~n",[Hunk]),
        iolist_to_binary(lists:reverse([Hunk|Acc]));
    get_streamed_body({Hunk,Next},Acc) ->
        io:format("RECEIVED ~p~n",[Hunk]),
        get_streamed_body(Next(),[Hunk|Acc]).

If you use this resource in place of the file `/tmp/mywebdemo/src/mywebdemo_resource.erl` in the quickstart setup, you should then be able to issue `curl -d '1234567890' http://127.0.0.1:8000/` on the command line and the `io:format` calls will show you what is going on.

Obviously, a realistic resource wouldn't use this API just to collect the whole binary into memory or break one up that is already present -- you'd use `req_body` and put a simple iolist into `set_resp_body` instead. Also, the choices of 3 and 4 bytes as hunk size are far from optimal for most reasonable uses. This resource is intended only as a demonstration of the API, not as a real-world use of streaming request/response bodies.