Redesigning hobby's handler model

[SeanTAllen]

hobby's current handler design doesn't work for real applications. Handler is interface val — stateless, shareable, called synchronously inside _Connection's behavior. That's fine for "parse request, return response" handlers. It falls apart the moment you need to do anything async.

Want to query Postgres? You need an actor to receive the result. Want to call another service? Same problem. The only escape hatch today is the streaming API, which is a heavyweight workaround for what should be the default case. Any real web application talks to databases, caches, and other services. If your handler can't do that without contortions, the framework is fighting the language instead of working with it.

This isn't theoretical. The pyrois design (Discussion #8) identified exactly this problem and responded by forking hobby's entire request pipeline. Pyrois reimplements routing, middleware, context, and chain runner — same interfaces, same patterns — because hobby's synchronous pipeline can't yield for async work. That fork is a clear signal that the foundation needs to change.

While we're fixing this, there are two other problems worth solving at the same time:

1. Response buffering. hobby sends bytes to the wire the moment ctx.respond() is called. That means after middleware can't modify response headers. This is why pyrois had to fork the pipeline for session support — the Set-Cookie header needs to be injected after the handler runs.

2. Sessions belong in hobby. Sessions aren't a "Phoenix-like framework" feature. They're a web framework feature. Any real app needs them. Pushing them to pyrois forces pyrois to duplicate hobby's entire pipeline just to get async middleware and response buffering.

What we considered

Actor-per-connection

Stallion's model, one layer up: the user's actor IS the connection. It embeds a hobby class that handles routing and middleware, and gets callbacks for routed requests. Same embed-and-delegate pattern the Pony ecosystem already uses.

The problem is organizational. All route handling logic lives in one actor. Every route handler method, every async callback behavior, all in one place. If two routes both query a database, the actor has to figure out which route's callback just fired. For a handful of routes it's fine. For a real application with twenty routes doing different async work, the actor becomes a tangled mess of state tracking and callback multiplexing.

Actor-per-route

One long-lived actor per route. Requests to that route are serialized through it. Can maintain state across requests, but a slow handler blocks every other request to that route.

Actor-per-connection with delegation

The connection actor dispatches to per-route handler actors. But this is just actor-per-request with an extra hop. The connection actor becomes pure plumbing and the handler actors are where the work happens. So skip the middleman.

The design: actor-per-request

Each route registers a factory. When a request matches a route, _Connection calls the factory to create a handler actor for that specific request. The handler actor does its work (sync or async), responds, and gets GC'd.

_Connection stays as plumbing. Long-lived, one per TCP connection. It handles TCP, HTTP parsing, routing, session loading, middleware, and response buffering. It creates handler actors as requests arrive. When a handler finishes, the connection picks up the next request. Request processing is serial — one handler actor active per connection at a time, one Responder held at a time.

Pipelining works the same way it does today. Requests are buffered while a handler is in progress and drained when it responds.

Handler actors

The handler actor embeds a class that gives it the tools to respond and stream. It gets everything it needs at creation: request data, route params, session, and a communication protocol back to the connection.

The current design uses Context ref specifically to avoid iso-consumption problems — if middleware errors after consuming an iso Context, the Context is lost. The actor-per-request model avoids this differently: middleware runs in the connection before the HandlerContext iso is created, so middleware never touches the iso. The iso is consumed exactly once, by the factory, which passes it to the handler actor's constructor. There's no risk of losing it to a middleware error because the middleware that could error runs before the iso exists.

actor GetUserHandler
  embed _handler: RequestHandler

  new create(ctx: HandlerContext iso, db: DatabasePool tag) =>
    _handler = RequestHandler(consume ctx)
    try
      let id = _handler.param("id")?
      db.query(id, this)
    end

  be query_result(rows: Rows val) =>
    _handler.respond(StatusOK, rows.to_json())

No contortions. The handler is an actor. It can receive behaviors from database drivers, cache lookups, other services. When the result arrives, it responds. This is just how Pony works.

For simple routes that don't need async, the factory can respond inline:

app.get("/", {(ctx: HandlerContext iso) =>
  RequestHandler(consume ctx).respond(StatusOK, "Hello!")
} val)

The connection↔handler protocol

The connection and handler communicate through two interfaces. The handler actor implements one (so the connection can tell it things), and the connection implements the other (so the handler can send responses back).

Handler-side interface (what the handler actor implements):

• be dispose() — the connection is done with you, clean up
• be throttled() — stop producing chunks, TCP buffer is full
• be unthrottled() — resume producing chunks

More behaviors will be added as needed. The point is that the connection has a typed way to communicate with handler actors.

Connection-side interface (what the handler sends to):

• Respond with a buffered response (status, headers, body)
• Start a chunked streaming response
• Send a chunk
• Finish a stream

The RequestHandler embedded class hides all of this from the handler author. The handler calls _handler.respond(StatusOK, body) and RequestHandler sends the right protocol message to the connection. Handler authors never deal with the protocol directly.

Tokens. The connection generates a token (an incrementing integer) when it creates a handler actor and passes it along. The handler includes this token in every protocol message back to the connection. The connection only accepts messages with a valid token. When the connection disposes a handler (timeout, connection close, request complete), it invalidates the token. If a stale handler sends a message after that, the connection drops it on the floor. This isn't a security boundary — it's protection against coding errors where a late response from a timed-out handler could corrupt the next request on the same connection.

Handler timeout

The current synchronous model guarantees a response for every request. _ChainRunner catches handler errors and sends 500. The actor-per-request model removes that guarantee — if a handler actor never calls respond() or finish(), the connection waits forever. This is a safety problem, not a nice-to-have. A single buggy handler becomes a permanent connection leak, and under load that exhausts file descriptors.

The timeout is an inactivity timer, not an absolute deadline. It resets on any activity from the handler: a response, a chunk, a finish. An absolute deadline would kill long-running streams (large file downloads). An inactivity timer only kills handlers that have gone silent.

The implementation rides on lori's user timers. _Connection starts the timer when it dispatches a request to a handler actor. Any handler activity resets it.

Non-streaming timeout: The connection sends 504, disposes the handler actor (invalidating its token), and moves to idle. Buffered pipelined requests drain normally.

Streaming timeout: Headers are already on the wire. There's no way to send a 504 — the response is already corrupted from the client's perspective. The connection closes the socket. No graceful recovery.

The default timeout is 30 seconds, configurable at the Application level. Per-route configuration can be added later if needed.

Pony actor finalisers (_final()) can't send messages, so a handler actor that gets GC'd after starting a stream can never send the terminal chunk. The timeout is the only mechanism that catches this.

Streaming

The handler actor IS the streamer. The handler starts a chunked response, receives data through its own behaviors, and sends chunks back to the connection through the protocol. A file-reading actor is still needed (Pony file I/O is synchronous and would block the handler), but the handler actor owns and creates it rather than _Connection managing a _FileStreamer behind the scenes.

actor StreamFileHandler
  embed _handler: RequestHandler

  new create(ctx: HandlerContext iso) =>
    _handler = RequestHandler(consume ctx)
    try
      let path = _handler.param("filepath")?
      _handler.start_streaming(StatusOK)
      FileReader(path, this)
    end

  be file_chunk(data: Array[U8] val) =>
    _handler.send_chunk(data)

  be file_done() =>
    _handler.finish()

Backpressure from Stallion (on_throttled/on_unthrottled) propagates to the handler actor through the handler-side interface. When the TCP send buffer is full, the connection tells the handler to stop producing. When it drains, the connection tells it to resume. Without this, a handler that keeps calling send_chunk() while the wire is saturated would cause unbounded memory growth in Stallion's response queue.

Route registration

let db: DatabasePool tag = ...

app
  .> get("/", {(ctx: HandlerContext iso) =>
    RequestHandler(consume ctx).respond(StatusOK, "Hello!")
  } val)
  .> get("/users/:id", {(ctx: HandlerContext iso)(db) =>
    GetUserHandler(consume ctx, db)
  } val)
  .> post("/users", {(ctx: HandlerContext iso)(db) =>
    CreateUserHandler(consume ctx, db)
  } val)
  .> serve()

Factories are val — shareable across connections. Each factory receives a HandlerContext iso with everything the handler needs. If a factory needs to capture state (a database connection tag, a config val), that works naturally with Pony lambda captures as long as the captured references are sendable. The capture syntax ({(args)(captures) => body}) is how you inject dependencies — virtually every real handler needs this.

Middleware

Middleware runs entirely in _Connection. Both before and after phases execute in the same actor, on the same connection, with no middleware abstraction needed in handler actors.

This is a different split from what you'd expect if you've used other frameworks. The usual pattern is before-middleware → handler → after-middleware as a single pipeline. Here, before-middleware runs before the handler actor exists, and after-middleware runs after the handler actor sends its response back. The handler actor is just "produce a response" — it doesn't know about middleware at all.

Before-middleware runs before the handler actor is created. Auth checks, rate limiting, session loading. If before-middleware short-circuits (responds with 401 or 429), no handler actor is created and after-middleware runs on the short-circuit response.

After-middleware runs after the connection receives a response — whether that response came from a handler actor or from before-middleware short-circuiting. The response is buffered: status, headers, and body are stored, not sent. After-middleware can inspect and modify the buffered response. Inject a Set-Cookie header, add CORS headers, set security headers. Once the after chain completes, the response goes to the wire via Stallion.

This means after-middleware always runs, on every response, in one place. No special case for short-circuits. No question of "who runs the after chain when there's no handler actor." The connection runs it, every time.

Since handler actors are user-written actors, anything you'd want "handler-level middleware" to do, you just do in the actor. Code at the top of the constructor is your "before." Code before _handler.respond() is your "after." There's no need for a middleware abstraction when you control the entire actor.

Both before and after need new parameter types. before currently takes Context ref which won't exist anymore. It needs a context type with respond() capability (via the connection's local Responder), the request, and a data map for passing information to the handler. after needs a response view type that exposes mutable headers, status, and body. These types need design.

The Middleware interface stays val. Since val is freely shareable, the same middleware instances work across all connections. The val constraint means self is box, but method parameters can be any declared capability — so val middleware receiving ref contexts is correct Pony.

Handled vs. errored

The original hobby design drew a distinction between "handled" (middleware responded to the request, chain should stop, after phases still run) and "errored" (something genuinely broke). That distinction carries forward. Before-middleware that short-circuits with a 401 is a normal control flow decision, not an error. After-middleware runs on the response either way.

The handler actor model makes the errored case cleaner for middleware. If before-middleware errors, no handler actor is ever created. The connection handles the error response and runs after-middleware on it. There's no question of what happens to a half-initialized handler because there is no handler.

Handler actor errors are a different story. If the handler starts work and then fails to respond, the inactivity timer catches it and the connection sends 504. If the handler sends a response, after-middleware runs on it in the connection regardless of what happened inside the handler.

Response buffering

respond() in the handler actor doesn't send bytes to the wire. It sends the response back to the connection through the protocol. The connection buffers it, runs the after-middleware chain (which can modify headers), and then sends it through Stallion to the wire.

The flow:

1. Handler calls _handler.respond(status, headers, body).
2. RequestHandler sends the response to _Connection via the protocol.
3. _Connection receives it, runs after-middleware on the buffered response.
4. After-middleware completes, response goes to the wire.

Streaming responses bypass the buffer for body data. start_streaming() sends headers immediately (through the protocol to the connection, then to the wire), and chunks flow as they're produced. After-middleware runs after finish() is called. After-middleware can't modify headers that are already on the wire, but it can do cleanup, logging, and session persistence.

One tradeoff: buffering means the entire response body sits in memory through the after-middleware chain. For typical API responses this is negligible. Streaming exists for large payloads.

Connection buffering

_Connection buffers requests when it's busy. Three situations now:

Waiting for session data. The connection sends load to the session store and keeps processing TCP data. It has to — Pony actors don't selectively receive messages. New completed requests get buffered. When _session_loaded fires, the pending request processes and the buffer drains.

Handler in progress. While a handler actor is working on a request, any pipelined requests that arrive get buffered. When the handler responds, the buffer drains.

Streaming. While a handler actor is streaming chunks, same as handler in progress.

The connection state machine is more complex than today. Currently it has two states: normal and streaming. The new design has at least five:

1. Idle — ready for next request
2. Running before-middleware — synchronous, transient
3. Waiting for session data — async gap between load and _session_loaded callback
4. Handler in progress — waiting for handler actor to respond
5. Streaming — handler actor has started chunked response, chunks flowing

States 3, 4, and 5 all buffer incoming pipelined requests. The state machine needs to be formalized with explicit transitions, especially edge cases like connection close during handler execution or session callback timing. The state machine should enforce constraints (e.g., session loading only starts after the previous request completes) rather than assuming them.

Sessions

Sessions are opt-in. An application that doesn't need sessions pays no session overhead. Enable them at the Application level:

let key = CookieSigningKey.from_secret(env, secret)?
let store = MemorySessionStore(key)
app.use_sessions(store, key)

The session design follows what was worked out in pyrois Discussion #8, moved into hobby where it belongs.

Sessions are server-side. A MemorySessionStore actor holds session data in memory. The cookie carries only a signed session ID, not the data itself. No 4KB cookie size limit, the server can revoke sessions immediately, and session data isn't readable by the client.

Session loading is async, and that's the whole reason the connection handles it rather than the handler. The connection reads the session cookie, verifies the signature, sends load(session_id, this) to the store, and waits for the callback. The handler actor is created after the session arrives. From the handler's perspective, the session is just there — no async machinery visible.

The SessionEditor ref on the context gives handlers read/write access. When the handler responds and the response arrives at the connection, session middleware's after phase checks if the session was modified. If so, it serializes the session into a Set-Cookie header on the buffered response and fires off a save to the store. The save is fire-and-forget: Pony's causal messaging guarantees FIFO delivery per sender-receiver pair, so a subsequent load from the same connection will see the saved data.

Security defaults are hardcoded: Secure, HttpOnly, SameSite=Lax, __Host- prefix. These are the only correct settings for session cookies. Secure and __Host- mean sessions won't work over plain HTTP — local development requires TLS (e.g., a self-signed cert).

Session store eviction

The session store needs configurable eviction. Without it, each cookieless request (bots, health checks, curl) creates a new session, and the store grows without bound. Some form of bounded storage with eviction (maximum count, TTL, or both) is required. The specifics are part of the session store API design.

Streaming and session limitations

For streaming responses, session cookies are pre-set before headers go to the wire. Session modifications during streaming get saved to the store, but new cookies can't be injected since headers are already sent.

regenerate_id() is partial and errors when streaming is active. Without this guard, calling regenerate_id() during streaming would delete the old session, create a new one, and have no way to send the new cookie to the client. On the next request, the client sends the old (now-deleted) session ID, the store doesn't find it, and the user's session data is gone. Making it partial forces the handler author to actively decide what to do — they can't accidentally destroy a user's session.

Concurrent session access

Concurrent requests from different connections can race on the same session. Two requests loading the same session, both modifying it, both saving — last write wins. This is inherent to server-side sessions without locking, and it's the same behavior as every other in-memory session store.

One specific edge case: if request A calls regenerate_id() while request B (same session, different connection) is also in flight, request B can save with the old session ID after A deleted it, re-creating the old session and defeating fixation protection. The trigger is narrow (overlapping requests during login), but the consequence is real. This is a known limitation of optimistic concurrent sessions.

Relationship with pyrois

With sessions in hobby and an async-capable handler model, pyrois can build on hobby's pipeline instead of forking it. The intended boundary:

• hobby: web framework. Routing, middleware, async handlers, response buffering, sessions.
• pyrois: real-time web. WebSockets, LiveView. Builds on hobby.

If hobby's redesign supports pyrois's needs (async middleware, response buffering, sessions), the duplicated pipeline code in pyrois — routing, middleware chain, context, chain runner — can be replaced with hobby imports. Whether this actually happens depends on the details working out, but the goal is to make it possible.

Open questions

HandlerContext shape. What exactly goes into the iso context passed to the factory? Request, params, session editor, data map, response buffer reference, protocol token, connection tag reference. How these are packaged — one big class, a struct, multiple arguments — needs working out. Note that ref fields (like SessionEditor) must be created inside a recover iso block alongside the HandlerContext — _Connection can't create them separately and insert them afterward.

RequestHandler embedded class API. What methods does it expose? respond, respond_with_headers, start_streaming, send_chunk, finish, param, body, session. It hides the protocol (tokens, connection-side interface) from handlers.

Before-middleware context type. before currently takes Context ref. The new type needs respond() capability (for short-circuiting), the request, cookies, and a data map for passing information to the handler (auth middleware identifying the user, for example). The connection maintains a data map that gets packaged into the HandlerContext iso before creating the handler actor.

After-middleware response view type. after needs mutable access to buffered response headers, read access to status and body.

HEAD request handling. Currently handled by _ResponseMode (a strategy interface with _StandardResponseMode and _HeadResponseMode). In the new model, response building happens in RequestHandler, so the response mode strategy likely moves there. HEAD-to-GET fallback stays in _Connection. Needs confirmation.

ServeFiles. Currently a Handler val. Needs to become a handler actor factory. File streaming is a natural fit for actor-per-request — the handler actor owns the file reader, receives chunks, and sends them to the connection. But ServeFiles is more than just file streaming. It implements ETag, conditional requests (If-None-Match, If-Modified-Since), directory index serving, content type mapping, and HEAD optimization. All of that needs to be reworked in the new model. This is a significant chunk of implementation work.

Per-request actor overhead. Actor-per-request means an allocation per request. Pony actors are lightweight, but this needs to be verified with benchmarks under load, not just assumed to be fine.

Backpressure details. The design calls for throttled/unthrottled behaviors on the handler-side interface, but the interaction with Stallion's ChunkSendToken flow control (on_chunk_sent) needs to be worked out. Whether handler actors can use acknowledge-then-send-next flow control, or whether fire-and-forget streaming with connection-level throttle notifications is sufficient, is an implementation question.

[SeanTAllen]

Implemented by #42