Streaming, and designing for slow

Twenty seconds of blank screen feels broken. The same twenty seconds with tokens arriving feels fast.

Add "stream": true and you get server-sent events:

curl -N https://animica.dev/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"animica-chat-small","stream":true,
       "messages":[{"role":"user","content":"Write a haiku about hashing."}]}'

Each line arrives as data: {...} with a token in choices[0].delta.content, ending with data: [DONE].

The pattern animica.dev uses on its own homepage, and worth copying:

• Stream, so the first token ends the "is it broken?" question.
• Show a reassuring waiting message that changes over time — *"waiting for a GPU provider to pick this up…"* — rather than a spinner that says nothing.
• Never apologise with "this is taking too long". The network is doing unpaid work on donated hardware; say what is happening instead.
• Set a generous client timeout. A 10-second abort will fail almost every cold start and teach your users the service does not work.