The AI network · ≈ 10 min

Prompting that survives a small model

Techniques that matter more, not less, when the model is small and the latency is high.

Prompting advice written for frontier models often assumes the model will recover from a vague instruction. A small model on shared hardware will not. Three things carry most of the weight:

1. State the output format, exactly. Not "list some keywords" but:

Reply with ONLY a comma-separated list of 5-10 lowercase keywords.
No sentences, no explanation, no punctuation other than commas.

2. Put the instruction before the data. Models attend more reliably to instructions that precede a long input than to ones buried after it.

3. Give one example if the format is unusual. One is usually enough; ten costs latency you cannot afford at 20 seconds a call.

A worked example from the Animica x402 marketplace, which uses this network for search query expansion:

You expand search queries for a marketplace of paid APIs.
Given what a user wants, reply with ONLY a comma-separated list of 5-10
lowercase keywords that would appear in the name, description or tags of a
matching API. No sentences, no explanation, no punctuation other than commas.

User wants: I need to read data from a blockchain

Real output from animica-chat-small:

blockchain, read, contract, storage, bytes

Short, parseable, no preamble. That is what the format instruction bought.

Never trust model output as code or as a query. That expansion goes into a full-text search index, so before use it is reduced to bare [a-z0-9_] terms — no quotes, no operators, no punctuation. A model that returns "; DROP contributes the word drop and nothing else.

The rule generalises: a model's output is attacker-influenced text whenever any part of the prompt came from a user. Treat it exactly like form input.

Community models sometimes emit a reasoning block before the answer:

<think>Let me consider what the user wants…</think>weather, forecast, climate

Strip it before parsing — re.sub(r'<think>.*?</think>', '', text, flags=re.S). Feeding the reasoning into your parser is a real bug that has shipped more than once on this stack.

Stuck? Ask

Answered by Animica's own free inference network. It is donated GPU capacity, so give it 20-30 seconds.