Developers · September 21, 2026

Test an Agent's Inbound Email Loop in CI

A routable test mailbox needs a message actually transmitted. Inject one instead: a CI job asserting received, parsed, threaded and replied.

Your pipeline can prove the agent composes a good reply. It cannot prove the agent ever saw the message, because in most suites the message never existed: it was a fixture object handed to a function. To test agent email inbound without sending anything you need the message to arrive the way a real one arrives, in the inbox the agent actually reads, through the code that actually parses it.

This is the CI job that does that. It injects a synthetic message through the documented test inbound endpoint and asserts four links of the chain in order: received, parsed, threaded, replied. It also covers the two things that will break the job later, which are the test-mode counters and the one response that returns HTTP 200 while telling you it has no body. For the wider sandbox tour rather than a pipeline, read testing an agent’s email without sending real mail.

What every inbound test tool actually requires

Sort the market by one question: what has to happen before a message exists in the mailbox your assertion reads?

Verified on 21 September 2026 from the vendors’ own documentation:

Two rows are routinely misdescribed in comparison posts, so state them accurately. Mailtrap Agent Inbox is a genuine inbound product, not an outbound interceptor: it provisions a hosted address, notifies you when mail lands and serves the parsed message over an API. And Mailosaur can originate the message itself, with its own documentation describing the feature as a way to “simulate someone sending an email to your product”. The first three rows are the pure outbound interceptors. The rest can all get a message into a mailbox.

The line that still separates them is narrower and more useful than “inbound versus outbound”. It is whether a message has to cross a transport, and whose mailbox it lands in.

Every routable option needs an actual transmission. That means your CI job depends on an MTA hop it does not control, with the latency and the tail behaviour that come with one, which is where the familiar waitFor with a thirty second timeout comes from and why those tests go amber on a slow afternoon. It also means the mailbox is a stand-in. It is their mailbox, holding their copy of the message, with their permission model, and your agent reads it over a second integration that only exists in test. A non-routable test inbox with an injection endpoint removes both. HTTP 201 means the row is already written, and it is written into the same inbox object, the same thread and message tables and the same event stream that live mail uses.

Say plainly where the others win. If the assertion is about MIME wire format, about SPF and DKIM alignment on a real hop, about IMAP or POP3 client behaviour, about spam scoring, or about mail your application sends to an address you do not control, a simulated transport tells you nothing and those products tell you a great deal. Nothing below tests deliverability, and nothing below should be read as a substitute for a staging run that puts real mail on the wire.

The four assertions that make up the loop

A passing “the agent replied” test proves less than it looks. Break it into four assertions, each of which can fail independently and each of which catches a different class of bug:

1. Received. A message row exists in the inbox, and a message.received event was emitted carrying its ID. This catches an agent wired to the wrong inbox, which is invisible in any test that starts from a fixture object.
2. Parsed. The stored body matches what was injected, and the response says the body is genuinely available. This catches encoding damage, over-eager sanitisation, and the truncation trap described further down.
3. Threaded. The reply carries the same thread_id as the inbound message, and the thread lists both messages in sequence order.
4. Replied. The send call returned the outcome you expected, which for a draft-scoped agent is an approval, not a message.

Assertion four is where most suites quietly assert the wrong thing. If the agent’s credential holds drafts:submit and not messages:send, a correct run returns HTTP 202 with result: "approval_required", a draft_id, an approval_id, and no message_id at all. A test that reads message_id throws on the correct behaviour and passes only when the agent has more authority than you meant to give it. Branch on result before you touch any other field. The approval row is created with a seven day expiry, so a suite that leaves approvals pending will find them gone next week rather than accumulating forever.

The same shape holds over MCP, with one difference worth building on. An agent granted read and draft gets a session in which send_message is not present in the tool list, because the tool list is filtered by the connection’s scopes before it is serialised. Over MCP, create_draft is dispatched through the same send service as send_message with messages:send stripped from the acting principal, so the draft path cannot escalate into a send even if the arguments say otherwise. Asserting “no mail was sent” is weaker than asserting the capability was absent. How scopes narrow what an agent can do covers the grant model, and the MCP server covers the session.

One practical consequence for CI: there is no MCP tool that injects inbound mail. The six tools are list_inboxes, list_threads, list_messages, get_message, create_draft and send_message. Injection is REST only, so even an MCP-native agent needs a REST test key in the pipeline to set up the fixture.

Creating the test inbox, and the 422 you will hit first

A project’s environment is fixed when the project is created, so the job runs against a test project and a key minted inside it. Create the inbox the way your setup script would:

curl -X POST "https://api.emailforagents.ai/v1/projects/$EFA_PROJECT_ID/inboxes" \
  -H "Authorization: Bearer $EFA_TEST_KEY" \
  -H "Content-Type: application/json" \
  -d '{"display_name":"ci-inbound-loop"}'

Note what is absent. Passing local_part returns HTTP 422 test_mode_unsupported, “Test inboxes use generated non-routable addresses”, and so does passing domain_id. This trips up nearly every first CI run, because the setup script was copied from a live-project example where naming the mailbox is normal. You do not choose a test address. The one you are given is shaped {inbox_id}@{project_id}.mail.invalid, and .invalid is reserved by RFC 6761 precisely so that no resolver can ever answer for it.

That is the property the whole job rests on. The inbox is not merely flagged as non-sending, it is unreachable. Combine it with the key prefix rule, where an efa_test_ credential resolves only to test projects and test inboxes and no flag lets it cross into live, and the worst outcome of a broken CI job is a failed assertion rather than an email to a customer. A misconfigured pipeline fails closed.

How to test agent email inbound without sending a message

One call injects the message. The path, verbatim:

POST /v1/projects/{project_id}/test/inbound

curl -X POST "https://api.emailforagents.ai/v1/projects/$EFA_PROJECT_ID/test/inbound" \
  -H "Authorization: Bearer $EFA_TEST_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inbox_id": "'"$EFA_INBOX_ID"'",
    "from": {"email": "ap@example.org", "name": "Accounts Payable"},
    "to": [{"email": "intake@example.invalid"}],
    "subject": "PO 8841 correction [ci-8841-a3f0]",
    "text": "Line 2 is duplicated. Please confirm. [ci-8841-a3f0]"
  }'

HTTP 201, and the body gives you both identifiers the rest of the job needs:

{
  "id": "msg_...",
  "thread_id": "thd_...",
  "environment": "test",
  "simulated": true,
  "state": "received"
}

Five details decide whether your job is meaningful.

It requires test:simulate and messages:read, and refuses anything else. The test check is applied twice, once on the project and once on the inbox row, and both failures return HTTP 403 test_mode_required. There is no argument that relaxes it.

It writes through the same storage path live inbound mail uses. A thread row, a message row with direction inbound and state received, one recipient row per address, and a content manifest holding a normalised text body and a sanitised HTML body. Your reader is not reading a mock.

It emits a real message.received event whose payload carries environment: "test" and simulated: true, so your consumer, your cursor handling and your handler idempotency all execute production code paths. The listing wraps each event with a top-level environment taken from the project, which is the field to assert on. The shape is the one in the events and webhooks reference.

It always starts a new thread, and stores no Message-ID. The request body accepts inbox_id, from, to, subject, text, html and attachment_ids, and rejects anything else outright. There is no In-Reply-To and no References, so this endpoint cannot exercise inbound header matching. It exercises the half you control, which is whether your reply groups correctly. The threading mechanics and how they break covers what the injected side leaves untested.

attachment_ids is accepted and then ignored. The request schema takes up to ten of them and validates them, and the injection path never reads the field: no attachment rows are written and the stored manifest contains text and HTML only. So a test asserting that your agent handles an inbound PDF will pass while proving nothing, which is worse than failing. Inbound attachments are not injectable in this build. Test the attachment path from the outbound side, where uploads are real and a send with an unscanned file returns HTTP 422 attachment_not_ready.

The parsed assertion, and the 200 that means “no body”

Read the body back through the documented endpoint:

GET /v1/projects/{project_id}/inboxes/{inbox_id}/messages/{message_id}/body

The response is { "text", "html", "original_raw_available", "body_available" }, and the trap is in the last field. When a message has no content manifest, this endpoint returns HTTP 200 with body_available: false and puts the message preview in text. The preview is whitespace-collapsed and cut at 240 characters. There is no error status, no error code, and nothing in text that looks wrong.

So an assertion of the form body.text.includes(tag) has two failure modes that both mislead. Put the correlation tag early in the message and the assertion passes on a truncated preview while the real body is missing. Put it late and the assertion fails with “token missing from body” when the actual fault is that no content was stored. Assert body_available === true first, then assert on the text. It is one extra line and it turns a confusing failure into an accurate one.

The HTML side is sanitised twice, once when the content is written and once when it is read, so the value you get back is not byte-identical to what you injected. Assert on structure or on a plain-text token, not on an exact HTML string.

The assertion script

Inject, read, wait for the event, reply, assert. Deliberately boring.

const base = "https://api.emailforagents.ai/v1";
const P = process.env.EFA_PROJECT_ID;
const I = process.env.EFA_INBOX_ID;
const token = process.env.EFA_TEST_KEY;
const tag = `ci-${process.env.GITHUB_RUN_ID}-${process.env.GITHUB_JOB}`;

const api = async (method, path, body, headers = {}) => {
  const res = await fetch(`${base}${path}`, {
    method,
    headers: { authorization: `Bearer ${token}`, "content-type": "application/json", ...headers },
    body: body ? JSON.stringify(body) : undefined,
  });
  const json = await res.json();
  if (!res.ok)
    throw new Error(
      `${method} ${path} ${res.status} ${json.error?.code} ${json.error?.request_id}`,
    );
  return json;
};

// 1. received
const inbound = await api("POST", `/projects/${P}/test/inbound`, {
  inbox_id: I,
  from: { email: "ap@example.org", name: "Accounts Payable" },
  to: [{ email: "intake@example.invalid" }],
  subject: `PO 8841 correction [${tag}]`,
  text: `Line 2 is duplicated. Please confirm. [${tag}]`,
});
if (inbound.state !== "received" || inbound.simulated !== true) throw new Error("not received");

// 2. parsed: body_available first, then the token
const body = await api("GET", `/projects/${P}/inboxes/${I}/messages/${inbound.id}/body`);
if (body.body_available !== true)
  throw new Error("no content manifest; text is the 240-char preview");
if (!body.text.includes(tag)) throw new Error("correlation token missing from stored body");

// 3. the event, found by id rather than consumed from the cursor
const deadline = Date.now() + 20_000;
let seen = null;
while (!seen && Date.now() < deadline) {
  const page = await api("GET", `/projects/${P}/events?limit=100`);
  seen = page.data.find((e) => e.type === "message.received" && e.data.message_id === inbound.id);
  if (!seen) await new Promise((r) => setTimeout(r, 1000));
}
if (!seen) throw new Error("no message.received event within 20s");
if (seen.environment !== "test") throw new Error("event escaped the test environment");

// 4. threaded and replied
const reply = await api(
  "POST",
  `/projects/${P}/inboxes/${I}/messages`,
  {
    to: [{ email: "ap@example.org" }],
    subject: `Re: PO 8841 correction [${tag}]`,
    reply_to_message_id: inbound.id,
    text: `Confirmed, line 2 was duplicated. [${tag}]`,
  },
  { "Idempotency-Key": `${tag}-reply-v1` },
);

if (reply.result === "approval_required") {
  if (!reply.draft_id || !reply.approval_id) throw new Error("approval without a draft");
} else if (reply.result === "queued") {
  if (reply.thread_id !== inbound.thread_id) throw new Error("reply opened a new thread");
} else {
  throw new Error(`unexpected result ${reply.result}`);
}

const thread = await api(
  "GET",
  `/projects/${P}/inboxes/${I}/threads/${inbound.thread_id}/messages?limit=25`,
);
console.log(`ok: ${thread.data.length} message(s) in ${inbound.thread_id}`);

Two choices in there are load-bearing. The Idempotency-Key is derived from the run and the intent, not from the attempt, so a retried job step replays the stored response rather than duplicating. The idempotency and retry rules explain why the key must name the message rather than the attempt. And the event is located by message_id inside a page, never by advancing a cursor. The cursor is a per-project sequence shared by every job in the project, so a parallel run that persists next_cursor makes another run’s events unreachable. The event list is capped at 100 per page and the cursor is rejected as cursor_expired if it falls behind, so treat the cursor as production machinery and leave it out of CI entirely.

Wiring it into GitHub Actions

name: agent-inbound-loop
on: [pull_request]

concurrency:
  group: efa-test-inboxes
  cancel-in-progress: false

jobs:
  inbound-loop:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "22" }
      - name: assert environment before doing anything
        env:
          EFA_TEST_KEY: ${{ secrets.EFA_TEST_KEY }}
        run: |
          test "${EFA_TEST_KEY:0:9}" = "efa_test_" || { echo "not a test key"; exit 1; }
          curl -fsS https://api.emailforagents.ai/v1/me \
            -H "Authorization: Bearer $EFA_TEST_KEY" | grep -q '"environment":"test"'
      - name: inbound loop
        timeout-minutes: 3
        env:
          EFA_TEST_KEY: ${{ secrets.EFA_TEST_KEY }}
          EFA_PROJECT_ID: ${{ vars.EFA_TEST_PROJECT_ID }}
          EFA_INBOX_ID: ${{ vars.EFA_CI_INBOX_ID }}
        run: node scripts/inbound-loop.mjs

The preflight is three lines and it is the highest-value step in the file. The server already refuses to let an efa_test_ key touch live, so this is not the safety boundary, it is the thing that turns a confusing cascade of 401s into one clear failure message. GET /v1/me also returns the credential’s scopes, project_ids and inbox_ids, so if you want the job to be strict about which authority it runs under, assert the scope set here rather than inferring it from a later 403. Keep the secret names distinct, EFA_TEST_KEY and EFA_LIVE_KEY, because a single EFA_API_KEY is what a deploy script confuses.

Note what is missing: there is no webhook step. Most CI email guides recommend webhook-first with polling as a fallback, and inside an ephemeral runner that advice does not apply. A registered endpoint must be a public HTTPS URL and a tunnel pointing at a private address is refused, so the runner has nowhere to receive a callback. Poll the event list in CI and test signature verification separately, against a stable endpoint, with a webhook.test. The dedupe rules that make a handler safe are in handling duplicate webhook events.

The test-mode counters only go up

Here is where the standard advice breaks, and it is the part no ranking page covers, because no ranking page is priced the way test mode is. Every guide says the same thing: one fresh inbox per test worker, let it expire, tear it down. That advice assumes inboxes are cheap and reclaimable. In this build they are neither.

The API exposes no route that deletes an inbox, and neither the inbox counter nor the message counter is decremented anywhere. So the three test inboxes are a lifetime allowance per workspace, not a concurrency ceiling, and the fourth POST /inboxes returns HTTP 403 test_limit_exceeded forever rather than until something expires. A matrix build with six workers, each minting its own inbox, does not fail intermittently on the fourth worker. It burns the workspace’s entire test-inbox allowance on its first green run and then never passes again.

The seven day content expiry does not rescue you. It clears stored bodies, which is why the byte counter has a release path for uploads, and it leaves retained_test_messages where it was. Read the 100 as a hundred committed test sends for the life of the workspace.

Two things follow, and together they make the job essentially free.

Isolate on a correlation token, not on an inbox. One long-lived CI inbox, a tag built from the run ID and the job name, every subject and body carrying it, every assertion filtering on it. Stale messages from previous runs become harmless because no assertion can match them. This is stricter than inbox isolation rather than weaker: a test that filters by tag cannot pass by reading a neighbour’s message, whereas a test that trusts “the newest message in my private inbox” can.

Run the loop under a draft-scoped credential. Injection touches none of the three counters, because the inbound path writes content without going near the test usage row. The counter is incremented inside the send commit, and a send that returns approval_required returns before that commit. So a job whose agent holds drafts:submit and not messages:send can run indefinitely without moving any counter, while asserting the behaviour you actually want in production. If you also need to cover the committed send path, budget it: roughly a hundred runs of that variant, ever, and the failure surfaces as test_limit_exceeded on the reply rather than on the injection, which reads like a send bug and is not. Keep one of the three inboxes unclaimed for manual debugging, because the moment the job fails you will want to inject a message by hand.

What the job costs and what it still cannot prove

It costs nothing against your billed allowance. Test mail is never charged as an email unit. An email unit is one message received or one recipient sent to, and the free tier includes 100 of them a month per verified owner identity along with one complimentary live inbox, but the quota reservation runs only for live inboxes. The test-mode counters above are a separate thing entirely: they are fixed constants in the code rather than plan entitlements, and no billing state moves them.

What a green run does not establish:

• Nothing about deliverability. No message crossed a network. SPF, DKIM, reputation and inbox placement are untested here, and provider acceptance is not proof of delivery in any environment.
• Nothing about real provider behaviour. Test sends use the internal fake provider. The states are real states, but a genuine bounce or an interrupted submission is simulated rather than observed.
• Nothing about inbound headers or inbound attachments. The injection endpoint stores no Message-ID, accepts no threading headers, and silently drops attachment_ids.
• Nothing about the model. The loop proves your integration moved the message. Whether the agent’s reply was a good idea is a separate question with separate evidence, and inbound mail is attacker-controlled text in every environment.

Start from the quickstart if you do not yet have a test project and a key, then read the request and response shapes for the send half on the developer surface. When the loop is green and the remaining unknowns are deliverability and judgement, a staging run with a routable mailbox is the right next test, and pointing a live key at the same code is the one after that.

testing api reliability

Keep reading

All guides · Documentation · Developers