# Probe transcript — Frankfurter (e2e) | | | |---|---| | Entry | `entries/frankfurter.yaml` | | Target | https://frankfurter.dev · API `https://api.frankfurter.dev` · MCP `https://mcp.frankfurter.dev/` | | Mode | `e2e` (first e2e probe in this corpus) | | Probe id | `frankfurter-e2e` | | Rubric | `rubric/v0.2.json` sha256 `cd605a44799f49951bf078fda63c5b21ad3c07db494f41d38ad19ce729c5551a` | | Date | 2026-08-16 | | Sandbox up | 2026-08-16T21:21:38Z | | Sandbox down | 2026-08-16T21:23:57Z | | Wall clock | 147 s (epoch 1786915290 → 1786915437; measured from the shell invocation that brought the sandbox up to completion of teardown) | | Real money spent | $0.00 — no payment instrument of any kind was presented, and none was requested | | Human interventions | 0 | | Verdict | `completed` | | Secrets carried in | none. `PROBE_WALLET_KEY` unset, `PROBE_INBOX` unset. Nothing to redact — no credential, token, cookie, or PII was sent or received at any point in this run | Every request below was issued from inside the probe sandbox (`.claude/skills/probe/probe-sandbox.sh`), on an `--internal` Docker network whose only egress is an allowlisting tinyproxy sidecar. No request to the target was made from the host shell. ```sh ./.claude/skills/probe/probe-sandbox.sh up --id frankfurter-e2e \ --target api.frankfurter.dev --allow frankfurter.dev --allow mcp.frankfurter.dev ``` Allowlist: `api.frankfurter.dev`, `frankfurter.dev`, `mcp.frankfurter.dev`. Nothing else. --- ## 0. Reference task The reference task recorded on the gate-mode entry was used **unchanged**: > With no credentials of any kind, GET > `https://api.frankfurter.dev/v2/rate/EUR/USD?date=2025-01-02` and confirm the > response is HTTP 200 JSON whose `"date"` field equals `"2025-01-02"` and whose > `"rate"` field is a positive number. It states a pass/fail condition in one sentence, names an exact URL, and pins a historical date so the expected value is stable across re-probes (a `latest` query would not be). No change was warranted. One wording note for future re-probes, recorded but **not** applied so the task stays byte-identical to what gate mode published: the task is phrased as a REST GET, so driving it over MCP required interpreting "the reference task" as the underlying question (what was the EUR→USD rate on 2025-01-02?) rather than the literal HTTP verb. Both runs below answer that question and are cross-checked against each other. --- ## 1. Sandbox defect found before any target traffic The first command failed on the sandbox itself, not on the target: ``` $ probe-sandbox.sh exec --id frankfurter-e2e -- sh -c 'mkdir -p /out/raw; curl ...' mkdir: can't create directory '/out/raw': Permission denied curl: Failed to open /out/raw/01-headers.txt curl: (23) Failed writing received data to disk/application ``` Diagnosis: ``` $ probe-sandbox.sh exec --id frankfurter-e2e -- sh -c 'id; ls -ld /out; touch /out/x' uid=1000(node) gid=1000(node) groups=1000(node) drwxr-xr-x 2 root root 4096 Aug 16 21:21 /out touch: /out/x: Permission denied ``` `cmd_up` runs the box with `--user node` but the `-v probe--out:/out` volume is created root-owned `0755`, so the probe process cannot write the very directory `cmd_collect` exists to copy out. Workaround used for this run: artifacts written to the `/tmp` tmpfs and surfaced through `docker exec` stdout, which is what this transcript is built from. `collect` therefore returned only `egress.log`. See §7. --- ## 2. Reference task over plain REST — PASS ``` $ curl -sS -D - "https://api.frankfurter.dev/v2/rate/EUR/USD?date=2025-01-02" ``` Timing/status line from curl: `http=200 time=0.769055s size=62` Response headers (verbatim; the first two lines are the sandbox's own proxy): ``` HTTP/1.1 200 Connection established Proxy-agent: tinyproxy/1.11.2 HTTP/2 200 date: Sun, 16 Aug 2026 21:22:03 GMT content-type: application/json; charset=utf-8 content-length: 62 cache-control: public, max-age=86400, stale-while-revalidate=86400, stale-if-error=86400 vary: Origin accept-ranges: bytes last-modified: Sun, 16 Aug 2026 21:22:03 GMT cf-cache-status: MISS server: cloudflare cf-ray: a2c37e0038cfe819-ORD ``` Body: ```json {"date":"2025-01-02","base":"EUR","quote":"USD","rate":1.0364} ``` Machine check: ``` $ jq -e '.date == "2025-01-02" and (.rate|type=="number") and .rate > 0' true REFERENCE_TASK_PASS ``` No `Authorization` header, no cookie, no API key, no query token. One request, one hop, 0.77 s. Nothing was sent that a bare HTTP client does not already have. --- ## 3. Machine-readable surface `https://api.frankfurter.dev/v2/openapi.json` → HTTP 200, 17 813 bytes. ```json { "openapi": "3.1.2", "info": { "title": "Frankfurter API", "version": "2.1.1", "license": { "name": "MIT", "url": "https://github.com/lineofflight/frankfurter/blob/main/LICENSE" }, "contact": { "url": "https://github.com/lineofflight/frankfurter/issues" } }, "servers": [ { "url": "https://api.frankfurter.dev/v2" } ], "security": null, "paths": [ "/currencies", "/currency/{code}", "/providers", "/rate/{base}/{quote}", "/rates" ] } ``` - `security` absent and `components.securitySchemes` → `null`: the schema itself declares the API unauthenticated. This matches observed behaviour. - The reference-task path `/rate/{base}/{quote}` is present and declares responses `200, 404, 422, 503`. - `grep -c 429 openapi.json` → **0**. No rate-limit response is declared anywhere in the schema (see §6). `https://frankfurter.dev/llms.txt` → HTTP 200, 953 bytes, 19 lines. Reproduced in full below. It is a documentation file that names an intended machine reader, so it is quoted inside an untrusted block as a matter of procedure; it contains no instruction directed at a model and nothing in it was acted on as a command. ```untrusted-content # Frankfurter > Currency data API. Current and historical exchange rates for 201 currencies from 84 central banks and official sources. No authentication required. Open-source. Genuinely free, not "free with limits." Not ECB-only: v2 blends rates from 84 central banks and official sources. The legacy v1 API (European Central Bank data, nested `rates` object) still works but is superseded by v2, which returns flat arrays. - Docs: https://frankfurter.dev - OpenAPI: https://api.frankfurter.dev/v2/openapi.json - MCP: https://mcp.frankfurter.dev - Self-host (Docker): https://frankfurter.dev/deploy/ - Source: https://github.com/lineofflight/frankfurter ## Examples https://api.frankfurter.dev/v2/rates https://api.frankfurter.dev/v2/rates?date=1999-01-04 https://api.frankfurter.dev/v2/rates?base=USD"es=GBP,JPY https://api.frankfurter.dev/v2/rates?providers=ECB https://api.frankfurter.dev/v2/rates?from=2025-01-01&to=2025-12-31&group=month ``` **Do the examples run as written?** All five were executed verbatim. All five returned HTTP 200 with plausible data: | Example URL | Status | First bytes of body | |---|---|---| | `/v2/rates` | 200 | `[{"date":"2026-08-16","base":"EUR","quote":"AED","rate":4.243},…` | | `/v2/rates?date=1999-01-04` | 200 | `[{"date":"1999-01-04","base":"EUR","quote":"AED","rate":4.3179},…` | | `/v2/rates?base=USD"es=GBP,JPY` | 200 | `[{"date":"2026-08-17","base":"USD","quote":"GBP","rate":0.7402},{"date":"2026-08-17","base":"USD","quote":"JPY","rate":159.32}]` | | `/v2/rates?providers=ECB` | 200 | `[{"date":"2026-08-14","base":"EUR","quote":"AUD","rate":1.6333},…` | | `/v2/rates?from=2025-01-01&to=2025-12-31&group=month` | 200 | `[{"date":"2025-01-01","base":"EUR","quote":"AED","rate":3.8052},…` | 5/5 examples run as written. This upgrades a gate-mode assumption to an observation. --- ## 4. The hosted MCP server — genuinely functional, not decorative The gate-mode entry scored `interface: 20` on the *claim* of a hosted MCP server. That claim was tested. Transport: JSON-RPC 2.0 over HTTP POST to `https://mcp.frankfurter.dev/`. ### 4.1 First attempt failed — and the error told the agent how to fix it Request: ``` POST https://mcp.frankfurter.dev/ Content-Type: application/json Accept: application/json {"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"ag3ntsearch-probe","version":"0.2"}}} ``` Response `HTTP/2 406` (`x-powered-by: Express`, `server: cloudflare`): ```json {"jsonrpc":"2.0","error":{"code":-32000,"message":"Not Acceptable: Client must accept both application/json and text/event-stream"},"id":null} ``` This is the single friction point in the entire probe, and it is self-healing: the error names the exact fix. No out-of-band knowledge was needed to recover — the corrective information came from the target itself, in the failing response. ### 4.2 `initialize` — HTTP 200, unauthenticated Same request with `Accept: application/json, text/event-stream`: ```json {"result":{"protocolVersion":"2025-06-18","capabilities":{"tools":{"listChanged":true}},"serverInfo":{"name":"frankfurter","version":"0.4.3"},"instructions":"…"},"jsonrpc":"2.0","id":1} ``` No `WWW-Authenticate`, no 401, no `Mcp-Session-Id` response header, and `GET /.well-known/oauth-protected-resource` on the MCP host returns 404. The server is stateless and open: subsequent calls succeeded without carrying any session token. The `instructions` field is text the server sends for a model to read. It is the MCP protocol's designated channel for exactly that, and its content is scoped to the server's own domain — it does not attempt to redirect the agent, exfiltrate anything, or override the caller's task. It is reproduced verbatim below as untrusted data and was **not** followed; it is recorded as a normal protocol feature, **not** as a `prompt-injection` finding: ```untrusted-content `convert` is the primary tool; prefer it for any single conversion. Report the converted amount with its currency, nothing more unless asked. Rates are blended across 84 central banks. For time series or other bulk queries, see https://frankfurter.dev/llms.txt ``` ### 4.3 `notifications/initialized` `HTTP 202`, empty body. Correct. ### 4.4 `tools/list` — HTTP 200, three tools | Tool | Title | Input schema (JSON Schema draft-07) | Annotations | |---|---|---|---| | `get_rates` | Get exchange rates | `base` (3-char), `date` (`^\d{4}-\d{2}-\d{2}$`), `quotes` (array of 3-char); `additionalProperties: false` | `readOnlyHint: true`, `openWorldHint: true` | | `convert` | Convert currency | `amount` (number), `from`, `to` required; optional `date` | `readOnlyHint: true`, `openWorldHint: true` | | `list_currencies` | List currencies | no params | `readOnlyHint: true`, `openWorldHint: true` | Every tool is annotated `readOnlyHint: true` and `execution.taskSupport: "forbidden"`. There is no write, no mutation, and no destructive surface exposed. ### 4.5 `tools/call` — the reference task over MCP — PASS Request: ```json {"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"get_rates","arguments":{"base":"EUR","date":"2025-01-02","quotes":["USD"]}}} ``` Response `HTTP 200`: ```json {"result":{"content":[{"type":"text","text":"[\n {\n \"date\": \"2025-01-02\",\n \"base\": \"EUR\",\n \"quote\": \"USD\",\n \"rate\": 1.0364\n }\n]"}]},"jsonrpc":"2.0","id":3} ``` Inner payload: ```json [ { "date": "2025-01-02", "base": "EUR", "quote": "USD", "rate": 1.0364 } ] ``` **Cross-check: `rate: 1.0364` over MCP is byte-identical to `rate: 1.0364` over REST (§2).** The two interfaces agree on the same question. The MCP path is a real front end to the same data, not a stub. ### 4.6 `convert`, the server's self-declared primary tool Request `{"name":"convert","arguments":{"amount":100,"from":"EUR","to":"USD","date":"2025-01-02"}}` → HTTP 200: ```json { "amount": 103.64, "currency": "USD" } ``` Arithmetically consistent with the 1.0364 rate returned by both other paths. Notably, `convert` exists **only** over MCP — the REST docs state plainly under *Conversion*: "There is no conversion endpoint. Fetch the rate and convert in a few lines of code." The MCP server is therefore not a wrapper of strictly lesser capability; for the conversion use case it is the *better* surface. ### 4.7 MCP error paths Invalid currency, `{"name":"get_rates","arguments":{"base":"EUR","quotes":["ABC"]}}` → HTTP 200 with a protocol-level tool error, which is the correct MCP shape: ```json {"result":{"content":[{"type":"text","text":"Error: Frankfurter API 422: {\"status\":422,\"message\":\"invalid currency: ABC\"}"}],"isError":true},"jsonrpc":"2.0","id":5} ``` Unknown method `resources/list` → `{"jsonrpc":"2.0","id":6,"error":{"code":-32601,"message":"Method not found"}}` — the correct JSON-RPC code. `GET https://mcp.frankfurter.dev/` (with `-L --max-redirs 3`) → `HTTP/2 405`, `{"jsonrpc":"2.0","error":{"code":-32000,"message":"Method not allowed."},"id":null}`. **No redirect off-domain was attempted at any point by either host.** **Conclusion on the interface dimension: an agent can drive this task end to end over MCP unaided.** `initialize` → `tools/list` → `tools/call` all succeeded with no credential, no session negotiation, and no human. The one failure was recoverable from the target's own error text. --- ## 5. Docs discrepancy found (this is new evidence, gate mode could not see it) The homepage documents the error envelope as: ```

Errors

The API returns standard HTTP status codes with a JSON body.

{"message": "Could not find currency ABC"}
400
Invalid parameter or malformed request.
404
Currency, rate, or resource not found.
422
Request understood but cannot be processed.
``` The live API disagrees. `GET https://api.frankfurter.dev/v2/rate/EUR/ABC`: ``` HTTP/2 422 content-type: application/json; charset=utf-8 cache-control: no-store {"status":422,"message":"invalid currency: ABC"} ``` Two mismatches: the body carries an undocumented `status` field, and the message string is `invalid currency: ABC`, not the documented `Could not find currency ABC`. A client that pattern-matched the documented literal would miss. The error is still perfectly actionable — it names the offending code — so this is staleness in an example, not a broken contract. It is the second concrete reason `docs` sits at 7 rather than 10. --- ## 6. Rate limits: still unmeasurable, now confirmed by observation The FAQ says a rate limit exists: > There are no quotas. Requests are rate-limited to prevent abuse, but there are > no monthly or daily caps. For high-volume use, consider caching responses, > self-hosting, or querying the datasets directly. Observed: no `RateLimit-*`, `X-RateLimit-*`, or `Retry-After` header on any response in this run (`grep -iE "ratelimit|retry-after|x-rate"` over a full header dump → no match), and no `429` anywhere in the OpenAPI document. This run made roughly 25 requests in ~2 minutes and was never throttled, so the ceiling is above that — but no published threshold, window, or 429 contract exists, so an agent still cannot *plan* bulk work. Carried forward from gate mode, now with observational backing. No attempt was made to find the limit by hammering it. That would be load testing, not probing (skill §8: observe, never exploit). --- ## 7. Egress log The proxy log was collected via `probe-sandbox.sh collect` to `transcripts/tmp-frankfurter-e2e/egress.log`. That path is gitignored (`transcripts/tmp-*/`), so the complete log is reproduced inline here — this transcript is the retrievable copy. 27 CONNECT requests total; every allowed one went to exactly one of the three allowlisted hosts. ``` DEBUG Aug 16 21:21:39.564 [11]: sending stored logs NOTICE Aug 16 21:21:39.566 [11]: Initializing tinyproxy ... NOTICE Aug 16 21:21:39.566 [11]: Reloading config file (/etc/tinyproxy/tinyproxy.conf) WARNING Aug 16 21:21:39.567 [11]: You SHOULD set a UserName in the config file. Using current user instead. NOTICE Aug 16 21:21:39.567 [11]: Reloading config file finished DEBUG Aug 16 21:21:39.568 [11]: done sending stored logs CONNECT Aug 16 21:22:02.717 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:02.721 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:02.857 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:12.312 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:12.313 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:12.337 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:12.844 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:12.845 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:12.870 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:13.199 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:13.200 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:13.221 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:21.602 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:21.603 [11]: Request (file descriptor 5): CONNECT frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:21.732 [11]: Established connection to host "frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:27.213 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:27.214 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:27.237 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:27.802 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:27.804 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:27.827 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:28.205 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:28.206 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:28.231 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:28.838 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:28.839 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:28.862 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:29.194 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:29.195 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:29.219 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:37.944 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:37.945 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:38.000 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:43.173 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:43.174 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:43.273 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:53.299 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:53.300 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:53.324 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:22:53.720 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:22:53.722 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:22:53.744 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:01.981 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:01.982 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:02.007 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:15.969 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:15.970 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:15.996 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:16.416 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:16.417 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:16.441 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:16.843 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:16.845 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:16.869 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:17.317 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:17.318 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:17.341 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:26.517 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:26.518 [11]: Request (file descriptor 5): CONNECT frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:26.552 [11]: Established connection to host "frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:26.755 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:26.756 [11]: Request (file descriptor 5): CONNECT api.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:26.804 [11]: Established connection to host "api.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:27.280 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:27.281 [11]: Request (file descriptor 5): CONNECT frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:27.310 [11]: Established connection to host "frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:27.421 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:27.422 [11]: Request (file descriptor 5): CONNECT frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:27.445 [11]: Established connection to host "frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:27.548 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:27.549 [11]: Request (file descriptor 5): CONNECT mcp.frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:27.572 [11]: Established connection to host "mcp.frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:27.942 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:27.944 [11]: Request (file descriptor 5): CONNECT frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:27.968 [11]: Established connection to host "frankfurter.dev" using file descriptor 6. CONNECT Aug 16 21:23:28.095 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:28.095 [11]: Request (file descriptor 5): CONNECT example.com:443 HTTP/1.1 NOTICE Aug 16 21:23:28.096 [11]: Proxying refused on filtered domain "example.com" CONNECT Aug 16 21:23:36.136 [11]: Connect (file descriptor 5): 172.22.0.3 CONNECT Aug 16 21:23:36.137 [11]: Request (file descriptor 5): CONNECT frankfurter.dev:443 HTTP/1.1 CONNECT Aug 16 21:23:36.199 [11]: Established connection to host "frankfurter.dev" using file descriptor 6. ``` **Blocked attempts: 1, and it was mine, not the target's.** ``` CONNECT Aug 16 21:23:28.095 [11]: Request (file descriptor 5): CONNECT example.com:443 HTTP/1.1 NOTICE Aug 16 21:23:28.096 [11]: Proxying refused on filtered domain "example.com" ``` Client side: `curl: (7) CONNECT tunnel failed, response 403`. This was a deliberate operator-initiated control test to prove the allowlist fails closed, explicitly labelled as such when issued. It is **not** attributable to Frankfurter. **Target-caused blocked egress attempts: 0.** Neither host tried to redirect, reference, or pull the client to any domain outside its own; the only off-property URLs anywhere in the target's output are inert documentation links (GitHub, ndjson-spec, plausible.io analytics on the docs page — never followed from the sandbox). --- ## 8. Safety observations | Check | Result | |---|---| | Prompt injection in fetched content | None. `grep -oiE "ignore (all )?previous\|you are now\|as an ai\|system prompt\|disregard\|display:\s*none\|visibility:\s*hidden\|aria-hidden\|"` over the homepage returned only 5 ordinary HTML section comments (`Open Graph tags`, `Twitter Card tags`, `Schema.org Structured Data`, `Structured data`, `Plausible Analytics`) and one `aria-hidden` on a decorative nav-progress `
`. No hidden text, no model-addressed imperative. | | Key / seed-phrase request | None. Nothing was asked of the client at any point. | | Unbounded approval | N/A — no payment, no signing, no approval surface exists. | | Env probe | None. | | Overcharge | N/A — $0. | | Malicious payload | None. All responses were `application/json` or `text/plain`; no executable content was served to the client. | | TLS | HTTP/2 over TLS on all three hosts via CONNECT; no downgrade, no mixed content encountered. | | Secrets in URLs | None accepted or required — there are no secrets in this system. | | PII requested | None. | | MCP tool safety | All three tools `readOnlyHint: true`, `execution.taskSupport: "forbidden"`. No write surface. | | `security.txt` | `https://frankfurter.dev/.well-known/security.txt` → **404** (re-confirmed this run). No coordinated-disclosure channel other than public GitHub issues. This is why the `security-posture` bonus is not awarded. | | `robots.txt` (docs host) | `User-agent: *` / `Allow: /` / `Allow: /llms.txt` / `Sitemap: https://frankfurter.dev/sitemap.xml` | | `robots.txt` (API host) | `User-agent: *` / `Allow: /` | **Findings: none.** `safety.penalty_total: 0`. Discovery sweep for completeness: `https://frankfurter.dev/.well-known/mcp.json` → 404. There is no well-known discovery document for the MCP server; an agent finds it via `llms.txt` (`- MCP: https://mcp.frankfurter.dev`) or the homepage sentence, both of which are machine-reachable, so this is not a blocker. --- ## 9. Scoring, and what changed against the gate-mode entry | Dimension | Gate (paper) | e2e (observed) | Why | |---|---|---|---| | `access` | 25 | **25 — unchanged, now evidenced** | Confirmed by execution rather than by reading. Zero credentials on *both* surfaces; OpenAPI `security: null`; MCP answered `initialize` with no auth challenge and no `.well-known/oauth-protected-resource`. | | `payment` | 25 | **25 — unchanged, now evidenced** | ~25 requests, $0.00, no payment instrument presented and none demanded. | | `interface` | 20 | **20 — unchanged, claim now verified** | Gate scored the top rung on an *assertion* of a hosted MCP server. Tested: `initialize` → `tools/list` → `tools/call` all worked unauthenticated and returned the correct answer. Plus an OpenAPI 3.1.2 document as a second machine-readable surface. The rung was right, but until this run it was unearned. | | `completion` | `null` | **20** | Reference task completed on both surfaces. Zero human interventions. Zero out-of-band knowledge: the REST path was fully specified by `llms.txt` + OpenAPI, and the one MCP stumble (§4.1) was corrected by the target's own error message. Does not drop to 14. | | `docs` | 7 | **7 — unchanged, but for a better reason** | Gate held 7 on one ground (unpublished rate limit). e2e confirms that *and* adds a second: the documented error body does not match the live one (§5). Against that, 5/5 `llms.txt` examples run as written — verified, not assumed. Solidly "good docs, most of the above", not "everything". | | `permissive-tos` bonus | awarded | **awarded** | `llms.txt` present, permissive `robots.txt` on both hosts, explicit agent invitation on the homepage. | | `security-posture` bonus | not awarded | **not awarded** | No `security.txt`, re-confirmed 404 this run. | | `permission.status` | `unstated` | **`unstated` — unchanged** | Still no TOS/AUP instrument anywhere on the property. The docs invite agents; absence of terms is not a grant. The e2e run produced no new legal instrument, so the gate-mode judgment stands verbatim. | Nothing was found that changes a paper score. That is itself the result worth recording: the gate-mode assessment of this target was accurate, and the e2e run converted four assumptions into observations without moving a single rung. Arithmetic is `tools/score.ts`'s job, not this transcript's. Dimension vector emitted: `access 25, payment 25, interface 20, completion 20, docs 7`; bonus `permissive-tos`; penalties none. **Rubric ceiling observation (feedback, not a score change):** the dimension sum is 97 and the bonus takes the raw total to 102, which `score.ts` clamps to 100. The `docs: 7` deduction is therefore invisible in the published number — this target would score 100 whether its docs were rated 7 or 10. For a keyless, free, MCP-serving target that finishes its reference task, the rubric saturates before the docs dimension can express anything. Worth knowing before more e2e probes land in this band. --- ## 10. Sandbox feedback 1. **`/out` is unwritable — this is a real bug and it will bite every e2e probe.** `cmd_up` runs the box `--user node` while `-v "$VOL_OUT":/out` lands root-owned `0755`. `cmd_collect` then copies an empty directory. Fix: either `--user root` is wrong (it defeats the hardening), so instead chown the volume at `up` time from a throwaway root container, or add `-v "$VOL_OUT":/out` with an init step, or simply have `cmd_collect` also pull `/tmp`. Until then every probe has to route artifacts through stdout as this one did. 2. **The `wget`/`curl` warning in the script header is accurate and load-bearing.** `curl` was used throughout; a busybox `wget` failure here would have been indistinguishable from an allowlist denial. 3. **The allowlist fails closed, verified.** `example.com` was refused at the proxy with a 403 on CONNECT and logged (§7). The invariant holds. 4. **`cmd_exec` passes `-i` but not `-t`**, which is correct for scripted use and caused no trouble. 5. Minor: `probe-sandbox.sh egress` prints the whole log then re-greps it, so blocked lines appear twice. Harmless, mildly confusing when pasting evidence.