# Probe transcript — ntfy.sh (e2e) | | | |---|---| | Entry | `entries/ntfy.yaml` | | Target | https://ntfy.sh · docs `https://docs.ntfy.sh` | | Mode | `e2e` (gate + e2e in one run; gate reads preceded every state-changing request) | | Probe id | `ntfy-e2e` | | Rubric | `rubric/v0.2.json` sha256 `cd605a44799f49951bf078fda63c5b21ad3c07db494f41d38ad19ce729c5551a` | | Date | 2026-09-05 | | Sandbox up | 2026-09-05T13:07:20Z | | Sandbox down | 2026-09-05T13:11:29Z | | Wall clock | 249 s (epoch 1788613640 → 1788613889; measured from the shell invocation that brought the sandbox up to completion of teardown) | | Real money spent | $0.00 — no payment instrument presented, none requested | | Human interventions | 0 | | Verdict | `completed` (on the second poll; see §3–§4) | | Secrets carried in | none. `PROBE_WALLET_KEY` unset, `PROBE_INBOX` unset, `PROBE_SPEND_CAP_USD=0`. The only secret created in the run is the topic name, which is a burner; it is reproduced in full below because the message on it expires 12 h after publish and contains nothing but a nonce | Every request below was issued from inside the probe sandbox (`.claude/skills/probe/probe-sandbox.sh`), on an `--internal` Docker network whose only egress is an allowlisting tinyproxy sidecar. No request to the target was made from the host shell. Gate-mode reads of `docs.ntfy.sh/publish/`, `/subscribe/api/`, `/privacy/`, `/config/`, `ntfy.sh/`, `ntfy.sh/terms`, `ntfy.sh/robots.txt`, `ntfy.sh/.well-known/security.txt` and the raw GitHub markdown for two docs pages were additionally made with the host `WebFetch` tool (rendered-page reads only) before the sandbox came up; every quote that matters was re-retrieved as raw bytes inside the sandbox. ```sh ./.claude/skills/probe/probe-sandbox.sh up --id ntfy-e2e \ --target ntfy.sh --allow docs.ntfy.sh ``` Allowlist: `ntfy.sh`, `docs.ntfy.sh`. Nothing else. Rate-limit discipline: the documented ntfy.sh limits are 60 requests burst, refilled at one per 5 s, 250 messages/day. This run made **4** requests to `ntfy.sh` (1 publish, 3 polls) and 10 to `docs.ntfy.sh`, spaced ≥ 1 s apart. --- ## 0. Reference task > With no credentials, POST a unique message body to > `https://ntfy.sh/ag3ntsearch-probe-<24 random hex>` and then GET > `https://ntfy.sh//json?poll=1`; pass if the publish returns HTTP > 200 JSON with an `"id"` and the poll returns HTTP 200 whose one > `event=="message"` line has a `"message"` field equal to the body that was > sent. Pass/fail is one jq expression over the poll body. The topic is random so no other tenant's data can be touched and no one can guess it during the run. --- ## 1. Permission gate (read before any state was created) ### 1.1 The terms instrument `https://ntfy.sh/terms` is not a terms page — it serves the web app shell. The Terms of Service live at **`https://docs.ntfy.sh/terms/`** (HTTP 200, 40 271 bytes, raw bytes retrieved in-sandbox; found via the docs navigation, which lists "Privacy policy" and "Terms of Service"). Operator: ntfy LLC. "Last updated: January 26, 2026". The operative passages, as rendered text: ```untrusted-content Service description ntfy (pronounced "notify") is a simple HTTP-based pub-sub notification service. It allows you to send push notifications to your phone or desktop via scripts from any computer, using a REST API. The Service includes: The ntfy.sh hosted server The ntfy web application The ntfy mobile applications (Android and iOS) The ntfy command-line interface (CLI) The server software and mobile applications are open source and can be self-hosted. These Terms apply specifically to the ntfy.sh hosted service. Free tier You may use the Service without creating an account or subscribing to a paid plan. Free usage is subject to rate limits and other restrictions as described in our documentation. Acceptable use You agree not to use the Service to: Send spam, unsolicited messages, or messages to recipients who have not consented to receive them Distribute malware, viruses, or any other malicious software Transmit illegal content or content that violates the rights of others Harass, abuse, or harm another person or group Impersonate any person or entity, or falsely state or misrepresent your affiliation with a person or entity Interfere with or disrupt the Service or servers or networks connected to the Service Attempt to gain unauthorized access to the Service, other accounts, or computer systems Use the Service for any illegal purpose or in violation of any applicable laws or regulations Circumvent rate limits or other technical restrictions Use the Service in a manner that could reasonably be expected to impose an unreasonable or disproportionately large load on our infrastructure Topic names Topic names on ntfy.sh are public. If you use the Service without access controls, your topic name functions as a password. You are responsible for choosing topic names that cannot be easily guessed. We are not responsible for any unauthorized access to messages published to easily guessable topic names. ``` Scripted REST use is the *described product*; the free tier is granted without an account; the acceptable-use list bans spam, malware, abuse, unauthorized access, rate-limit circumvention and disproportionate load — none of which an agent publishing to its own random topic within the published limits does. There is no "no bots / no automation / human only" language anywhere in the document. ### 1.2 robots.txt, llms.txt, security.txt (raw bytes, in-sandbox) | URL | Status | Body | |---|---|---| | `https://ntfy.sh/robots.txt` | 404 `application/json` | `{"code":40401,"http":404,"error":"page not found"}` | | `https://ntfy.sh/.well-known/security.txt` | 404 `application/json` | same | | `https://ntfy.sh/llms.txt` | 404 `application/json` | same | | `https://docs.ntfy.sh/robots.txt` | 404 `text/html` | mkdocs 404 page | | `https://docs.ntfy.sh/llms.txt` | 404 `text/html` | mkdocs 404 page | No robots file exists on either host, so there is no disallow to honour. ### 1.3 Privacy policy (`https://docs.ntfy.sh/privacy/`, Last updated June 15, 2026) ```untrusted-content You can use ntfy without creating an account. Anonymous usage is fully supported. … Message content - Messages you publish are temporarily cached on our servers (default: 12 hours) to support message polling and to overcome client network disruptions. Messages are deleted after the cache duration expires. … IP addresses - Used for rate limiting to prevent abuse. ``` **Gate decision: `granted`.** The ToS describes scripted REST use as the service and grants account-free use. The run proceeded. **Ordering note, for honesty:** the ToS page was located and read in full *after* the four `ntfy.sh` requests in §2–§4 had already been made. The gate reads that preceded those requests were `ntfy.sh/robots.txt`, `security.txt`, `llms.txt`, `docs.ntfy.sh/publish/`, the privacy policy and the homepage (via WebFetch) — none of which carried a prohibition signal, and the publish page's own first example is the exact request the task makes. On reading the ToS, no signal was found that would have changed the decision; had one been found, this entry would be `prohibited` regardless of the traffic already sent. --- ## 2. Publish — `POST https://ntfy.sh/` — HTTP 200 Topic: `ag3ntsearch-probe-b421cc230029e9873ec0bfe3` (`openssl rand -hex 12`). Body: `ag3ntsearch probe 1788613702 nonce=c9a0e6ce4084`. ``` $ curl -sS -D - -d "$MSG" "https://ntfy.sh/$T" http=200 time=0.290003s size=192 ``` Response headers (verbatim, proxy preamble stripped): ``` HTTP/2 200 server: nginx date: Sat, 05 Sep 2026 13:08:22 GMT content-type: application/json content-length: 192 access-control-allow-origin: * ``` Body: ```json {"id":"UnA20NQTt6aF","time":1788613702,"expires":1788656902,"event":"message","topic":"ag3ntsearch-probe-b421cc230029e9873ec0bfe3","message":"ag3ntsearch probe 1788613702 nonce=c9a0e6ce4084"} ``` No header of any kind was sent beyond what curl adds. The server assigned an id and an `expires` 43 200 s (12 h) after `time`, matching the privacy policy's "default: 12 hours". No `RateLimit-*` or `Retry-After` header. --- ## 3. Poll, 1 s after publish — `GET /json?poll=1` — HTTP 200, **empty** ``` $ sleep 1; curl -sS -D - "https://ntfy.sh/$T/json?poll=1" http=200 time=0.163483s size=0 ``` ``` HTTP/2 200 server: nginx date: Sat, 05 Sep 2026 13:08:23 GMT content-type: application/x-ndjson; charset=utf-8 content-length: 0 access-control-allow-origin: * ``` Body: **0 bytes.** ``` $ jq -e --arg m "$MSG" 'select(.event=="message") | .message == $m' 02-poll.body REFERENCE_TASK_FAIL ``` The publish had returned an id, so the message existed; the poll, on the exact documented endpoint with the documented parameter, one second later, returned nothing. On the literal reference task this is a fail. Nothing in the publish or subscribe documentation says a freshly published message may not yet be pollable. --- ## 4. Re-poll, 80 s after publish — PASS (twice) Identical request, no parameter changed: ``` $ curl -sS -D - "https://ntfy.sh/$T/json?poll=1" # 13:09:42Z http=200 time=0.204496s size=192 {"id":"UnA20NQTt6aF","time":1788613702,"expires":1788656902,"event":"message","topic":"ag3ntsearch-probe-b421cc230029e9873ec0bfe3","message":"ag3ntsearch probe 1788613702 nonce=c9a0e6ce4084"} ``` And with the documented default made explicit: ``` $ curl -sS -D - "https://ntfy.sh/$T/json?poll=1&since=all" # 13:09:43Z http=200 time=0.151843s size=192 {"id":"UnA20NQTt6aF",… identical …} ``` ``` task/08-poll2.body PASS task/09-poll3.body PASS ``` Same id `UnA20NQTt6aF`, same message string, byte-identical to the publish response. The reference assertion holds on both re-polls. ### 4.1 Why — the mechanism is documented, on the wrong page `https://docs.ntfy.sh/config/` (self-hosting configuration), rendered text: ```untrusted-content In addition to that, for very high load servers (such as ntfy.sh), it may be beneficial to write messages to the cache in batches, and asynchronously. This can be enabled with the cache-batch-size and cache-batch-timeout. If you start seeing database locked messages in the logs, you should probably enable that. Here's how ntfy.sh has been tuned in the server.yml file: cache-batch-size : 25 … … cache-batch-timeout NTFY_CACHE_BATCH_TIMEOUT duration 0s Timeout for batched async writes to the message cache (if zero, writes are synchronous) … cache-duration NTFY_CACHE_DURATION duration 12h Duration for which messages will be buffered before they are deleted. This is required to support the since=... and poll=1 parameter. ``` ntfy.sh writes messages to its cache in asynchronous batches; `poll=1` reads from that cache. A message can therefore be acknowledged with an id and not yet be readable. The publish page (`/publish/`) and the subscribe page (`/subscribe/api/`) — the two pages an API consumer reads — say nothing about this; the subscribe page says only that `poll=1` "will end after all available messages have been read" and that `since=` "defaults to since=all". An agent verifying its own publish through the API docs alone would conclude the publish was lost. **Completion rung: 14.** The task completed with zero human input, but the agent needed a fact — retry after a delay, because the cache is write-behind — that the API-facing documentation does not supply. It is on the property (the self-hosting config page), which is why this is not scored lower as a workaround; it is not where a consumer looks, which is why it is not 20. Re-probes should bound the poll ("within 120 s") so the score measures the service rather than this race. --- ## 5. Docs surface `https://docs.ntfy.sh/publish/` → 200, 529 561 bytes. Limitations section, rendered text (the part relevant to ntfy.sh): ```untrusted-content Limitations There are a few limitations to the API to prevent abuse and to keep the server healthy. Almost all of these settings are configurable via the server side rate limiting settings. Most of these limits you won't run into, but just in case, let's list them all: Message length — Each message can be up to 4,096 bytes long. Longer messages are treated as attachments. Requests — By default, the server is configured to allow 60 requests per visitor at once, and then refills the your allowed requests bucket at a rate of one request per 5 seconds. Daily messages — By default, the number of messages is governed by the request limits. This can be overridden. On ntfy.sh, the daily message limit is 250. E-mails — By default, the server is configured to allow sending 16 e-mails per visitor at once, and then refills the your allowed e-mail bucket at a rate of one per hour. On ntfy.sh, the daily limit is 5. Subscription limit — By default, the server allows each visitor to keep 30 connections to the server open. Attachment size limit — … On ntfy.sh, the attachment size limit is 2 MB, and the per-visitor total is 20 MB. Title and tag size — The message title is limited to 1 KB, and all tags combined to 512 bytes. Requests exceeding either are rejected with HTTP 400. ``` `https://docs.ntfy.sh/subscribe/api/` → 200, 66 161 bytes: ```untrusted-content Poll for messages You can also just poll for messages if you don't like the long-standing connection using the poll=1 query parameter. The connection will end after all available messages have been read. This parameter can be combined with since= (defaults to since=all). curl -s "ntfy.sh/mytopic/json?poll=1" ``` Machine-readable schema: `grep -oiE "openapi|swagger|mcp[^a-z]|llms\.txt|json schema"` over the publish page → **1 hit**, and it is a hyperlink to swagger.io's bearer-authentication explainer inside the access-token section, not a schema. No OpenAPI document, no MCP server, no llms.txt. **Interface rung: 12** — documented REST, no schema file. Do the examples run as written? The publish example (`curl -d "…" ntfy.sh/mytopic`) and the poll example (`curl -s "ntfy.sh/mytopic/json?poll=1"`) are exactly the two requests this run made, with the topic swapped; both returned 200. Rate limits are documented with numbers specific to ntfy.sh. Errors are JSON with a code (`{"code":40401,"http":404,"error":"page not found"}`). Missing: llms.txt, any schema, and the publish→poll lag. **Docs rung: 7.** --- ## 6. Egress log Collected via `probe-sandbox.sh collect` to `transcripts/tmp-ntfy-e2e/egress.log` (gitignored, so reproduced here in full). **14 CONNECT requests, every one to an allowlisted host. Blocked: 0.** ``` DEBUG Sep 05 13:07:21.399 [11]: sending stored logs NOTICE Sep 05 13:07:21.400 [11]: Initializing tinyproxy ... NOTICE Sep 05 13:07:21.401 [11]: Reloading config file (/etc/tinyproxy/tinyproxy.conf) WARNING Sep 05 13:07:21.403 [11]: You SHOULD set a UserName in the config file. Using current user instead. NOTICE Sep 05 13:07:21.403 [11]: Reloading config file finished DEBUG Sep 05 13:07:21.404 [11]: done sending stored logs CONNECT Sep 05 13:07:21.698 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:07:21.703 [11]: Request (file descriptor 5): CONNECT ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:07:21.758 [11]: Established connection to host "ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:07:21.843 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:07:21.845 [11]: Request (file descriptor 5): CONNECT ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:07:21.884 [11]: Established connection to host "ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:07:22.006 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:07:22.007 [11]: Request (file descriptor 5): CONNECT ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:07:22.046 [11]: Established connection to host "ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:07:22.129 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:07:22.131 [11]: Request (file descriptor 5): CONNECT docs.ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:07:22.142 [11]: Established connection to host "docs.ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:07:22.331 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:07:22.331 [11]: Request (file descriptor 5): CONNECT docs.ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:07:22.344 [11]: Established connection to host "docs.ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:07:22.413 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:07:22.413 [11]: Request (file descriptor 5): CONNECT docs.ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:07:22.426 [11]: Established connection to host "docs.ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:08:22.260 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:08:22.261 [11]: Request (file descriptor 5): CONNECT ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:08:22.324 [11]: Established connection to host "ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:08:23.562 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:08:23.563 [11]: Request (file descriptor 5): CONNECT ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:08:23.626 [11]: Established connection to host "ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:09:42.034 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:09:42.036 [11]: Request (file descriptor 5): CONNECT ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:09:42.076 [11]: Established connection to host "ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:09:43.252 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:09:43.254 [11]: Request (file descriptor 5): CONNECT ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:09:43.318 [11]: Established connection to host "ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:09:43.419 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:09:43.420 [11]: Request (file descriptor 5): CONNECT docs.ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:09:43.437 [11]: Established connection to host "docs.ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:09:43.572 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:09:43.573 [11]: Request (file descriptor 5): CONNECT docs.ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:09:43.585 [11]: Established connection to host "docs.ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:11:28.283 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:11:28.284 [11]: Request (file descriptor 5): CONNECT docs.ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:11:28.299 [11]: Established connection to host "docs.ntfy.sh" using file descriptor 6. CONNECT Sep 05 13:11:28.428 [11]: Connect (file descriptor 5): 172.26.0.3 CONNECT Sep 05 13:11:28.430 [11]: Request (file descriptor 5): CONNECT docs.ntfy.sh:443 HTTP/1.1 CONNECT Sep 05 13:11:28.444 [11]: Established connection to host "docs.ntfy.sh" using file descriptor 6. ``` (The three 13:07:21 `ntfy.sh` CONNECTs are the robots/security/llms gate reads; the four task requests are 13:08:22, 13:08:23, 13:09:42, 13:09:43.) Neither host attempted to redirect the client off-property. **Target-caused blocked egress attempts: 0.** --- ## 7. Safety observations | Check | Result | |---|---| | Prompt injection / hidden text | None. `grep -oiE "ignore (all )?previous\|you are now\|as an ai\|system prompt\|disregard\|display:\s*none\|visibility:\s*hidden"` over the publish page → one `display: none;` inside the mkdocs-material stylesheet, nothing else. The ToS, privacy, config and subscribe pages contain no model-addressed imperatives. | | Key / seed-phrase request | None. | | Unbounded approval | N/A — no payment or signing surface on the free tier. | | Env probe | None. | | Overcharge | N/A — $0. | | Malicious payload | None. Bodies `application/json`, `application/x-ndjson`, `text/html` (docs). | | TLS | HTTP/2 over TLS on both hosts via CONNECT; no downgrade. | | Secrets in URLs | The topic name is in the URL by design and is the only secret; the docs say so plainly ("the topic is essentially a password"). Recorded as a design property the agent controls, not a finding — a random topic is a burner. | | PII requested | None. | | Rate-limit headers | None on any response, so budget is not readable from the wire. | | `security.txt` | 404 on both hosts. `security-posture` bonus not awarded. | | `robots.txt` | Absent on both hosts. | **Findings: none.** `safety.penalty_total: 0`. --- ## 8. Scoring rationale | Dimension | Rung | Why | |---|---|---| | `access` | **25** | Nothing beyond HTTP. No header, key, cookie or account at any point. | | `payment` | **25** | Free tier, no account, no card. ToS: "You may use the Service without creating an account or subscribing to a paid plan." | | `interface` | **12** | Documented REST, no OpenAPI or other schema file, no MCP. | | `completion` | **14** | Completed with zero human input, but the first poll — on the documented endpoint, 1 s after a 200 publish — returned nothing, and the retry-after-delay fact that fixed it is absent from the API-facing docs (§4.1). | | `docs` | **7** | Runnable examples, numeric ntfy.sh rate limits, JSON errors; but no llms.txt, no schema, and the write-behind cache lag is undocumented where a consumer would read. | | `permissive-tos` | **awarded** | ToS names scripted REST use as the product and grants account-free use. | | `security-posture` | **not awarded** | No `security.txt`. | | `permission.status` | **`granted`** | See §1. | Dimension vector emitted: `access 25, payment 25, interface 12, completion 14, docs 7`; bonus `permissive-tos`; penalties none. Arithmetic belongs to `tools/score.ts`. **As the user:** the publish half is as close to a library call as HTTP gets — one POST, no setup, a JSON receipt with an id. The read-back half bit me: I polled a second later, got an empty 200, and if I had trusted the docs I would have concluded the publish was lost. Once I knew to wait, it was flawless, and `since=all` plus a bounded retry makes it dependable. I would reach for it again for fire-and-forget notifications to a human; I would not build a request/response loop on `poll=1` without that retry, and I would want the lag mentioned on the subscribe page. --- ## 9. Sandbox feedback 1. `/out` writable, `collect` returned `gate/` and `task/` intact. 2. Busybox `grep -E` rejects `{0,N}` for N > 255 ("Invalid contents of {}"); use `awk 'match($0,/…/){print substr($0,RSTART,N)}'` inside the box. 3. `egress` prints the whole log then re-greps it; harmless. 4. The allowlist saw nothing to block this run; the fail-closed invariant was last exercised on 2026-08-16 (frankfurter transcript §7).