# Probe transcript — arXiv API (e2e) | | | |---|---| | Entry | `entries/arxiv.yaml` | | Target | API `https://export.arxiv.org/api/query` · docs https://info.arxiv.org/help/api · ToU https://info.arxiv.org/help/api/tou.html | | Mode | `e2e` | | Probe id | `arxiv-e2e` | | Rubric | `rubric/v0.2.json` sha256 `cd605a44799f49951bf078fda63c5b21ad3c07db494f41d38ad19ce729c5551a` | | Date | 2026-09-05 | | Sandbox up | 2026-09-05T13:12:23Z | | Sandbox down | 2026-09-05T13:12:51Z | | Wall clock | 28 s (epoch 1788613943 → 1788613971), of which 24 s was deliberate sleeping to honour the ToU's one-request-per-three-seconds rule | | Real money spent | $0.00 | | Human interventions | 0 | | Verdict | `completed` | | Secrets carried in | none. `PROBE_WALLET_KEY` unset, `PROBE_INBOX` unset, `PROBE_SPEND_CAP_USD=0`. Nothing to redact | | User-Agent sent | `ag3ntsearch-probe/0.2 (https://ag3ntsearch.com; agent-nativeness index probe) curl` on every request | | Rate discipline | 9 requests to arxiv.org hosts, strictly sequential (one connection), each preceded by `sleep 3` | Every request below was issued from inside the probe sandbox on an `--internal` Docker network whose only egress is an allowlisting tinyproxy sidecar. No request to any target host was made from the host shell. ```sh ./.claude/skills/probe/probe-sandbox.sh up --id arxiv-e2e \ --target export.arxiv.org --allow info.arxiv.org --allow arxiv.org ``` Allowlist: `export.arxiv.org`, `info.arxiv.org`, `arxiv.org`. Nothing else. --- ## 0. Permission gate (gate mode, before any sandbox traffic) Read via WebFetch on 2026-09-05: - **https://info.arxiv.org/help/api/tou.html** — "By using arXiv APIs, you are agreeing to the terms and conditions outlined on this page." Rate limits: "When using the legacy APIs (including OAI-PMH, RSS, and the arXiv API), make no more than one request every three seconds, and limit requests to a single connection at a time." "Things that you can (and should!) do" includes "Retrieve, store, transform, and share descriptive metadata about arXiv e-prints." and "Provide tools and services to users that helps them to discover or be notified about arXiv e-prints." "Things that you must not do" includes "Attempt to circumvent rate limits." The full text was re-fetched from the sandbox (§4) and matches. - **https://info.arxiv.org/help/api/user-manual.html** — base URL `http://export.arxiv.org/api/query`; "In cases where the API needs to be called multiple times in a row, we encourage you to play nice and incorporate a 3 second delay in your code."; "Everything returned by the API in the body of the HTTP responses is Atom 1.0, including errors." - **https://export.arxiv.org/robots.txt** — `User-agent: *` / `Disallow: /`. - **https://arxiv.org/robots.txt** — `User-agent: *`, `Crawl-delay: 15`, a list of Allows (`/abs`, `/pdf`, `/list`…) and Disallows including **`Disallow: /api`** and `Disallow: /search`. - **https://info.arxiv.org/robots.txt** — HTTP 404 (no robots file on the docs host). Decision: `granted`, with binding conditions. The operator publishes a Terms of Use for exactly this endpoint that welcomes programmatic clients, prescribes their rate and connection count, and enumerates what they may build. Both robots.txt files disallow the API path to crawlers; that is read as a crawler-indexing directive — search engines spidering `/api/query?…` URLs would violate the very rate the ToU sets — not as a statement that the operator does not want API clients, which the ToU contradicts in its first paragraph ("Providing free Application Programming Interfaces (APIs) helps us to advance that mission"). Recorded verbatim on the entry as a counter-signal; a consumer who treats robots.txt as binding on all non-human clients regardless of a more specific grant should treat this entry as unresolved. Consequences applied to the run: - **`sleep 3` before every request to any arxiv.org host**, sequential, one connection. The egress log (§6) shows the spacing: 13:12:24, :27, :30, :34, :37, :40, :43, :47, :50. - **Nothing on export.arxiv.org except the API path and `/robots.txt`.** `/llms.txt` and `/.well-known/security.txt` were not requested there. --- ## 1. Reference task — PASS > With no credentials of any kind, GET > `https://export.arxiv.org/api/query?search_query=all:agent&max_results=1` and > confirm the response is HTTP 200 with an Atom 1.0 `` containing exactly one > `` whose `` is a non-empty string. Authored for this run. `https` rather than the documented `http` (see §2). ``` GET https://export.arxiv.org/api/query?search_query=all:agent&max_results=1 http=200 time=0.155s size=2957 ``` Response headers (verbatim; first two lines are the sandbox proxy): ``` HTTP/1.1 200 Connection established Proxy-agent: tinyproxy/1.11.2 HTTP/2 200 via: 1.1 google, 1.1 varnish, 1.1 varnish, 1.1 varnish x-cloud-trace-context: 297b6b8544fb461167987d2ed4b6ff12 content-type: application/atom+xml; charset=utf-8 server: Google Frontend accept-ranges: bytes age: 0 date: Sat, 05 Sep 2026 13:12:24 GMT x-served-by: cache-lga21944-LGA, cache-lga21944-LGA, cache-lga21974-LGA, cache-msp11833-MSP x-cache: MISS, HIT, MISS x-cache-hits: 0, 1, 0 x-timer: S1788613945.840118,VS0,VE28 strict-transport-security: max-age=300 content-length: 2957 ``` Body (verbatim, abstract elided at `…`): ```xml <?xml version='1.0' encoding='UTF-8'?> <feed xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/" xmlns:arxiv="http://arxiv.org/schemas/atom" xmlns="http://www.w3.org/2005/Atom"> <id>https://arxiv.org/api/ml33m7s5M0A73YLzE8Fh2XvKA+w</id> <title>arXiv Query: search_query=all:agent&id_list=&start=0&max_results=1 2026-09-05T12:56:21Z 1 77380 0 http://arxiv.org/abs/2203.08975v2 A Survey of Multi-Agent Deep Reinforcement Learning with Communication 2024-10-18T10:14:58Z Communication is an effective mechanism for coordinating the behaviors of multiple agents, … 2022-03-16T22:39:46Z 34 pages, 5 figures, 13 tables; published on Autonomous Agents and Multi-Agent Systems Auton. Agents Multi Agent Syst. 38(1): 4 (2024) Changxi Zhu Mehdi Dastani Shihan Wang ``` Machine check (node, regex over the body): ```json {"isAtom":true,"entries":1,"totalResults":"77380","entry_id":"http://arxiv.org/abs/2203.08975v2","entry_title":"A Survey of Multi-Agent Deep Reinforcement Learning with Communication"} REFERENCE_TASK_PASS ``` No `Authorization` header, no cookie, no key. One request, 0.155 s. --- ## 2. The documented base URL is `http://` — what happens? Every example in the user manual and the basics page is `http://export.arxiv.org/api/query…`. ``` GET http://export.arxiv.org/api/query?search_query=all:agent&max_results=1 http=301 time=0.046s size=0 HTTP/1.1 301 Moved Permanently Location: https://export.arxiv.org/api/query?search_query=all:agent&max_results=1 Strict-Transport-Security: max-age=300 ``` Cleartext is refused with an immediate upgrade and a (short, 5-minute) HSTS. A client that follows redirects works as documented; one that does not gets an empty 301. Documentation staleness, not a transport weakness; recorded, not penalised. (This request appears in the egress log as a proxied `GET http://…` line rather than a `CONNECT`, which is how tinyproxy logs cleartext.) --- ## 3. Docs: example runs as written; errors are self-describing **Example** `search_query=all:electron` (the manual's first example) → HTTP 200, 23,335 bytes, `entries=10 totalResults=185795`. Default page size 10 as documented. 1/1. **Error path** `search_query=all:agent&start=notanumber` → **HTTP 400**, `content-type: application/atom+xml; charset=utf-8`: ```xml https://arxiv.org/ arXiv Search Results 2026-09-05T13:12:34Z 1 1 0 https://arxiv.org/api/errors#start_must_be_an_integer Error 2026-09-05T13:12:34Z start must be an integer arXiv api core ``` Exactly as the manual promises ("Atom 1.0, including errors"): the failure is a parseable entry with a human sentence and a link into the error docs. A client that handles the happy path handles this for free. This is the best error contract seen on a keyless API in this wave. --- ## 4. Permission source, re-fetched from the sandbox (verbatim, HTML stripped) `GET https://info.arxiv.org/help/api/tou.html` → HTTP 200, 85,842 bytes. The operative text, reproduced as data: ```untrusted-content Terms of Use for arXiv APIs Our mission is to provide rapid dissemination of scientific results at no cost to authors or readers. Providing free Application Programming Interfaces (APIs) helps us to advance that mission by enabling platforms and projects that extend the discoverability of arXiv e-prints and provide valuable services to scientists and interested readers. Our APIs include our OAI-PMH interface, RSS feeds, the legacy arXiv API, bulk data downloads, and SWORD bulk deposit API. Understanding and observing the following terms of use will help us to ensure that arXiv continues to be able to fulfill its mission. By using arXiv APIs, you are agreeing to the terms and conditions outlined on this page. You understand that e-prints from arXiv are subject to all applicable copyright protections. […] You are free to use descriptive metadata about arXiv e-prints under the terms of the Creative Commons Universal (CC0 1.0) Public Domain Declaration. […] You agree to abide by the arXiv Code of Conduct in all of your interactions with arXiv staff and those involved in its operation, and understand that if you violate the CoC we may prohibit you from using arXiv APIs. You agree to respect limitations on use of arXiv APIs, including rate limits and authorization mechanisms. If we think that you are attempting to circumvent those limitations, that your use of arXiv APIs threatens normal operation or availability of the arXiv platform, or that you are doing something illegal or unethical, we may further limit or block your access. Limitations Rate limits Please note that the following rate limits apply to all of the machines under your control as a whole. You should not attempt to overcome these limits by increasing the number of machines used to make requests. If your use-case requires a higher request rate, please contact our support team. When using the legacy APIs (including OAI-PMH, RSS, and the arXiv API), make no more than one request every three seconds, and limit requests to a single connection at a time. These limits may change in the future. Things that you can (and should!) do - Retrieve, store, transform, and share descriptive metadata about arXiv e-prints. - Retrieve, store, and use the content of arXiv e-prints for your own personal use, or for research purposes. - Provide tools and services to users that helps them to discover or be notified about arXiv e-prints. For example: A better search interface; A mobile app that notifies users about e-prints that might be of interest to them; A visualization of topics in arXiv e-prints over time; A citation graph using bibliographic references from e-prints. - Build other kinds of interfaces that help users to interact with arXiv in new and useful ways, leveraging our APIs. - Direct users to arXiv.org to retrieve e-print content (PDF, source files, etc). We encourage you to link to the abstract page. Things that you must not do - Store and serve arXiv e-prints (PDFs, source files, or other content) from your servers, unless you have the permission of the copyright holder or are permitted to do so by the license with which the e-print was submitted. […] - Represent your project as endorsed or supported by arXiv.org without our permission. - Attempt to circumvent rate limits. - Use someone else's credentials to access arXiv APIs. ``` Nothing in it is addressed to a model or attempts to redirect the reader's task; it is a usage policy and was treated as one. `GET https://info.arxiv.org/help/api/user-manual.html` → 200. Contains: "In cases where the API needs to be called multiple times in a row, we encourage you to play nice and incorporate a 3 second delay in your code." and "returns results in the Atom 1.0 format, and not HTML." `GET https://info.arxiv.org/help/api/basics.html` → 200. "Please review the Terms of Use for arXiv APIs before using the arXiv API." --- ## 5. Discovery surface | URL | Result | |---|---| | `https://export.arxiv.org/robots.txt` | 200, 26 bytes: `User-agent: * ` / `Disallow: /` | | `https://arxiv.org/.well-known/security.txt` | **404** (HTML 404 page, 7,237 bytes). `security-posture` bonus not awarded. | | `https://export.arxiv.org/llms.txt`, `/.well-known/security.txt` | **not requested** — robots-disallowed. | | OpenAPI / JSON Schema | none published. The Atom feed uses the `http://arxiv.org/schemas/atom` namespace but no schema document is served for it. | | MCP | none official. | --- ## 6. Egress log Collected to `transcripts/tmp-arxiv-e2e/egress.log` (gitignored), reproduced in full. 10 proxied requests: 9 to the three allowlisted hosts (note the ≥3 s spacing), 1 operator-initiated control test that was refused. ``` DEBUG Sep 05 13:12:24.331 [11]: sending stored logs NOTICE Sep 05 13:12:24.332 [11]: Initializing tinyproxy ... NOTICE Sep 05 13:12:24.332 [11]: Reloading config file (/etc/tinyproxy/tinyproxy.conf) WARNING Sep 05 13:12:24.334 [11]: You SHOULD set a UserName in the config file. Using current user instead. NOTICE Sep 05 13:12:24.335 [11]: Reloading config file finished DEBUG Sep 05 13:12:24.335 [11]: done sending stored logs CONNECT Sep 05 13:12:24.691 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:24.693 [11]: Request (file descriptor 5): CONNECT export.arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:24.789 [11]: Established connection to host "export.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:27.884 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:27.885 [11]: Request (file descriptor 5): GET http://export.arxiv.org/api/query?search_query=all:agent&max_results=1 HTTP/1.1 CONNECT Sep 05 13:12:27.918 [11]: Established connection to host "export.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:30.943 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:30.945 [11]: Request (file descriptor 5): CONNECT export.arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:30.962 [11]: Established connection to host "export.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:34.137 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:34.139 [11]: Request (file descriptor 5): CONNECT export.arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:34.209 [11]: Established connection to host "export.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:37.350 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:37.363 [11]: Request (file descriptor 5): CONNECT export.arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:37.382 [11]: Established connection to host "export.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:40.443 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:40.445 [11]: Request (file descriptor 5): CONNECT arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:40.502 [11]: Established connection to host "arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:43.890 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:43.892 [11]: Request (file descriptor 5): CONNECT info.arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:43.986 [11]: Established connection to host "info.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:47.261 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:47.262 [11]: Request (file descriptor 5): CONNECT info.arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:47.339 [11]: Established connection to host "info.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:50.643 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:50.645 [11]: Request (file descriptor 5): CONNECT info.arxiv.org:443 HTTP/1.1 CONNECT Sep 05 13:12:50.681 [11]: Established connection to host "info.arxiv.org" using file descriptor 6. CONNECT Sep 05 13:12:51.091 [11]: Connect (file descriptor 5): 172.20.0.3 CONNECT Sep 05 13:12:51.092 [11]: Request (file descriptor 5): CONNECT example.com:443 HTTP/1.1 NOTICE Sep 05 13:12:51.093 [11]: Proxying refused on filtered domain "example.com" ``` **Blocked attempts: 1, and it was mine.** `CONNECT example.com:443` → refused, client `curl: (7) CONNECT tunnel failed, response 403`. Fail-closed control test. **Target-caused blocked egress attempts: 0.** The target's only cross-host reference is the 301 from `http://export.arxiv.org` to `https://export.arxiv.org` (same host). --- ## 7. Safety observations | Check | Result | |---|---| | Prompt injection / hidden text | None. The scan over all nine bodies matched only MkDocs template comments (``, ``, etc.) in the three docs pages and nothing in any API response. Zero occurrences of `llm`, `assistant`, `ai agent`. | | Key / seed-phrase request | None. | | Unbounded approval | N/A. | | Env probe | None. | | Overcharge | N/A — $0. | | Malicious payload | None. API responses `application/atom+xml`; docs `text/html`. | | TLS | HTTPS on every request; cleartext base URL 301s to HTTPS with HSTS `max-age=300`. | | Secrets in URLs | None exist. | | PII requested | None. The ToU mentions collecting "your name and email address" for developer support, but nothing on the query API asks for either. | | `security.txt` | 404 on arxiv.org. | | Rate-limit headers | None (`RateLimit-*`, `X-RateLimit-*`, `Retry-After` absent). The limit is a written term, not an enforced header — an agent must self-police. | **Findings: none.** `safety.penalty_total: 0`. --- ## 8. Scoring rationale | Dimension | Rung | Why | |---|---|---| | `access` | **25** | Nothing beyond HTTP; no key anywhere on the query API. | | `payment` | **25** | Free, no instrument. | | `interface` | **12** | Documented REST returning a standard format (Atom + OpenSearch), but no OpenAPI/schema file and no MCP. | | `completion` | **20** | One request, no human, no out-of-band knowledge — the manual gives the exact URL and the ToU gives the rate. | | `docs` | **7** | Thorough manual; example runs verbatim; rate limit documented precisely; errors actionable and self-describing. Missing for 10: no llms.txt, no machine-readable schema, and every documented URL is stale `http://`. | | `permissive-tos` | **awarded** | An explicit API Terms of Use that welcomes programmatic use and lists what you "can (and should!) do". | | `security-posture` | not awarded | No security.txt. | | `permission.status` | `granted` | See §0; robots.txt counter-signal recorded. | Dimension vector emitted: `access 25, payment 25, interface 12, completion 20, docs 7`; bonus `permissive-tos`; penalties none. **Rubric observation (feedback, not a score change):** the ToU's 3-second / single-connection rule is the dominant practical cost of this API for an agent and the rubric has no dimension that can express it — a documented limit *raises* `docs`. That is arguably right (a stated term beats an undocumented 429) but it means two APIs with 1 req/3 s and 1000 req/s score identically here. --- ## 9. Sandbox feedback 1. `/out` writable; `collect` worked. 2. tinyproxy logs cleartext requests with their full URL (`GET http://export.arxiv.org/api/query?…`) — useful, and worth knowing that a probe carrying a secret in a cleartext query string would put it in the published egress log. 3. `sleep` inside the box is the only way to honour a per-host rate term; the script has no rate-shaping option. Fine for now, but a `--min-interval =` on `up` would make the discipline enforced rather than cooperative.