Debugging Tips

Before submission

"No policy found. Looking for functions: ..."

The loader looked in your ZIP and found no entry point. It expects a function named act, policy, my_policy or get_action, or a class named Policy, MyPolicy, Agent or Bot with an act method. Check the names and that the file is in the ZIP:

unzip -l policy.zip
zip policy.zip policy.py     # zip the file itself

The file can sit inside a folder; the loader searches the whole ZIP.

"Policy lint failed for policy.py: ..."

The static check rejected an import or a builtin. Allowed imports are numpy, torch, jax, scipy, PIL, cv2 and a short list of standard-library modules (math, collections, functools, itertools, typing, dataclasses, enum, abc, copy, json, struct, array, random). Anything touching the system or network (os, sys, subprocess, socket, requests, pickle, ...) and the builtins exec, eval, compile, open, __import__, getattr and setattr are blocked. The message lists the violations. If you need them, serve the policy yourself.

"Syntax error in policy.py" or "Failed to execute module policy.py"

Your file does not parse, or it raised while importing. Run python policy.py locally to see the error.

"Invalid ZIP file" or "No Python files found in submission"

The upload was not a valid ZIP, or the ZIP has no .py files. Re-create it with zip.

400 from POST /verify

• scenario is required or an unknown scenario: use an id from the scenario list.
• policy_url must be a valid http or https URL or a rejection of a private address: the URL must be public. localhost and private ranges are blocked.

401 / AuthenticationError

The API key is missing or wrong. Check BOTMANIFOLD_API_KEY and that the key was not revoked.

While it runs

"Policy function failed at step N: ..."

Your policy raised an exception at step N. The text after the colon is your exception. Common causes: a missing observation key (each scenario has its own; see the scenario reference), or an array indexed past its length.

"Action shape mismatch. Expected (3,), got (7,)"

Your policy returned an action of the wrong size. The size is fixed per scenario (3 or 6 controls). Return a 1-D NumPy array of exactly that length.

"Policy execution exceeded N s timeout"

The run hit its time limit. Look for slow per-step work (model loading inside the step, large copies), and load models once in __init__. A hosted policy server gets 30 seconds per step.

The job stays running

A job stuck in running for more than 30 minutes (or building for more than 10) is failed automatically. If you host the policy server, check its logs for requests that never returned.

After it finishes

Verdict error

The run could not be graded, for example the policy crashed or the grader failed. Read the details of the report; the episode video shows how far it got.

Verdict unsafe

For a multi-episode job the composite score was below 50. Open the report and watch the episode videos: they show collisions and where the task went wrong. The per-episode scores tell you whether progress or safety was the problem.

The result looks wrong

Flag the verdict on the report page. Low-confidence verdicts are reviewed by a human and used to calibrate the grader.

Rate limits

429 means you are submitting too quickly (10 submissions per minute). Wait and retry; the SDK and BotCI back off automatically.