10. Test and diagnose the integration
A clip appearing once is useful evidence, but it does not prove that an application handles retries, cancellation or restart. Studio separates fast local tests from managed-host integration and real inference qualification.
Build confidence in layers
| Layer | Exercise | What it proves |
|---|---|---|
| Pure contracts and planning | Request shapes, modes, duration/frame grids, reference collisions, plan digests | Resolution is consistent without network access. |
| Fake Skulk™ HTTP service | Catalog/state, job creation/status, media, chat and cancellation | Adapter behavior and bounded decoding. |
| SDK operations | Reservation, duplicate starts, scope, uncertainty, journal restart | Work is tracked without blind replay. |
| Application storage/HTTP | Asset import, digest checks, retention, ticket paths, origin checks | Files and interface boundaries behave correctly. |
| Managed host | Real startup, IPC, health, invocation and shutdown | The child actually integrates with its owner. |
| Signed package | Build validation, install, manifest and schema agreement | The installed artifact matches the advertised contract. |
| Browser | Client URL resolution, stale replies, polling, batches and controls | Operator actions reach the intended application behavior. |
| Real inference | Mounted models, long jobs, cancellation, output review and soak | The selected model/engine deployment works in practice. |
A fake backend can prove a render was submitted once. It cannot prove a GPU produces useful video.
Read the existing tests as examples
Studio's Python tests and integration tests cover:
- A single submission followed by collection of the completed clip.
- Accepted-job reconciliation after reopening the journal.
- Refusal when the reviewed plan no longer matches.
- Digests binding reviewed settings and references.
- Corrupt, oversized or uncollectable output.
- Unknown backend states remaining outstanding instead of guessed completion.
- Outages before submission being definite failures.
- Asset validation, reference uploads, keep/delete behavior and cleanup races.
The managed-owner tests additionally check that the child remains outside Skulk™ implementation imports. A capability author calls the supported boundary instead of importing worker internals.
Some integration tests currently depend on managed-host and publisher tooling distributed separately from the authoring SDK. A public Studio repository by itself does not install that tooling. Run the layer you have prerequisites for; report skipped layers explicitly.
Keep generated contracts consistent
Once the repository's pinned Python dependencies are available, its schema/web checks are:
uv run python tools/export_schemas.py
npm --prefix web run gen:api
npm --prefix web run check
web's check runs lint, type checking, tests and a production build. Review schema/type diffs when you change Python contracts. Type generation succeeding is not proof that the server implements the new behavior.
Run focused Python tests for the boundary you changed, then the repository's required full validation before publishing an artifact. See the Studio repository's contribution instructions for the exact dependency and host setup.
Diagnose by the last proven boundary
| Symptom | First evidence to inspect |
|---|---|
| Capability missing | Installed manifest, announced descriptors, child startup/health and host admission. |
| Interface missing | Declared surface id, reported URL, serving address and browser reachability. |
| Plan refused | Card mode/bounds, references, ready placement and current plan digest. |
| Start returns but no clip | Operation state and submission state, backend reference and adapter observations. |
| Job completed but operation failed | Output metadata, bounded download, digest/size check and local storage. |
| Cancel seems slow | Actual backend terminal state; a request acknowledgment is not completion. |
| Refinement failed after restart | Whether the owned in-process chat task survived—it cannot at this revision. |
| UI shows old status | Client/store generation guards, list hydration and polling lifecycle. |
Do not restart a healthy child repeatedly to repair a missing model. Do not resubmit uncertain work merely because the interface has no clip yet. Find the last boundary with confirmed evidence, then investigate the next one.
Your minimum failure exercise
Before publishing your own application, deliberately lose a start reply, interrupt collection, restart after backend acceptance and race cancellation with completion. Check that the application preserves uncertainty, does not duplicate work and never reports success for corrupt output.
Next: packaging and installation.