One successful reply is the beginning of verification, not the end. It proves that several layers worked together at once, and it hides which of them is fragile.
This guide checks the Agent Runtime, its Communication Connections, provider delivery, Conversation history, and Tool Call telemetry as separate operational paths, so a failure points at one layer instead of the whole system.
What you will verify
By the end of this guide, you will have confirmed that:
- The Agent has the intended Runtime and pinned configuration
- The Agent Runtime starts and reports healthy
- The intended Communication Connection is enabled and healthy
- One allowed interaction succeeds and one blocked interaction does not
- The Conversation appears under the correct Connection
- Tool Call telemetry and cost attribution are checked separately
The verification layers
Agent verification has distinct layers, and each one has its own source of truth.
| # | Layer | What it proves |
|---|---|---|
| 1 | Agent configuration | The Agent is the one you intended to build |
| 2 | Agent lifecycle and Runtime health | Runtime resources were created and the Runtime is reachable |
| 3 | Communication Connection configuration and provider health | A provider session or webhook path is configured and usable |
| 4 | Inbound and outbound communication delivery | Provider messages reach the Agent and replies reach the provider |
| 5 | Connection-scoped Conversation persistence | The exchange is recorded under the correct Connection and location |
| 6 | Tool Call telemetry | The Runtime reported tool execution through Ingest |
| 7 | Cost attribution | Model usage is reported through LiteLLM, when configured |
These layers can succeed or fail independently. For example:
- The Agent Runtime can be running while one Connection is disconnected.
- A provider message can be accepted and persisted even if Runtime processing later fails.
- A reply can fail provider delivery after the Runtime generated it.
- An Agent can respond normally even when a request does not produce any Tool Calls.
- A headless Agent with no Connections can still have a healthy Runtime.
- A healthy Product API does not prove that Ingest, Communications, a provider Connection, or an Agent Runtime is healthy.
Before you begin
You need:
- An active Organization
- Access to the Agent
- A hired Agent with a selected Runtime and pinned Template Version
- A successfully started Agent, unless you are diagnosing startup
- At least one Communication Connection, when provider messaging is being tested
- A controlled provider location where allowed and blocked messages can be sent
- Permission to read Agent configuration, Activity, health, and logs
- Permission to inspect Connection health
- A non-sensitive test prompt
You do not need a Platform to verify that the Runtime itself starts successfully. Steps 1 through 3 apply to a headless Agent.
The Agent Creator receives explicit Agent Owner access and can perform every check in this guide. Other users see fewer controls, depending on their effective Agent Access.
Required permissions
| Verification task | Required authority |
|---|---|
| Open the Agent and review its configuration | Agent read access |
| View health, Conversations, Tool Calls, or logs | activity.read |
| Start or pause the Agent | Agent lifecycle permission |
| Review or change a Communication Connection | Agent update permission |
| Replace Connection or Integration credentials | Agent secret-management permission |
| Review or change sharing | Agent access-management permission |
| View Organization-wide costs | Organization Owner or Organization Administrator |
Verification plan
Work through the layers in order. Each check has its own pass condition.
| Check | Requirement | Pass condition |
|---|---|---|
| Agent configuration | Required | Runtime, pinned Template Version, Skill Versions, model, and Agent Access match your intent |
| Lifecycle state | Required | The Agent reaches the running state without an unresolved error |
| Runtime health and logs | Required | Runtime health is reachable and startup logs show no unexplained failure |
| Connection configuration and health | Conditional | The intended Connection is enabled and shows no unresolved provider error |
| Allowed interaction | Conditional | An allowed message receives a reply through the same Connection |
| Blocked interaction | Conditional | A deliberately blocked message receives no reply |
| Conversation persistence | Conditional | Inbound and outbound messages appear under the expected Connection and location |
| Tool Call telemetry | Conditional | A controlled tool request produces the expected Tool Call |
| Agent Access | Required | General access and direct assignments match the intended audience |
| Cost attribution | Optional | Model usage and cost appear under the correct Agent |
The conditional checks apply once the Agent has at least one Communication Connection. If a required check fails, investigate it before adding more Skills, credentials, Connections, or users.
Verify the Agent configuration
Review the configuration before testing anything operational. Open the Agent, then confirm:
- The intended Organization owns the Agent
- The intended Hermes or OpenClaw Runtime is selected
- The intended Template and exact Template Version are pinned
- Required Skill Versions are assigned
- Required tool Integration credentials are configured
- Agent Access is appropriate
- Agent General Access remains Restricted, unless broader Organization access was intentionally configured
- The model follows the Organization default or pins the intended explicit model
Make sure you understand the effective model: the model the Agent resolves to right now. If the Agent is already running, the displayed running model may differ from the currently effective model until the Agent restarts.
If anything here is wrong, correct it in Configuration before continuing. See Hire your first Agent for how these choices were made.
Verify the lifecycle state
Inspect the Agent's lifecycle state on its own, before considering communication. Agent Barn persists three states.
| State | Meaning |
|---|---|
STOPPED | No active Runtime is expected |
RUNNING | Runtime resources were created successfully |
ERROR | The latest Runtime lifecycle operation failed |
The interface presents these as operator-facing labels such as Idle, Initializing, Working, Disconnected, and Needs attention. The underlying meanings above do not change.
How an Agent arrives at a state:
- Agent creation first persists a headless
STOPPEDAgent. - The web hiring flow may issue a separate start operation immediately afterward.
- Starting renders the pinned configuration and creates Runtime Kubernetes resources.
If the Agent is stopped and you hold lifecycle permission, select Start. See Manage the Agent lifecycle.
Verify Runtime health and logs
This check applies whether or not the Agent has any Connections.
- Confirm the Agent is in the active running state.
- Open the Agent health surface.
- Confirm the Runtime health endpoint is reachable.
- Review recent Runtime logs for startup or execution errors.
- Confirm the configured Runtime image was pulled.
- Confirm Kubernetes created the expected Deployment, Service, Secret, ConfigMap, and PVC, where those details are exposed.
- Confirm the effective configuration rendered without missing Template, Skill, model, or Integration requirements.
Expected: Runtime health is reachable and the startup sequence completes without an unexplained failure.
Runtime logs primarily diagnose:
- Template rendering
- Skill materialization
- Integration configuration
- Model access
- Kubernetes startup
- Runtime execution
- Local Runtime request processing
Runtime logs are not the canonical source for provider Connection health or Communication Delivery history. Use the Connection surface for those. See Review Agent health and logs.
Optional Kubernetes check
Self-hosted operators can confirm that Agent resources exist in the namespace:
kubectl get pods \
--namespace agent-farm \
--selector agentbarn.io/component=agentFor staging, use the configured staging namespace, commonly agent-farm-staging. The relevant Agent pod should be running and ready.
Verify the Communication Connection
When provider messaging is being tested, inspect the specific Connection rather than the Agent as a whole. Confirm:
- The intended Connection belongs to this Agent
- The intended Platform Plugin is selected
- The Connection is enabled
- Provider credentials were accepted
- The provider application or bot is installed in the intended location
- The Connection's routing and admission settings allow the intended location and user
- The Connection does not show an unresolved provider health error
- The provider-side scopes, permissions, events, intents, channel configuration, or webhook are complete
Connection health is independent of Agent lifecycle. A running Agent can hold one healthy Connection and one failing Connection at the same time, and each is diagnosed on its own.
For provider-side completeness, use the guide for your Platform: Slack, Microsoft Teams, Telegram, or Discord.
Verify one allowed interaction
Use a controlled provider location.
- Confirm the Agent Runtime is running.
- Confirm the Connection is enabled.
- Send a message from an allowed user in an allowed location.
- Include an explicit mention when required by the Connection policy.
- Wait for the Agent's response.
- Confirm the reply arrives through the same provider application and Connection.
Use a deterministic prompt:
@agent-bot Respond with exactly: verification-okReplace @agent-bot with the provider-side handle of the application attached to this Connection.
Expected: the Agent returns verification-ok in the same location, through the same Connection.
Admission behavior differs by Platform and by Connection policy:
- Shared spaces may require an explicit mention.
- Direct messages may be off, open, or allowlisted.
- Slack thread behavior depends on the Connection's thread mention policy.
- Discord server messages may be narrowed by server, channel, user, role, and mention settings.
- Microsoft Teams and Telegram retain their provider-specific admission rules.
Record the Connection, the location, the approximate send time, and the expected response. Minor formatting differences do not indicate a delivery failure; what matters is that the correct Agent produced one relevant response on the expected Connection.
Verify one blocked interaction
A reachable Agent must also ignore what its policy excludes. Test one deliberately blocked case that applies to this Connection, such as:
- A location outside the allowlist
- A user outside the user or role policy
- A direct message when direct messages are off
- An unmentioned shared-space message when mention gating applies
For the mention case, send a new message in the shared location without mentioning the Agent:
verification-no-mentionExpected: the Agent does not respond.
What should happen behind that silence:
- A policy-rejected provider payload does not create a canonical inbound Conversation Message.
- Operational diagnostics may record a content-free policy disposition.
Verify the Conversation under the correct Connection
Return to the Agent and check that the exchange was recorded where you expect it.
- Open Agent Activity.
- Open Conversations.
- Select the expected Communication Connection and provider location.
- Confirm the allowed inbound message appears.
- Confirm the outbound Agent response appears in the same conversation.
- Confirm sender and location names, where provider enrichment is available.
- Confirm the blocked test did not become a canonical Conversation Message.
How Conversation identity works:
- The Communications Gateway writes canonical Conversation Messages.
- Conversation identity includes both
connection_idand providerchannel_id. - Two Connections can safely use the same provider channel identifier.
- Replies remain bound to the source Connection.
- Conversation persistence is not written through Ingest.
If the reply arrived in the provider but no Conversation appears, treat it as a Communications or permission question, not a Runtime telemetry question. See Review Agent activity.
Verify Tool Calls separately
Tool Calls follow a different path from Conversations.
- Hermes or OpenClaw reports Tool Call telemetry through Ingest.
- Ingest authenticates with Agent identity and a per-start Ingest key.
- Tool Calls use Runtime invocation identity for correlation.
- A request that invokes no tool is not expected to create a Tool Call.
- A successful chat reply does not guarantee that a Tool Call should exist.
So No tool calls yet can be the correct result for a first verification prompt. To verify the path itself, send a controlled request that is expected to invoke one configured tool, then open Tool calls.
Prefer a read-only request. Avoid creating, updating, deleting, sending, or publishing external data during initial verification: for example, ask a repository-enabled Agent to list an allowed repository rather than to modify it.
| Status | Meaning | Verification action |
|---|---|---|
PENDING | The Runtime reported that the call began | Wait for its result |
SUCCESS | The result completed successfully | Confirm the expected tool and scope were used |
ERROR | The call failed | Expand the row and inspect its arguments and result |
Confirm that:
- The Tool Call appears under the correct Agent
- The tool name is expected
- The status reaches the expected terminal state
- The result is correlated to the intended invocation
- No unexpected write operation occurred
Review a failure as an Integration or Runtime problem first, rather than automatically as a Connection problem. Treat Tool Call arguments and results as potentially sensitive operational information.
Confirm Agent Access
Two access surfaces exist, and verifying one says nothing about the other:
- A Communication Connection's admission policy determines who can talk to the Agent through a provider.
- Agent Access determines who can open and operate the Agent inside Agent Barn.
If you have access-management permission, open Share on the Agent page and confirm that General access matches the intended policy. New Agents default to:
RestrictedWith Restricted access, only users with direct Agent Access can open the Agent. Organization Owners and Organization Administrators retain implicit full authority over every Agent, and they are not listed as direct assignments.
| Role | Intended authority |
|---|---|
| Agent Viewer | Read Agent information, Conversations, Tool Calls, logs, and Agent-specific costs |
| Agent Editor | Viewer authority, plus configuration, lifecycle, Skills, credentials, and Connections |
| Agent Owner | Editor authority, plus deletion and Agent access management |
The Agent Creator should appear with explicit Agent Owner access. For an observer helping with verification, Agent Viewer is normally sufficient.
Review cost attribution
Optional This check applies only when LiteLLM cost reporting is configured.
Organization Owners and Organization Administrators can open Costs from the Organization navigation. Under the Agent breakdown, locate the verified Agent and confirm the row attributes the correct Agent, model, tokens, and total cost.
Cost reporting follows a separate path:
Agent model request
↓
LiteLLM
↓
LiteLLM key identity
↓
Agent Barn cost attribution- Costs come from LiteLLM reporting.
- Costs are attributed through the Agent's LiteLLM key identity.
- Conversation Messages and Tool Calls do not calculate cost.
- A Conversation can exist before cost reporting appears.
- Tool Call state does not prove that LiteLLM cost ingestion is healthy.
See Review costs for the full reporting model.
Where to look when a layer fails
Match the symptom to its layer before changing anything.
| Symptom | Most likely layer | First place to inspect |
|---|---|---|
| Agent cannot start | Runtime lifecycle | Agent state and Runtime logs |
| Agent runs but Connection is disconnected | Communications provider session | Connection health and setup |
| Allowed provider message never appears | Provider ingress or admission | Connection policy and diagnostics |
| Inbound Conversation appears but no Runtime response | Runtime processing | Agent health and Runtime logs |
| Runtime response exists but provider reply is missing | Outbound Communication Delivery | Connection delivery diagnostics |
| Conversation works but Tool Calls are empty | Request did not invoke a tool, or Ingest issue | Tool Call view and Ingest path |
| Tool Call fails | Runtime tool or Integration | Tool Call result and Integration configuration |
| Activity works but costs are missing | LiteLLM reporting | Cost surface and LiteLLM configuration |
| Another Member cannot inspect the Agent | Agent Access | Effective Agent Access and Permissions |
Connection diagnosis and Runtime diagnosis stay separate. A provider problem is not fixed by restarting the Agent, and a Runtime problem is not fixed by re-entering provider credentials.
Acceptance checklist
The Agent is ready for further controlled use when you can confirm each of these.
Runtime and configuration
- The Agent has the intended Runtime and pinned configuration
- The Agent Runtime starts successfully
- Runtime health and logs show no unexplained failure
- A headless Agent is understood as valid
- Agent Access allows the intended Members to inspect the result
Communication
- The intended Communication Connection exists and is enabled
- Connection health is checked separately from Agent health
- One allowed provider interaction succeeds
- One blocked provider interaction receives no response
- The allowed Conversation appears under the correct Connection
Telemetry and cost
- A controlled tool request produces the expected Tool Call, when applicable
- Cost attribution is checked separately, when applicable
Troubleshooting
The Agent will not start
Runtime lifecycle, not communication
Review the lifecycle state and Runtime logs, then check:
- Model availability
- Runtime image access
- Kubernetes capacity, scheduling, and volumes
- Template rendering
- Skill materialization
- Integration credentials
For a self-hosted installation:
kubectl get pods \
--namespace agent-farm \
--selector agentbarn.io/component=agentCorrect the cause, then start the Agent again. A successful start clears the previous error.
The Agent runs but a Connection is disconnected
Provider session, not Runtime
Review Connection health and the Platform Plugin's setup guidance, then confirm:
- The Connection is enabled
- Provider credentials are still valid
- The provider application or bot is enabled on the provider side
- The relevant supervised session or webhook path is available
Do not restart the Agent unless there is a separate Runtime problem.
An allowed message receives no response
Separate admission from Runtime processing
First confirm the message should have been admitted:
- The provider application is installed in that location
- The location, user, and role satisfy the Connection policy
- The direct-message policy allows the test, if you used a DM
- The message satisfies the Connection's mention policy
Then check whether the inbound Conversation Message was persisted. If it was, the admission layer worked and the question moves to Runtime processing: review Agent health and Runtime logs around the send time. If it was not, stay in the Connection layer.
The Runtime replied but the provider shows nothing
Outbound Communication Delivery
Confirm the Runtime is running, then review Connection delivery diagnostics and delivery state. Confirm outbound provider permissions and that the source Connection is still enabled.
This is a delivery problem, not a reason to change the Agent's Template, model, or Skills.
The Agent responds to a message it should have ignored
Connection admission policy
Confirm the test was constructed correctly: a new message in a shared location rather than a direct reply, sent by the intended user, in the intended location.
Then review the Connection's routing and admission settings, including allowlists, user and role policy, direct-message policy, and mention policy. Saving the Connection reconciles the provider session; no Agent restart is required.
The Agent replied, but Conversations look empty
Check permission and the selected Connection
Reload the Agent page after the response completes, then confirm:
- Your account holds
activity.read - You selected the Connection and provider location used for the test
- The exchange belongs to this Agent
Conversation history is Connection-scoped, so an exchange on one Connection does not appear under another.
Tool Calls are empty or a Tool Call fails
Runtime telemetry and Integrations
An empty view is expected when the request invoked no tool. Send a request that should invoke one configured tool before treating this as a fault.
For a failing or stuck Tool Call, expand it and review the tool name, arguments, result, associated logs, assigned Skill Version, Integration credential, provider-side permissions, and allowed resource scope. Repeat a test only after confirming it cannot create a duplicate external action.
The Agent does not appear under Costs
LiteLLM reporting is separate
Confirm that:
- You are an Organization Owner or Organization Administrator
- LiteLLM is configured
- The Agent has a per-Agent LiteLLM key
- The Agent completed a model request
- The selected date range contains the request
Successful Conversations and Tool Calls do not guarantee that LiteLLM cost reporting is configured.
Another Member cannot inspect the Agent
Agent Access
Agent General Access defaults to Restricted, so Members do not gain access automatically. Review their effective Agent Access and Permissions, then assign explicit Agent Access or deliberately change General Access.
Next steps
After verification:
- Add or review Communication Connections for the Platforms this Agent should serve
- Connect another Platform with Connect a Platform
- Replace temporary policies with production channel and group allowlists
- Monitor Agent health and logs separately from Connection health
- Review Agent activity and costs as distinct operational surfaces
- Grant the minimum necessary Agent Access Roles
- Add and verify one Skill at a time
- Pause the Agent until its production scope has been approved
Chat in the dashboard
For an initial conversation without external provider setup, use the experimental Chat tab once the Agent is Running and Working. See Chat with an Agent in the dashboard for permissions and thread behavior. External Connections can be configured separately.