Skip to main content

Preview Testing

Every Chippy app has a preview MCP endpoint you can connect to ChatGPT for testing. This lets you test the full conversation experience (tool selection, parameter passing, and widget rendering) before real users interact with your app.

Testing via ChatGPT

The primary way to test your app is by connecting ChatGPT to your preview MCP endpoint and having real conversations.
1

Copy your preview MCP URL

Your preview URL is:
You can find this in the builder.
2

Add it to ChatGPT

In ChatGPT, go to Settings > Connected Apps and add the preview URL as an MCP server. See Connecting to ChatGPT for detailed instructions.
3

Run through your test scenarios

Open a new ChatGPT conversation and try prompts that should trigger your tools.Example test prompts:
Show me all high-urgency tickets from today.
Assign ticket #1042 to the billing team.
What’s the status of the ticket I just reassigned?
4

Check the results

Verify that ChatGPT picks the right tool, passes the right parameters, and displays the widget correctly.

What to Test

Happy Path

Run through your app’s core use case from start to finish. If the main flow doesn’t work, nothing else matters. Example: For a support triage app, test: view dashboard, filter by urgency, assign a ticket, verify the status updates.

Edge Cases

Push the boundaries of your inputs to see how the app handles unusual data.

Multi-Turn Conversations

Test that your app works across multiple messages in the same conversation.
  • Does ChatGPT remember context from earlier tool calls?
  • Can a user perform sequential actions (search, then update, then verify)?
  • Does the app handle a second invocation of the same tool correctly?

Connecting Traces to Your Tests

After testing in ChatGPT, open the Traces dashboard to verify what happened behind the scenes.
1

Open Traces

Go to Traces in your Chippy dashboard. Filter by your app and the time window of your test session.
Traces dashboard showing recent tool invocations

Traces dashboard filtered to recent test session

2

Verify tool behavior

For each tool invocation, check:
  • Was the correct tool selected?
  • Were the input parameters what you expected?
  • Did the output match the expected result?
  • How long did the invocation take?
3

Save important patterns

If you find a trace that represents an important test case (especially edge cases or failure modes), click Add to Golden Set to save it as an evaluation case so you can rerun it later.
Traces require Solo plan or higher. See Traces for the full guide.

Troubleshooting

What’s Next

Traces

Investigate tool behavior in detail after testing.

Evaluations

Save test cases and run them to catch regressions.