Voxli Voxli

GitHub Integration

Run Voxli tests directly from GitHub Actions. When you trigger a test run, Voxli dispatches your workflow with the test details—your workflow handles the conversation loop.

How It Works

1. Trigger: When you run tests with a GitHub agent, Voxli triggers your workflow via workflow_dispatch.

2. Run: Your workflow runs and iterates over each test result ID.

3. Loop: The script relays messages between Voxli and your agent until the conversation should end.

Prerequisites

Before setting up a GitHub agent:

  • Install the Voxli GitHub App via Settings → Integrations → Add Integration → GitHub
  • Create a workflow file in your repository’s .github/workflows/ directory
  • Add your API token as a repository secret named VOXLI_API_TOKEN

Workflow Setup

Your workflow needs the workflow_dispatch trigger so that Voxli can trigger it. Voxli will pass test_result_ids as a JSON array input to it.

name: Run Voxli Test


on:
  workflow_dispatch:
    inputs:
      test_result_ids:
        description: "JSON array of test result IDs"
        required: true


jobs:
  run-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4


      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"


      - name: Install dependencies
        run: pip install requests


      - name: Run test
        env:
          VOXLI_API_TOKEN: ${{ secrets.VOXLI_API_TOKEN }}
          VOXLI_TEST_RESULT_IDS: ${{ github.event.inputs.test_result_ids }}
        run: python run_test.py

Running the Conversation

The workflow runs a Python script that handles the conversation loop. This script receives the test_result_ids from the workflow input and runs each test:

"""
Run Voxli test conversations.


Environment variables:
- VOXLI_API_TOKEN: Your API key (required)
- VOXLI_TEST_RESULT_IDS: JSON array of test result IDs (required)
"""


import json
import os
import sys
import time
import requests




def get_agent_response(message: str) -> str:
    """
    Replace this with your actual agent integration.


    Receives a message from the Voxli tester and returns
    your chatbot's response.
    """
    return "The meaning of life is 42."




def poll_next_message(endpoint: str, headers: dict, timeout: int = 30) -> str | None:
    start_time = time.time()
    while True:
        response = requests.post(endpoint, headers=headers)
        response.raise_for_status()
        data = response.json()
        if data["ready"]:
            return None if data.get("end_chat") else data["message"]
        if time.time() - start_time > timeout:
            raise TimeoutError("Timed out waiting for message")
        time.sleep(1)




def run_test(test_result_id: str, api_key: str, base_url: str):
    headers = {"Authorization": f"Bearer {api_key}"}
    conversation_endpoint = f"{base_url}/test-results/{test_result_id}/conversation"
    generate_endpoint = f"{base_url}/test-results/{test_result_id}/next-message"


    # Get first message from Voxli
    tester_message = poll_next_message(generate_endpoint, headers)


    # Conversation loop
    for _ in range(20):  # Max turns safety limit
        agent_response = get_agent_response(tester_message)


        # Record agent response
        response = requests.post(
            conversation_endpoint, headers=headers, json={"type": "message", "content": agent_response}
        )
        response.raise_for_status()


        # Get next message from Voxli
        tester_message = poll_next_message(generate_endpoint, headers)
        if tester_message is None:
            print(f"Test {test_result_id} completed")
            break




def main():
    api_key = os.getenv("VOXLI_API_TOKEN")
    test_result_ids_raw = os.getenv("VOXLI_TEST_RESULT_IDS")
    base_url = os.getenv("VOXLI_API_URL", "https://api.voxli.io")


    if not api_key or not test_result_ids_raw:
        print("Error: VOXLI_API_TOKEN and VOXLI_TEST_RESULT_IDS are required")
        sys.exit(1)


    test_result_ids = json.loads(test_result_ids_raw)


    for test_result_id in test_result_ids:
        run_test(test_result_id, api_key, base_url)




if __name__ == "__main__":
    main()

Environment

Local