Audio Events | Machani Robotics SDK

curl https://api.machanirobotics.com/v1/conversation/audio/events:audio \
     -H "Content-Type: application/json"

{
  "audio": {
    "service": 2,
    "languageTag": {
      "countryTag": 840,
      "languageTag": 1033,
      "inputAudio": "raw-audio-stream",
      "inputText": "Hello, how can I assist you today?",
      "inputSpeech": "hello_how_can_i_assist_you_today",
      "inputNone": false,
      "outputAudio": "processed-audio-stream",
      "outputText": "I am here to help you with your request.",
      "outputSpeech": "i_am_here_to_help_you_with_your_request",
      "outputNone": false
    },
    "name": "audio-session-20240612-001",
    "audioData": "UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAIA+AAACABAAZGF0YQAAAAA=",
    "uploadPath": "s3://machani-audio-uploads/session-20240612-001.wav",
    "livePipelineFilePath": "/tmp/transcribe.wav",
    "vadThreshold": 0.35,
    "sampleRate": 16000,
    "wordwiseTimestamps": [
      {
        "word": "Hello",
        "start": "0.00s",
        "end": "0.45s",
        "probability": 0.98
      },
      {
        "word": "how",
        "start": "0.46s",
        "end": "0.70s",
        "probability": 0.95
      },
      {
        "word": "can",
        "start": "0.71s",
        "end": "0.85s",
        "probability": 0.96
      },
      {
        "word": "I",
        "start": "0.86s",
        "end": "0.95s",
        "probability": 0.97
      },
      {
        "word": "assist",
        "start": "0.96s",
        "end": "1.20s",
        "probability": 0.94
      },
      {
        "word": "you",
        "start": "1.21s",
        "end": "1.35s",
        "probability": 0.96
      },
      {
        "word": "today",
        "start": "1.36s",
        "end": "1.60s",
        "probability": 0.93
      }
    ]
  },
  "state": 3,
  "message": {
    "id": "msg-20240612-001",
    "role": 2,
    "text": {
      "content": [
        "I have processed your audio input and am ready to assist."
      ]
    },
    "image": {
      "id": "img-20240612-001",
      "uris": [
        "https://cdn.machanirobotics.com/images/session-20240612-001/processed.png"
      ],
      "b64Json": [
        "iVBORw0KGgoAAAANSUhEUgAAAAUA"
      ],
      "savedPaths": [
        "/var/data/images/session-20240612-001/processed.png"
      ]
    },
    "vision": {
      "id": "vision-20240612-001",
      "description": "Detected a person standing near a chair in the room.",
      "objects": [
        {
          "object": "person",
          "confidence": 0.92,
          "boundingBox": {
            "x": 120,
            "y": 80,
            "width": 60,
            "height": 150
          }
        },
        {
          "object": "chair",
          "confidence": 0.88,
          "boundingBox": {
            "x": 200,
            "y": 180,
            "width": 80,
            "height": 90
          }
        }
      ]
    },
    "options": {
      "service": 2,
      "model": "whisper-large-v2",
      "languageTag": {
        "countryTag": 840,
        "languageTag": 1033,
        "inputAudio": "raw-audio-stream",
        "inputText": "I have processed your audio input.",
        "inputSpeech": "i_have_processed_your_audio_input",
        "inputNone": false,
        "outputAudio": "response-audio-stream",
        "outputText": "How can I assist you further?",
        "outputSpeech": "how_can_i_assist_you_further",
        "outputNone": false
      }
    }
  }
}

AudioEvents retrieves the current events of the audio processing session. This RPC returns an AudioEvents object containing the current events of the microphone and any associated audio data.

HTTP Mapping: GET “/v1/conversation/audio:audioEvents” The request body is empty.

Query parameters

namestringOptional

Unique identifier for the audio resource. This field uniquely distinguishes an audio instance within the system.

Response

audioobject

The current state of the audio processing session. Indicates relevant status information of the ongoing or completed audio processing.

stateintegerRead-only

The current state of the audio processing session.

messageobject

InferenceOutput is the transcribed audio message to text to pass it LLM.

Errors

400

Bad Request Error