User Tools

Site Tools


functions

VoxAI Plugin API Reference

Last Updated: 08/22/2026 (v1.3.2)

This reference lists the methods and attributes available via the app object passed to your plugin's register(app) function.


Security & API Boundary (AppProxy)

VoxAI routes all plugin interactions through an AppProxy boundary to safeguard the broadcaster's environment. Plugins do not have direct access to internal configuration files, OpenAI/Gemini keys, or Twitch secrets (app.config and app.twitch_settings will raise an Access Denied error). Always use the officially exposed methods and event handlers documented below.

πŸŽ™οΈ Audio Engine & Lip Sync

Plugins can listen to the raw volume levels of the bot's voice to create avatars, visualizers, or reactive lighting.

How to use: Register an event handler for audio_level.

  • tts_start β†’ No arguments. Fired the exact millisecond audio begins playing (useful for syncing hardware actions).
  • tts_end β†’ No arguments. Fired when playback finishes.
  • audio_level β†’ (level)
    • Fired continuously (~50 times/sec) while audio plays.
    • level (float): The RMS amplitude (loudness). Silence is 0.0; Max is 1.0+.

🧠 Core AI & Interaction

app.ask_ai(prompt) Sends a text prompt to the currently active AI provider (GPT or Gemini) and gets a direct response.

  • prompt (string): The full question or instruction.
  • Returns: A string containing the response. (Blocking call - use threading).

app.speak_text(text) Generates audio from text and plays it immediately through the configured output device.

app.synthesize_tts(text) Generates audio bytes but does not play them. Use this to prepare audio for the audio_queue.

  • Returns: Audio bytes, or None.

app.add_chat(message) Writes a message directly into VoxAI’s main chat log window.

app.persona (string) Read-only attribute containing the name of the currently active AI persona.


πŸ’Ύ Memory & Context

app.last_stream_category (string) Retrieve the name of the game currently being played on Twitch (e.g., β€œMon Bazou”). Updated automatically via EventSub. Use this to make AI prompts game-aware.

app.add_to_persona_local_memory(username, message) Adds a line to the persona's β€œshort-term memory” file for the current session.

app.get_persona_local_memory_lines() Retrieves the recent lines from short-term memory as a list.


πŸ“‘ Twitch Interaction

app.send_twitch_chat(message) Sends a message to Twitch chat. (Thread-safe).

app.send_api_whisper(target_username, message) Sends a reliable whisper via the Twitch API.

app.create_clip() Triggers a clip of the stream and posts the URL to chat.

app.channel (string) The streamer’s lowercase username.

app.is_channel_live() β†’ bool app.get_broadcaster_id() β†’ string


⚑ Event Hooks

Register these with app.register_event_handler(event_name, function).

Stream Events

  • sub β†’ (user, tier, streak)
    • Fired for ALL subs and resubs. For initial subs, streak is β€œ1”. For resubs, streak is cumulative months (e.g. β€œ5”).
  • prime_sub β†’ (user) (Fired on Prime paid upgrades)
  • gift β†’ (gifter, recipient, tier, is_anon)
  • giftbomb β†’ (gifter, count, tier, is_anon)
  • communitypayforward β†’ (gifter, recipient)
  • raid β†’ (raider, viewers)
  • shoutout β†’ (shouter, viewers)
  • follow β†’ (user_name)
  • cheer β†’ (user, amount, message) (Fired when cheer meets streamer's alert threshold)
  • raw_cheer_bits β†’ (user, amount, message) (Fired for ALL bit cheers regardless of threshold)
  • powerup β†’ (user, type, cost) (Fired on Gigantify, Message Effects, and Custom Power Ups)
  • redemption β†’ (user, reward_title, user_input, raw_event) (Channel Point Rewards)

Hype Train Events

  • hype_train_start β†’ (level)
  • hype_train_update β†’ (level) (Fires when leveling up)
  • hype_train_end β†’ (level)
  • hype_train_data β†’ (data) (Dictionary with 'level', 'progress', and 'goal')

System / OBS Events

  • obs_connection_changed β†’ (is_connected) (Fired when OBS WebSocket connects or disconnects)

Commands & Chat

app.register_command(name, function) Registers a '!command'.

  • Handler signature: function(args, user)
  • Note: If you need chatter badges (Mod, Sub, VIP), use app.plugin_chat_hooks instead.

app.plugin_chat_hooks (list) Append a function to this list to read every single chat message.

  • Hook signature: function(user, message, tags=None)
  • tags (dict): The raw Twitch EventSub payload containing 'badges', 'chatter_user_id', and chat metadata.

🎬 OBS Control

app.obs_switch_scene(scene_name) app.obs_set_source_visibility(scene, source, visible) app.obs_set_filter_enabled(source, filter, enabled) app.obs_flip_source(scene, source, direction)


🎨 Overlays & UI

app.add_plugin_menu_command(label, command) Adds a clickable item to the β€œPlugins” top dropdown menu.

app.master Reference to the root window (required for creating tk.Toplevel or customtkinter windows).


πŸ“š Included Libraries

VoxAI imports these libraries globally. You can use them in your plugins without any additional installation. Because VoxAI is compiled into a single executable, dynamically loaded plugins have direct access to these pre-packaged tools:

  • sqlite3: Built-in lightweight SQL database engine. Perfect for storing historical plugin data, archives, or complex user stats.
  • matplotlib: For generating advanced data visualizations, graphs, and charts. (Note: matplotlib.pyplot and matplotlib.backends.backend_tkagg.FigureCanvasTkAgg are explicitly bundled, allowing you to embed interactive dark-mode graphs directly into CustomTkinter UIs).
  • CustomTkinter: For building modern, dark-mode UI windows and dashboards.
  • pydirectinput: Hardware-level keyboard and mouse emulation for DirectX/Unity games.
  • pygetwindow (`gw`): OS-level window management (Focus detection/Window sizing).
  • BeautifulSoup (`bs4`): For web scraping and parsing HTML.
  • Requests: For making HTTP calls to external APIs.
  • Numpy: For advanced math operations and array handling.
  • Pillow (`PIL`): For image manipulation and processing.
functions.txt Β· Last modified: by voxai_wiki