Imagine creating virtual characters with seamless interaction and lifelike responses, easily tailored through powerful APIs. Convai's platform makes this possible, allowing developers and creators to build rich, dynamic characters that can interact naturally with users in virtual worlds.
Convai is a platform designed for developers and creators. It offers features that simplify creating and customizing virtual characters. With Convai, you can design characters with advanced multimodal perception abilities that respond to text and voice commands, making them suitable for virtual and real-world environments.
This article explores the benefits and functionalities of Convai's latest Core API release. The release provides developers and creators with tools to design, customize, and integrate dynamic AI-powered characters using the Character Tool API and the Standalone Voice API. You’ll see the APIs' features and offer a step-by-step guide on how to use them effectively within projects.
By the end of this article, you'll understand:
- The Character Tool API and Standalone Voice API.
- Insight into how these APIs can enable you to create more interactive and immersive virtual characters.
- Hands-on experience through practical implementation examples and tutorials through a Google Colab notebook.
Let’s get right into it! 🚀
By the way, here is the complete Colab Notebook that guides you through this tutorial step-by-step, in case you want to jump right into the code: Core APIs Notebook.ipynb
Overview of Convai's Core APIs
Core APIs empower developers to create dynamic, responsive characters in virtual worlds. The APIs are a link between your creative ideas and the technical implementation.
Convai offers two primary API categories:
- Character Tool API: This suite of APIs allows developers to modify the narrative, behavior, and knowledge of virtual characters in real-time. Developers can programmatically adjust the character’s backstories, chat histories, and actions to fit the narrative flow or virtual world requirements.
- Standalone Voice API: These APIs provide high-quality speech-to-text and text-to-speech capabilities to enable characters to communicate with users using natural language. Developers can also access a variety of voices and languages to cater to global audiences.
These APIs integrate into the Convai platform, which developers can use with popular engines like Unity and Unreal Engine or directly within web UIs. They provide the means to create immersive and responsive virtual characters that can adapt to user actions and deliver personalized experiences.
This level of sophistication in character behavior can increase user engagement, storytelling depth, and overall experience quality.
Features and Benefits of the Core APIs
Here are the major features you get with Convai’s Core APIs:
- Customization: Convai’s Core APIs allow deep customization of character attributes such as backstory, personality, and interactions. Developers can change how virtual characters behave, interact, and react to user actions in real-time. This level of customization is crucial for creating lifelike characters relevant to the story’s narrative.
- Multimodal Interaction: Convai’s Core APIs enable characters to engage users through multiple modes of communication, including voice, text, and actions. For example, a character can respond to a user’s voice command, display on-screen text, and then trigger an action based on that command.
- Easy Integration: Convai’s APIs allow developers to integrate them seamlessly across various platforms, such as Web UI, Unity, and Unreal Engine. This makes it possible to incorporate Convai’s AI features into any virtual world.
- Real-time Interaction Capabilities: With Convai’s Chat History API and Action API, developers can create characters that track user interactions and respond dynamically. This real-time interaction capability allows characters to engage with users fluidly and naturally. They can respond to past dialogues and in-world events with appropriate context.
- High-quality Voice Synthesis and Speech Recognition: The Standalone Voice API delivers high-quality voice synthesis and speech recognition that enable characters to understand spoken commands and respond naturally. This capability is crucial for developers integrating voice-based interactions into virtual worlds to create a more immersive and intuitive user experience.
Great! Now, let’s spend some time learning these APIs and showing you how to call them for your characters.
📝 NOTE: In this article, we will use the Character Tool API only because the Standalone Voice API is currently being maintained.
Character Tool API Deep Dive
Convai’s Character Tool API gives developers full programmatic control over their virtual characters. This API offers the flexibility to shape narratives, manage character backstories, fine-tune how characters respond to users, perform real-time actions, and define runtime behavior. It simplifies building interactive and engaging characters that adapt to user input.
Below, we’ll learn more about the various API endpoints for controlling characters.
1. Narrative Design API
The Narrative Design API allows developers to control and evolve the character's narrative in response to user actions. By defining specific narrative keys, developers can trigger distinct character responses based on the user's actions, which enhances the storytelling mechanics.
Here is a sample code for triggering specific responses based on user action (e.g., the user entered the scene):
1import requests
2import json
3
4url = "https://api.convai.com/character/narrative/create-trigger"
5headers = {
6 'CONVAI-API-KEY': '',
7 'Content-Type': 'application/json'
8}
9
10# Create a dictionary for the JSON payload
11payload = {
12 "character_id":"",
13 "trigger_message":"User has entered the museum.",
14 "trigger_name":"StartTrigger",
15 "destination_section": ""
16}
17
18# Convert the payload to JSON
19json_payload = json.dumps(payload)
20
21response = requests.post(url, headers=headers, data=json_payload)
22
23print(response.text)
24
Here’s what the Narrative Design section looks like in the Convai Playground:
2. Backstory API
The Backstory API helps developers build rich character histories, defining relationships, motivations, and traits that influence characters' behavior. This feature allows them to react in ways consistent with their backstory and established personalities.
Here is a sample code that sends an initial description of the character “Mindy” and its name to the API endpoint to generate a backstory:
1
2import json
3from sseclient import SSEClient
4import requests
5
6url = "https://api.convai.com/character/generate-backstory"
7
8headers = {
9 'CONVAI-API-KEY': '<Your-API-Key>',
10 'Accept': 'text/event-stream'
11}
12
13form_data = {
14 "inputText": "Lawyer in the New York City. Achiever. Stubborn.",
15 "charName": "Mindy"
16}
17
18# Create a session to manage cookies and keep-alive
19session = requests.Session()
20
21# Send the POST request with form-data and stream the response
22response = session.post(url, headers=headers, data=form_data, stream=True)
23
24# Create an SSE client from the response
25client = SSEClient(response)
26
27# Process the events
28full_response = ""
29for event in client.events():
30 if event.data:
31 full_response += event.data
32
33print(full_response)
Here’s what a Character Description section and a generated backstory look like in the Convai Playground:
3. Chat History API
The Chat History API provides access to a character's previous interactions. Developers can analyze, retrieve, and modify the chat history to improve the character's context-aware responses to user inputs. This is useful for building virtual characters with memory, enabling them to refer to previous conversations.
Here is a sample code to list a character's chat history:
1import requests
2import json
3
4url = "https://api.convai.com/character/chatHistory/details"
5
6headers = {
7 'CONVAI-API-KEY': '<Your-API-Key>',
8 'Content-Type': 'application/json'
9}
10
11# Create a dictionary for the JSON payload
12payload = {
13 "charID": "<Your-Character-Id>",
14 "sessionID": "<Your-Session-ID>"
15}
16
17# Convert the payload to JSON
18json_payload = json.dumps(payload)
19
20response = requests.post(url, headers=headers, data=json_payload)
21
22print(response.text)
4. Interaction API
With the Interaction API, developers can implement a chatbot session for end-users to converse with their characters. The users can maintain the context of a conversation by maintaining the session ID in the API requests made.
Here’s a sample code to request an audio response from the character during a conversation:
1import requests
2import json
3import base64
4
5url = "https://api.convai.com/character/getResponse"
6
7payload={
8 'charID': '<your character id>',
9 'sessionID': '-1',
10 'responseLevel': '5',
11 'voiceResponse': 'True'}
12files=[
13 ('file',('audio.wav',open('<path to the audio file audio.wav>','rb'),'audio/wav'))
14]
15headers = {
16 'CONVAI-API-KEY': '<your api key>'
17}
18
19response = requests.request("POST", url, headers=headers, data=payload, files=files)
20data = response.json()
21
22character_response = data["text"]
23
24decode_string = base64.b64decode(data["audio"])
25
26with open('audioResponse.wav','wb') as f:
27 f.write(decode_string)
5. Core AI Setting API
Through the Core AI Setting API, developers have access to all the relevant APIs needed to modify the Core AI setting of their Convai Character, such as changing the underlying models and adjusting parameters like the temperature settings for character humor, creativity, or formality.
Here is a sample code to update the temperature setting for a character’s AI model:
1<import requests
2import json
3
4url = "https://api.convai.com/character/update"
5
6headers = {
7 'CONVAI-API-KEY': '<Your-API-Key>',
8 'Content-Type': 'application/json'
9}
10
11# Create a dictionary for the JSON payload
12payload = {
13 "charID": "<Your-Character-Id>",
14 "temperature": 0.42
15}
16
17# Convert the payload to JSON
18json_payload = json.dumps(payload)
19
20response = requests.post(url, headers=headers, data=json_payload)
21
22print(response.text)
6. Character API
The Character API lets developers access Convai endpoints for developing and interacting with an intelligent character, starting with essential information like the character's name, background information, and voice selection.
This API offers control over character data, enabling developers to create and update character attributes.
1import requests
2import json
3
4url = "https://api.convai.com/character/update"
5
6payload = json.dumps({
7 "charID": "<character ID>",
8 "backstory": "Raymond Reddington is a highly intelligent, highly driven individual with developed sociopathic tendencies. This appears to be the product of PTSD (post-traumatic stress disorder) as there are no signs that he was born this way. Sly, manipulative, and charming, Red is always three steps ahead of everyone else, and is determined to keep himself a mystery. As he puts it, “I’m a criminal. Criminals are notorious liars. Everything about me is a lie.” That’s probably true, actually, but who knows for sure. He dislikes rude people, which is something that Agent Ressler pointed out after Red let a notorious drug dealer get away with false identification. Ressler mentioned that Red wouldn’t let the drug dealer get away because he was rude and Red doesn’t like rude people. Red responded with, “He is on my jet.”",
9 "voiceType": "US MALE 1",
10 "charName": "Raymond Reddington"
11})
12headers = {
13 'CONVAI-API-KEY': '<your api key>',
14 'Content-Type': 'application/json'
15}
16
17response = requests.request("POST", url, headers=headers, data=payload)
18
19print(response.text)
7. Knowledge Bank API
The Knowledge Bank API allows developers to manage a character's knowledge base, which consists of files containing specific facts or domain expertise that the character can reference during conversations.
Upon calling the API, the file will only be uploaded for processing and will not be available until the processing is complete. The upload API will return a unique ID assigned to the uploaded file, which must be used for all future interactions with the file.
Here is a sample code to upload a knowledge bank file:
1import requests
2import json
3
4url = "https://api.convai.com/character/knowledge-bank/upload"
5
6
7headers = {
8 'CONVAI-API-KEY': '<Your-API-Key>',
9}
10
11# Path to the file you want to upload
12file_path = "photosynthesis.txt"
13
14# Open the file in binary mode
15with open(file_path, "rb") as file:
16 # Create a dictionary for the form data
17 form_data = {
18 "file_name": file.name,
19 "file": file
20 }
21
22 # Send the POST request with multipart/form-data
23 response = requests.post(url, headers=headers, files=form_data)
24
25print(response.text)
8. Action API
The Action API enables developers to define the character's various actions, often tied to the available objects in the environment. These actions can include gestures, movements, or more complex behaviors based on in-world events or user commands.
Language List API
The Language List API helps developers choose from supported languages for speech synthesis, enabling multilingual characters to interact with global audiences.
Practical Guide: Using Convai’s Core APIs
Now that we've explored the capabilities of Convai's Core APIs, let's go into a hands-on tutorial to demonstrate their practical implementation.
Here is the complete Colab Notebook that walks you through this tutorial step-by-step: Core APIs Notebook.ipynb
We'll guide you through setting up the environment, initializing APIs, and creating a simple character that responds to user input and dynamically adapts its backstory.
Step-by-Step Tutorial on Building an Interactive Character with Convai APIs
In this tutorial, you will create a simple character that:
- Responds to user input in real-time.
- Adapts its backstory dynamically based on interactions.
- Engages with users using both voice and text.
Environment Setup:
Before diving into the code, you must obtain your personal API key. You'll need an API key to interact with the Convai platform. Sign up at Convai and access your API key from the dashboard.
Here’s a GIF showing how to access your API key:
Here is the complete Colab Notebook that walks you through this tutorial step-by-step: Core APIs Notebook.ipynb
Conclusion
Convai’s Core APIs provide developers with tools to create immersive and interactive virtual characters. Convai lets developers create realistic characters that improve the user experience by using dynamic narrative design, adaptive backstories, real-time interaction, and advanced voice capabilities.
Whether you’re creating characters for a web-based UI or integrating them into Unity or Unreal Engine projects, Convai’s APIs offer the flexibility and scalability needed for developing modern virtual worlds.
Following the step-by-step guide in this article and experimenting with the provided Colab notebook, you’ve gained hands-on experience with the Character Tool API and Standalone Voice API.
With Convai, you can seamlessly design and modify characters programmatically, enabling them to respond, adapt, and engage with users in ways that enrich storytelling and the virtual world.
Ready to Start Building?
- Create Your Free Convai Account Here to explore the Playground.
- Access the complete API documentation. Unlock the potential of Convai’s powerful AI tools and start building intelligent, more engaging characters for your virtual worlds.
