Imagine bringing AI characters to life within a sprawling urban simulation where every NPC behaves, responds, and interacts with you as if they had their own thoughts and emotions. With the power of Unreal Engine's City Sample Scene and Convai, this vision is becoming a reality for game developers, designers, and virtual world builders.
In this blog, we will break down how to integrate AI-powered characters into Unreal Engine's City Sample Scene, inspired by the iconic "Matrix Awakens" demo (Watch the launch video). This guide will be particularly useful for game developers, virtual world creators, and anyone interested in creating lifelike AI characters that inhabit 3D spaces. We’ll explore the integration process, its benefits, and how AI-driven characters can take your game or simulation to the next level.
You can check out the on how to set this project up within the city sample scene on the Convai YouTube Channel.
Why Use Unreal Engine’s City Sample Scene?
The City Sample Scene, developed by Unreal Engine, provides a massive, realistic urban environment. Originally created for the "Matrix Awakens" demo, this scene is now accessible to developers who want to build vast, bustling cities filled with Metahumans—Unreal’s high-fidelity, customizable 3D avatars. But what makes this scene truly exciting is the ability to populate it with AI-powered characters, adding a new layer of realism.
Key Features of the City Sample Scene:
- Highly detailed cityscapes: Buildings, streets, and vehicles, all meticulously modeled to offer realistic visuals.
- Metahumans integration: Leverage Unreal Engine's cutting-edge human character creation technology.
- Optimized for performance: Even with hundreds of AI-driven characters, the scene runs smoothly, thanks to Unreal Engine's powerful optimization tools.
- Flexible and scalable: Suitable for small indie projects or AAA games, the scene can be customized to fit various game designs and virtual experiences.
Using the City Sample Scene with AI characters opens the door to creating highly immersive simulations. From bustling metropolises with hundreds of citizens to futuristic dystopias filled with hostile agents, the possibilities are endless.
How to Add AI-Powered Characters Using Unreal Engine and Convai
In this section, we’ll walk through the steps of adding AI-powered characters into the Sample Scene using the Convai Plugin for Unreal Engine. Convai allows developers to imbue characters with spatial awareness, conversational abilities, and dynamic interactions, giving them memories, goals, and unique personalities. Think of it as bringing the digital world of "Free Guy" to life, where AI characters have agency and evolve over time.
Step 1: Download and Set Up the City Sample Scene
To begin, you need to download the City Sample Scene from Unreal Engine’s marketplace. This project is massive, and depending on your system and internet connection, it may take some time to download and set up. Make sure you are running Unreal Engine 5.3 or higher, as the City Sample Scene is optimized for the latest features of UE5.
Step 2: Install and Enable the Convai Plugin
Next, you'll want to install the Convai plugin from the Unreal Engine marketplace. Once installed, head to the "Plugins" section in Unreal Engine, search for "Convai," and enable it. After enabling the plugin, restart your project to ensure it’s fully integrated.
To learn more about setting up Convai in Unreal Engine, check out our detailed Unreal Engine documentation page.
Step 3: Add Metahuman Characters to Your Scene
With the City Sample Scene and Convai plugin set up, the next step is to add Metahumans to your scene. Metahumans are Unreal Engine’s incredibly lifelike, customizable 3D characters. To add a Metahuman, go to the Quixel Bridge within Unreal Engine, search for the character you want, and add them to your project.
Verify the Metahuman’s presence in your content browser and open the character for editing.
Step 4: Configure Player and AI Blueprints
Now comes the technical part—configuring the blueprints for player interaction and AI behavior. Blueprints in Unreal Engine are visual scripting tools that allow you to program behaviors and logic without needing to write code.
Player Blueprint Setup:
1. Access the Player Blueprint: Navigate to the gameplay character section and open the BP_CitySamplePlayer.
2. Add the Convai Chatbot Component: This component allows the player to converse with AI characters. Add this component to the player’s blueprint.
3. Copy Functions: Next, copy necessary functions from the Convai plugin to the player blueprint to handle interactions.
4. Refresh Nodes: After copying the functions, refresh all nodes and compile to ensure there are no errors.
Crowd Character Blueprint:
1. Add AI Behavior to NPCs: Now, add the Convai Chatbot component to the NPC characters. This gives NPCs the ability to converse and interact dynamically with the player.
2. Stop Movement During Conversations: Ensure that when an NPC starts conversing with the player, it stops moving, creating a more natural interaction.
3. Create Personalities: Use Convai to assign personalities, backstories, and goals to each character. This adds depth and realism to the NPCs, making them feel like living, breathing individuals.
Step 5: Add Body and Facial Animations
One of the key features that make AI characters lifelike is their ability to display convincing body and facial animations.
Body Animations:
In the character blueprint, you’ll want to integrate Convai’s body animation blueprints. These animations allow NPCs to move, turn, and look at the player when speaking, creating a more immersive experience.
1. Link the Animation Blueprints: Go to the animation graph, copy the necessary functions from the Convai plugin, and integrate them into the City Sample Scene character’s blueprint.
2. Customize the Animation Logic: Adjust the animation layers, such as turn animations and head rotations, to make the character’s movements more natural.
To learn more about setting up custom actions for Metahumans in Unreal Engine, check out our detailed blog.
Facial Animations:
To make conversations more engaging, add facial animations that reflect emotions and lip-syncing during dialogue.
1. Copy the Convai Facial Animation Blueprint: Go to the Convai metahuman facial animation blueprint and copy it into your project.
2. Link the Mesh: Ensure the facial animation blueprint is connected to the character’s facial mesh.
3. Test and Adjust: Run tests to ensure the facial animations align with the character’s speech.
Benefits of AI-Powered Characters in Virtual Worlds
Bringing AI characters to life using Unreal Engine and Convai offers numerous advantages for both developers and players.
1. Immersive Player Experiences
AI-powered characters can dynamically interact with players, responding to their actions, conversations, and even emotions. This creates a more engaging and immersive experience, as players feel like they are interacting with real individuals rather than pre-programmed NPCs.
2. Enhanced Storytelling Through Narrative Design
By using Convai’s narrative design tools, developers can craft unique, branching storylines where AI characters evolve based on the player’s decisions. This allows for personalized gameplay experiences, where no two players will have the same journey.
3. Realistic AI with Spatial Awareness
AI characters created with Convai have spatial awareness, meaning they can react to the environment and navigate through the city realistically. Whether it’s avoiding obstacles or interacting with other AI characters, spatial awareness is key to creating believable virtual worlds.
4. Digital Twin Applications
AI-powered simulations aren’t just for games—they can be used in digital twin applications, where real-world data is mirrored in a virtual environment. By populating digital twins with AI characters, industries such as urban planning, architecture, and simulation training can benefit from realistic, interactive environments.
Conclusion: Bringing the Matrix to Life in Unreal Engine
Adding AI-powered characters to Unreal Engine’s City Sample Scene is a game-changer for developers looking to create immersive, interactive worlds. With tools like Convai, bringing lifelike AI characters to your projects has never been easier. From dynamic conversations to realistic body and facial animations, you now have the tools to create Matrix-level experiences in your game or virtual world.
So, what are you waiting for? Start integrating AI-powered characters into your Unreal Engine projects today, and bring the Matrix to life in ways you never thought possible!
By following the steps outlined in this blog, you’ll be well on your way to creating immersive, AI-driven virtual worlds that captivate players and users alike. Happy developing!
For more detailed tutorials and support, check out our YouTube Channel. Also, reach out to us at support@convai.com for any questions or concerns. Also, join our amazing developer community on Discord to learn more about integrating AI characters into different types of projects.
