Creating an engaging AI-Powered Game using Unity and Hugging Face 🤗

Let's build a casual game where the AI asks the player to say nice, mean and interesting things

This tutorial is part of the Making Games with AI Course. A free upcoming course on creating AI-powered games for Unity and Unreal.

---

The integration of cutting-edge AI models in video games is opening a whole range of new exciting gameplays. For instance, in the last tutorial, we created a smart robot NPC that can understand your orders and perform them we used a Sentence Similarity model.

In this tutorial, you’re going to modify a game called “Say Something” where an AI asks you to write something either nice, mean or informative/interesting.

🤖 The AI understands you and rate what you said, and if ✅ gives some points and additional time ⏱️.

Your goal then is to be as fast as possible to win time and get the highest score.

You can play with my version on your browser 👉 here

To make this project, we’re going to use:

• Unity Game Engine (2022.3 and +).

• A Unity Package containing the assets we created 👉 here

• Hugging Face Unity API 🤗, a free and easy-to-use integration for the Hugging Face Inference API, allows developers to access and use Hugging Face AI models within their Unity projects.

The requirements:

• You need to know to use Unity. If it’s not the case, you can check this excellent free course 👉 Create with code

At the end of the project, you’ll build your own version of Say Something that you can publish for free online on Hugging Face Spaces.

You can find the assets for the project 👉 here

So let’s get started!

(Important note: In this tutorial, I interchangeably use the terms ‘AI’ and 'model.')

What is Say Something? 🗣️

Say Something is a casual game where the player needs to type something mean/nice/interesting based on what the AI asks.

The goal is to get the highest score.

The Game Rules 📜

The Game Rules are simple:

⏲️ 60 seconds: the player starts with 60 seconds countdown.

⌨️ On your keyboard: the AI asks the player to write something either mean, nice or neutral.

📏 Size Matters: larger player’s input yield higher scores and extra time.

🤖 AI Judgment: the AI understands player’s input and decides if it's mean/nice/neutral enough.

But how the AI judge if player’s phrase is "good enough" 🤔?

---

How does this game works? ⚙️

The model we use is quite simple, it’s a Text Classification model, more precisely a Zero Shot Text Classification model.

Text Classification

Text Classification models are able models that are able to understand the text player wrote and assign it a class.

For instance in this example, writing I love Hugging Face, the model understood that this is a positive phrase and labeled it as positive.

We also call that kind of model (that label a text as positive, negative or neutral) a sentiment analysis model.

Thanks to this model, we're able to create an original gameplay.

You can try by yourself on your browser here, and learn more about text classification 👉 https://huggingface.co/tasks/text-classification

But in our case, we don’t want our model to classify between positive, negative or neutral. We want to define specific labels: nice, mean, interesting. For that, we can use a Zero-Shot Text Classification model.

Zero-Shot Classification

Text Classification models we just talked about where trained on specific labels. For instance a simple sentiment analysis text classification model learned to class text between positive, negative or neutral.

But in our case, we want to do something funnier. We want to ask our player to write something mean, nice, informative and more! Why not something funny or scary?

For that we can use a Zero-Shot Classification model, it’s models that were trained on a set of labeled examples but is then able to classify new examples from previously unseen classes.

This means that we can ask them to classify the text player wrote on labels we decide.

We see in this example, that we want our model to be able to classify our phrase between 3 labels: Cinema, Art and Music.

You can try by yourself on your browser here, and learn more about zero-shot text classification 👉 https://huggingface.co/tasks/zero-shot-classification

To find a Zero-Shot Text Classification model, we’re going to use Hugging Face.

---

What is Hugging Face 🤗?

Hugging Face is a platform where users can upload or download pre-trained AI models in more than 35 different tasks from text generation with Llama v2, speech-to-text with Whisper, text-to-speech (AI voices) with Coqui to 3D generation.

For most of these models, Hugging Face provides a free API called Inference API that allows you use them via simple API calls.

So how I can select a Zero-Shot Classification Model on Hugging Face 🤗?

All the AI models are listed here https://huggingface.co/models. On the left of this page, you have the differents tasks.

In our case you want to select Zero-Shot Classification

Now we have a list of Zero-Shot Classification models (approximately 200). You can click on the top one, and test if they’re good or not.

Personally, I chose facebook/bart-large-mnli one.

Now that we finished with the theory, it’s time to build your version of Say Something!

---

Let’s make your version Say Something!

Step 1: Let’s review the Game Process

The game process is quite simple:

• A topic is chosen randomly.

• The player is asked to write something mean (for instance).

• If the player’s text is too short, the player is asked to write something longer (this avoid cheating on writing only one word, plus the player is pushed to write longer phrases to get more point and more additional time).

• This player input is sent to the model (AI) via a free API.

• The model (AI) classify the player input.

• The model gives a score for each label.

If the player input is mean ➡️ the player gets points and bonus time.

Else➡️ the player is asked to writer something meaner.

Now that you understood how the game works you’re ready to make your own version 🔥.

Step 2: Getting Started with Hugging Face 🤗

Hugging Face contains a lot of amazing AI models and an API (Accelerated Inference API) to directly plug them into your projects.

But first, you need to create a free account.

When your account is created, go to Settings by clicking on your profile picture (top right-hand corner).

In Settings, select Access Tokens

Create a new Access Token, give it the name you want, and select write Role:

Now that the token is created, copy the API Token, it’s the key you’ll need to be able to use the API.

⚠️ For security reasons, DO NOT SHARE THIS KEY to others, it’s a private key.

Step 2: Select your Zero-Shot Classification model 🤖

Now that we have the API key, we can try some models to choose one.

When you click on of the models, you can try the model directly on the website. Let’s make a test, go to the first 5 models and test them.

Here I tested this model: https://huggingface.co/facebook/bart-large-mnli

Most importantly, I wanted to test that the model is able to understand complex phrases such as this one which is clearly a mean one.

⚠️ Don’t forget when you test a model to uncheck 🔲 Allow multiple true classes since we don’t use this feature in our game.

I’m going to use this one in the game. But feel free, to test more models.

Now that we choose this one, we need to get the API URL, to do that click on Deploy > Inference API.

This opens a window where you can copy the API_URL:

You’re now ready to use this model in Unity.

Step 3: Create a new Unity Project and Install the Unity API Plugin

1. Create a new Unity Project (I used 3D URP project).

2. Open the Unity Project

3. Go to Window -> Package Manager

4. Click + and select Add Package from git URL

5. Enter https://github.com/huggingface/unity-api.git

5. Once installed, the Unity API wizard should pop up. If not, go to Window -> Hugging Face API Wizard

6. Enter your API key. Your API key can be created in your Hugging Face account settings.

7. Test the API key by clicking Test API key in the API Wizard.

8. Paste your model API_URL on the Zero-Shot Text Classification Task.

9. Check ✅ wait for model

10. Download the Unity package here and place it into your Unity project.

11. Open the Student_Version Scene

12. Click play and try the game.

⚠️: Sometimes the first API call you do during the game is long because the model needs to turn it on. One solution can be to add a loading screen with a fake Query() call to turn the model on.

You’re now ready to modify the game to make your own version 🔥.

Step 5: Check the Game Manager

If you check the GameManager.cs file in Scripts/GameManager.cs you’ll be able to see the whole codebase.

Each function is commented is explained, so it’s relatively straightforward, but the most important function is Query(). It’s the one that makes the API call to the model you choose.

Step 6: Add a topic/change a topic

The game is nice, but what about adding more interesting content? Like the AI is asking you to write something funny instead of interesting?

What’s amazing here, is that you don’t need to change a lot of things in GameManager.cs:

• You need to change the topics array, which are the labels sent to the model, by replacing “interesting” by “funny”.

• In DisplayTopic, we need to change the case of information with funny. So that the game display Say Something Funny!

• You can also change the SetAIEmojiEmotion by adding new emotion faces (for instance the robot laughing). I’ve made mine using emojimix

And boom in 3 small changes, you created your own version with new labels features 🤯.

I have three piece of advice if you want to add more labels:

1. Test first on the Hugging Face website widget like we did above, this way you’ll see if it works correctly.

2. For each new label you add decrease the thresold, because the classification of all labels sum to 1. So the more labels the smaller the best classification score.

3. Avoid having labels too close to each others: for instance interesting and funny. Or mean and insulting.

Step 7: Share your demo with the world 🔥

Congratulations on completing your demo! Now, let's make it accessible to everyone to play it by publishing it online for free.

Hugging Face offers a convenient and free solution called Spaces.

We've prepared a quick 5-minute tutorial to guide you through the process: https://huggingface.co/blog/unity-in-spaces

Follow the steps in the tutorial to showcase your creation and let others experience it 🚀

I published mine on Spaces, you can try 👉 here

Don’t hesitate to ping me @thomassimonini on twitter and Thomas Simonini on linkedin when you share your demo ❤️.

---

That’s all for today, you just created your own version of Say Something!

What you can do next is dive deeper into the code and modify the game, to add more features, to change the topics! I commented on every part of it so it should be relatively straightforward.

If you liked my article, please click the ❤️. And don’t hesitate to follow me on substack and Twitter. I write about AI in Games.

This tutorial is part of the Making Games with AI Course. A free upcoming course on creating AI-powered games for Unity and Unreal. Don’t forget to sign up 👉 here

If you want to know more about language models and check out the amazing HuggingFace course on it (it’s free) 👉 https://huggingface.co/course/chapter1

In our last two tutorials, we learned to utilize a model through an API. However, imagine the incredible possibility of running these AI models directly on the player's machine 🤯. This is made possible by Unity Sentis, and that's exactly what we'll be exploring in our next tutorial!

Also a big thanks to Dylan Ebert, for his help on making this demo possible 🤗.

See you next time,

Keep learning, stay awesome 🤗