Arduino Game Controller

This Rrduino Game Controller Project is an extension of my Mechatronics Final Year Project. It’s my Arduino Glove that I am using to control the video game. So before you continue it’s good too check my Mechatronics Final Year Project in order to see how the device works and how it’s made.

Here I will explain in details how I programmed the Arduino board using the Processing IDE to enable the control of the video game. Watch the video below to see the Arduino Game Controller in Action playing Need For Speed.

How It Works


This is how Arduino Game Controller works:

  • The sensors of the glove capture the movements of the arm.
  • The captured values are sent into the Arduino board Analog Inputs.
  • From the Arduino they are sent to the Processing IDE via Serial Communication.
  • From the Processing IDE they are sent to the video game.

Arduino IDE


First I programmed the Arduino board using the Arduino IDE to work as a server that would continuously run on the Arduino board. That code enables the Serial Communication between the Arduino board and the Processing IDE.

Here is the Source Code:

/*  
 *  Arduino Game Controller
 *  
 *  Crated by Dejan Nedelkovski,
 *   www.HowToMechatronics.com 
 *   
 */

// Defining variables

int pinX=A8;
int pinY=A7;
int pinZ=A6;
int pinA0=A0;
int pinA4=A4;
int pinA3=A3;
int pinA1=A1;
int pinA2=A2;

void setup() 
{
  Serial.begin(115200);     // starts the serial communication
}

void loop()
{
  int valX=analogRead(pinX);     // reads the Analog Input, t.e the value from the X - axis from the accelerometer
  Serial.print(valX);     // sends that value into the Serial Port
  Serial.print(",");     // sends addition character right next to the read value needed  later in the Processing IDE for indexing
   
  int valY=analogRead(pinY);
  Serial.print(valY);
  Serial.print("/");
  
  int valZ=analogRead(pinZ);
  Serial.print(valZ);
  Serial.print(";");
  
  int valA0=analogRead(pinA0);
  Serial.print(valA0);
  Serial.print(":");
  
  int valA4=analogRead(pinA4);
  Serial.print(valA4);
  Serial.print("<");
  
  int valA3=analogRead(pinA3);
  Serial.print(valA3);
  Serial.print("!");
  
  int valA2=analogRead(pinA2);
  Serial.print(valA2);
  Serial.print("?");
  
  int valA1=analogRead(pinA1);
  Serial.print(valA1);
  Serial.print(".");
  
  delay(30);
  
}

From the code above you can see that I am using the AnalogRead() function to read the values from the Accelerometer for the orientation of the arm (3 variables) and the Potentiometers for the position of the fingers (5 variables). Now these values are being sent via the Serial Port to the Processing IDE. Also right after each variable I send a specific character to the Processing IDE using the Serial,print() function which will work as an Index.

So when I have the glove connected to the computer I will continuously send the above (picture above) line of data through the Serial Port. The numbers are values from the sensors and the characters are for indexing them which will help when receiving them in the Processing IDE. Note: The numbers are just for example, they can varies according to your sensors readings.


Processing IDE


Now I had to receive the data coming from the Serial Port into Processing IDE and according to them send commands to the video game. For that, I developed the code below and this is how it works:

  • First it read the data from the Serial Port.
  • It separates that data into variables from each sensor.
  • According to the values from each sensors it simulates keyboard key press or release which actually are controlling the video game.

Go trough the code and find detailed explanations for a specific functions and lines of codes in the comments of the code.

Note: The values in the code are set according to my readings from my sensors. You should adjust them according to your readings. Also for explanation in the comments of the code I am using the values from the picture above, which are just example numbers.

/* 
* Arduino Game Controller Project
* 
* Crated by Dejan Nedelkovski,
* www.HowToMechatronics.com 
* 
*/

import processing.serial.*; // imports library for serial communication
import java.awt.Robot; // imports library for key press or release simulation
import java.awt.event.KeyEvent; // imports library for reading the data from the serial port
import java.io.IOException;

Serial port; // defines Object Serial
Robot robot; // defines Object Robot

//defining variables

String X= "";
String Y= "";
String Z= "";
String A0= "";
String A1= "";
String A2= "";
String A3= "";
String A4= "";
String data= "";
int index=0;
int index2=0;
int index3=0;
int index4=0;
int index5=0;
int index6=0;
int index7=0;
int iX=0;
int iY=0;
int iZ=0;
int iA0=0;
int iA1=0;
int iA2=0;
int iA3=0;
int iA4=0;

// creates new robot object

void setup()
{

try 
{
robot = new Robot();
}
catch (Exception e) {
e.printStackTrace();
exit();
}

delay(2000);
size (800, 800);
port = new Serial(this,"COM3", 115200); // starts the serial communication
port.bufferUntil('.'); // reads the data from the serial port up to the character '.'. So actually it reads this: 215,214/141;315:314<316!314?315.
}

void draw()
{

background(0,0,0);
fill(255, 255, 255);

//Simulating key press or release

// turn left

if(iX>320)
{ 
delay(40);
robot.keyPress(KeyEvent.VK_J); // Simulates "I" key press if the value from the accelerometer for the X axis is greater than 320 
}

if(iX<=320){
delay(40);
robot.keyRelease(KeyEvent.VK_J); // Simulates "I" key release if the value from the accelerometer for the X axis is less than 320 
}

// turn right

if( iX<280 ) 
{
delay(40);
robot.keyPress(KeyEvent.VK_L); 
}

if(iX>=280){
delay(40);
robot.keyRelease(KeyEvent.VK_L);
}

// turn up

if(iY>320)
{
delay(40);
robot.keyPress(KeyEvent.VK_I); 
}

if(iY<=320){
delay(40);
robot.keyRelease(KeyEvent.VK_I);
}

// turn down

if( iY<280 ) 
{
delay(40);
robot.keyPress(KeyEvent.VK_K);
}
if(iY>=280){
delay(40);
robot.keyRelease(KeyEvent.VK_K);
}

// accelerate - indexFinger

if( iA4<510 ) 
{
delay(40);
robot.keyPress(KeyEvent.VK_W); 
}

if(iA4>=510){
robot.keyRelease(KeyEvent.VK_W);
}

// handbrake - thumbFinger

if( iA0<500 ) 
{
robot.keyPress(KeyEvent.VK_SPACE);
}

if(iA0>=500){
robot.keyRelease(KeyEvent.VK_SPACE);
}

// reverse - middleFinger

if( iA3<560 ) 
{
robot.keyPress(KeyEvent.VK_S); 
}
if(iA3>=560){
robot.keyRelease(KeyEvent.VK_S);
}

// shift up - ringFinger

if( iA2<400 ) 
{
robot.keyPress(KeyEvent.VK_R); 
}

if(iA2>=400){
robot.keyRelease(KeyEvent.VK_R);
}

// shift down - littleFinger

if( iA1<250 ) 
{
robot.keyPress(KeyEvent.VK_F);
}

if(iA1>=250){
robot.keyRelease(KeyEvent.VK_F);
}

}

// Reading data from the Serial Port

void serialEvent (Serial port) // starts reading data from the Serial Port
{

data = port.readStringUntil('.'); // reads the data from the serial port up to the character '.' and it sets that into the String variable "data". So actually it reads this: 215,214/141;315:314<316!314?315.
data = data.substring(0,data.length()-1); // it removes the '.' from the previous read. So this will be the String "data" variable: 215,214/141;315:314<316!314?315

// Finding the indexes in the data and setting the variables from the sensors by taking from the String "data" the appropriate values that are between the characters in the "data" String

index = data.indexOf(","); // finds the index of the character "," from the String "data" variable
X= data.substring(0, index); // sets into the variable X the string from position 0 of the hole string to where the index was. That would mean that read will be : 215

index2 = data.indexOf("/"); // finds the index of the character "/" from the String "data" variable 
Y= data.substring(index+1, index2); // sets into the variable Y data the string from position where the character "," was +1, to where the index2 was. That would mean that the read will be: 214

// We keep reading this way and that's how we get only the numbers, the values from the sensors coming from the serial port.

index3 = data.indexOf(";");
Z= data.substring(index2+1, index3);

index4 = data.indexOf(":");
A0= data.substring(index3+1, index4);

index5 = data.indexOf("<");
A4= data.substring(index4+1, index5);

index6 = data.indexOf("!");
A3= data.substring(index5+1, index6);

index7 = data.indexOf("?");
A2= data.substring(index6+1, index7);

A1= data.substring(index7+1, data.length());

// Converting the String variables values into Integer values needed for the if statements above

iX= int(X);
iY= int(Y);
iZ= int(Z);
iA0= int(A0);
iA4= int(A4);
iA1= int(A1);
iA2= int(A2);
iA3= int(A3);
}

Feel free to ask any question in the comments section below.

29 Responses

  1. Awais Mumtaz

    I wish to make this. Please can u send me some litreture which can help me with this?

    Reply
  2. Nilesh Sable

    Which game u have played????? Is this program common for all the racing games???

    Reply
  3. Tejas

    when i uploaded the program in arduino and run the sketch in processing ide my laptop automatically started misbehaving…..and it started typing something sfwr….sfwr .how to get rid of that ??

    Reply
  4. Tejas D

    Hello Sir, i think the program which u have written for the Processing IDE has got some Errors.PLEASE check it once again and re upload .We need ur help.Actually we tried the same ,but its not working.

    Reply
  5. Erdi G

    Congratulations for the project!

    I am curious about if you ever tried this controller as a game controller device for Windows or such OS’s? If so is there any sources those I can get?

    Reply
      • itachi

        I get the idea how you used mpu6050 but i didn’t understand how are you changing the value of Potentiometers without touching it? Could you
        please explain it?

  6. Huda

    Congratulations for the project!

    I am curious about how to i change the control of my game to take orders from arduino?

    Reply
  7. Aradhana

    Hey, its an amazing project. I tried implementing it but found an error in processing2.2.1 as—“Error, disabling serialEvent() for COM3
    null”, can please guide me in finding a way out of it??

    Reply
  8. Orcin

    Congratulations. You’ve done a great job here.

    But let me ask few questions;

    I want to control my keyboard with MPU6050

    For example;

    If the angle bigger than X degrees, press “Q” and release if it is lower than X degrees etc..

    What sould I do? I am a beginner at arduino programming and I would appreciate with any help.

    My e-mail adress is: ozincir@gmail.com

    Thanks in advance and good luck with your next projects.

    Orcin.

    Reply
    • Dejan Nedelkovski

      Thanks. Well the point of this tutorial is exactly that one, just using a different sensor (accelerometer).
      Basically, you need to read the MPU6050 data, send it to Processing IDE and from there simulate the key strokes like in this project.

      Reply
      • Orcin

        I managed to work your code.

        When i open microsoft word, i can see keys typing accurately.

        But when i open a game, nothing happens.

        Is game disabling IDE?

        What should i do?

  9. Jonathan P.

    Hello Dejan.

    I’m assuming it’s possible to translate a simple digital buttons on the arduino into a keypress output, correct? Can you refer me to a simpler tutorial just using a few buttons instead of potentiometers and accelerometers? If there is none, could you briefly explain how I would simplify the code to accommodate this?

    Thanks.

    Reply
    • Dejan Nedelkovski

      Hi there. Well yes you can do that in similar way to this project. So first you need to understand how to use Arduino and Processing together. So from the Arduino you will send a signal to the Processing IDE for each button press. Then using the “Robot” library in the Processing IDE you can simulate the key strokes when the particular signal is received.
      So, first I suggest to check my Arduino and Processing IDE and then with the help of this Arduino Game Controller Project make your code.

      Reply
  10. sourabh

    Hi,
    All that you are controlling is digital signals,like key press and key release. Can we simulate analog signals in the game, like steering, Accelerator, Brakes?
    Please suggest.

    Reply
    • Dejan Nedelkovski

      That’s right, the glove is just simulating digital signals, key strokes. At this moment I couldn’t suggest anything for analog simulation, as it’s a completely different, in terms of communication with the game.

      Reply

Leave a Reply

Your email address will not be published.