Gamecontroller Bachelor Project - Advice and suggestions

[SaabFAN]

Hi @ all,

As some here might know, I'm studying Game Art and Interactive Animation, but I also do like to tinker with electronics and have managed to complete a few projects that brought in some money to buy me shiny new toys (mostly T&M-Equipment and parts for projects I have yet to finish ).
So now that I have to do my Bachelor-Thesis, I've decided to build a device that allows a game made with the Unreal Engine 4 to interface with a breadboard on which circuits can be built to complete quests the game gives the player.
The idea is to allow game designers and developer studios to make Education-Entertainment with physical experiments instead of only simulated experiments. This allows students to grab the Multimeter or Scope and see what the circuit actually does. WITH the possibility that, for example, the amplifier turns into a oscillator or the oscillator putting out an entirely different frequency than calculated. This, in my opinion, offers a much better and more fun learning experience. Especially if combined with a game that uses the experiments in a smart way to advance the story.

Based on the Book "Halbleiter Schaltungstechnik" by Ch. Tietze and U. Schenk, I've devised a few experiments:
- Voltage based Experiments with resistors (Voltage and Current - Ohm's law)
- Voltage based Experiments with Diodes (LEDs, Zeners)
- Building Amplifiers, Attenuators (Involves basic Transistor-Circuits and OpAmps)
- Experiments involving oscillators (measurement of Frequencies)
- Filters (Measuring of AC-Voltages on LowPass-Filters, and Frequencies on HighPass- and BandPass-Filters)
- Simple digital logic (Use of various 74-Series Chips to create the desired output-signal from a Input-Signal)
- Complex digital logic (Requires a Microcontroller or CPLD-Module that won't be part of my Bachelor-Thesis)
- RF-Experiments (Mixers, Receivers)

Do you guys miss something here?
At the moment I'm in the first design-phase of the project (My deadline for the 1st PCB-Design is early November - The due-date of the whole thing is 31st March 2018), where I define what Inputs and Outputs I need. That will determine what kind of Arduino-Compatible Microcontroller I'm going to use.

Thx in advance for any ideas and suggestions you have on this project.

Also what do you think about involving computer games in teaching electronics?

[rstofer]

I'm not up to date on how you tie external controllers into Unreal Engine.  I assume the device starts out as a HID of some kind.  For some variants of Arduino, creating a HID device is already done.

The other day I was messing around with Matlab and an Arduino.  That's pretty much a drag and drop operation using many of the Arduino IO peripherals.  It takes Matlab ($), Simulink($) and the Simulink Arduino Library (free).

I have created a couple of maps with Unreal and it is a very nice platform.  I wish it included more meshes but that's just because I am too cheap to buy them.

[SaabFAN]

The interface between the device and Unreal will be a PlugIn I found made especially for connecting an Arduino to the Unreal Engine.
It is a PlugIn capable of receiving and sending messages from / to the COM-Port. I'm pretty glad I found that, as it shaves quite a bit of work off of the entire project

[galvanix]

Based on the Book "Halbleiter Schaltungstechnik" by Ch. Tietze and U. Schenk, I've devised a few experiments:

The only book you'll ever need ^^

- RF-Experiments (Mixers, Receivers)

Those might not be possible on a breadboard depending on the frequencies involved because of the breadboard's parasitics.

[SaabFAN]

Those might not be possible on a breadboard depending on the frequencies involved because of the breadboard's parasitics.

Would it work if the frequencies involved were below 8 or 10MHz?
The Experiments would involve creating and receiving AM-Signals, as well as up- and down-converting frequencies and filtering them. The Controller will not have any fast ADC, so it won't be able to do FFT on anything higher than audio-frequencies. But it will have a frequency-counter, as well as digital inputs.

[markus-k]

Those might not be possible on a breadboard depending on the frequencies involved because of the breadboard's parasitics.

Would it work if the frequencies involved were below 8 or 10MHz?
The Experiments would involve creating and receiving AM-Signals, as well as up- and down-converting frequencies and filtering them. The Controller will not have any fast ADC, so it won't be able to do FFT on anything higher than audio-frequencies. But it will have a frequency-counter, as well as digital inputs.

My uneducated guess would be no more than a couple kHz for repeatable results, but I've seen an 8 MHz crystal work on a breadboard.

But sounds like a great project, I like the idea

[twiddle]

So a little context here - I teach game development at a private college here in Sydney.
I'm a contributor to Unreal's code base, amongst other things, and I run second-year courses that integrate custom HID-based game controllers with Unreal.



Sometimes we prototype using a STM32 'bluepill' on a breadboard, with a joystick and a few buttons, and so on, and that side of it works just fine on a breadboard, sending HID reports to the computer that are read using a simple C++ Unreal plugin that wraps the HIDAPI library and exposes that to Blueprint. We've never run into any issues with running that side of things on a breadboard, so I'd say that at least switches, ADC, simple digital logic, should all be do-able just fine.
 
You may want to consider using HID instead of COM depending on what OS you are targeting - before windows 10, HID was the only driver-less way to get this running (previous Windows versions required inf files for COM ports).

I think teaching electronics in conjunction with game dev, either using my approach (where the electronics is part of the implementation) or yours (where your mechanics are based on electronics) works just fine.
You could also consider looking at Shenzen I/O from Zachtronics for an example of a gamified approach to microcontroller development.

[SaabFAN]

I have to decide on the microcontroller now. Block-Diagram and the list of requirements are done.

I need:

I/O:
- 8bit I/O-Port + 2 Control-Pins (10 Pins)
- 4 GPIO-Pins
- i2c-Bus (2 Pins)
- SPI-Bus + 2 ChipSelect-Pins (5 Pins)

Input only:
- Timer-Input
- 4x Analog Input (Optional: Fast enough to do some FFT-Analysis)

Output only:
- 2x PWM-Output

Interface:
- UART to USB / USB only if I'm going the HID-Way (see more about that below)

Microcontrollers that I've evaluated so far: STM32 "Bluepill" (STM32 F103C8), Arduino Due (ATSAM3).

I'm leaning towards the Due, but integrating the micro might be easier with the "Bluepill".


@twiddle: How difficult is it to implement communication between a custom HID-Controller and Unreal Engine? Can it be done with Blueprints alone, or do I have to write some C++ Code for Unreal Engine?
As my project is centered around the Hardware, I want to keep the work required to make the engine talk to the controller at a minimum.

[twiddle]

I have to decide on the microcontroller now. Block-Diagram and the list of requirements are done.

I need:

I/O:
- 8bit I/O-Port + 2 Control-Pins (10 Pins)
- 4 GPIO-Pins
- i2c-Bus (2 Pins)
- SPI-Bus + 2 ChipSelect-Pins (5 Pins)

Input only:
- Timer-Input
- 4x Analog Input (Optional: Fast enough to do some FFT-Analysis)

Output only:
- 2x PWM-Output

Interface:
- UART to USB / USB only if I'm going the HID-Way (see more about that below)

Microcontrollers that I've evaluated so far: STM32 "Bluepill" (STM32 F103C8), Arduino Due (ATSAM3).

I'm leaning towards the Due, but integrating the micro might be easier with the "Bluepill".


@twiddle: How difficult is it to implement communication between a custom HID-Controller and Unreal Engine? Can it be done with Blueprints alone, or do I have to write some C++ Code for Unreal Engine?
As my project is centered around the Hardware, I want to keep the work required to make the engine talk to the controller at a minimum.

FWIW, I prototyped the current project with a Bluepill and it worked quite well, even though my actual target micro was STM32L073.

With regards to communication with Unreal - look into the RawInput plugin that's included with the engine. It's not that well documented but if you understand how HID works it looks like it will get the job done (I intend to try it next time around). In theory this should be a Blueprint-only solution.

For my own solution, I've written a custom C++ plugin that using the HIDAPI library to connect to my hardware directly and provide it's own custom Input Axes and Actions. If you didnt want to bother actually showing up as a controller within Unreal, and simply poll the device for reports, you could probably simplify the integration by just creating a Blueprint Function Library that reads said reports from your device.
I guess it would be even simpler to go with CDC rather than HID and just send the data over as text that you can parse. I know of at least one commercial game with a custom controller that is using this approach, and it's been robust enough that they've taken their controller to GDC and PAX without issues. That project wasn't Unreal-based, though.

[SaabFAN]

Today I spoke about the project with the academic coordinator at the SAE Institute in more detail and it turned out that I need to include a more in depth theoretical part.
This means I have to interview lecturers or professors at universities about what they think, what kind of features a game controller designed to teach electronics should have.

I'll write to the universities in Hamburg tomorrow, but I would also like to include, if possible, someone from a different country to get a more broader spectrum.

Anyone here teaching electronics and willing to answer some questions via Email?

[twiddle]

Today I spoke about the project with the academic coordinator at the SAE Institute in more detail and it turned out that I need to include a more in depth theoretical part.


Anyone here teaching electronics and willing to answer some questions via Email?

Not I, but I teach game dev at SAE's campus here in Sydney