For years, sound/lighting have known that in order to be able to discover and capture the “magic” of their various mediums, having real-time feedback is key and it’s almost asinine that as an industry that sees so clearly how important real-time feedback is for our games, the standard process for scripting and balancing by stopping, adjusting, building and running is anything but real-time.

“Ideas are very important to me. I think that bringing ideas into the world is one of the most important things that people do. And I think that great ideas, in the form of great art, stories, inventions, scientific theories, these things take on lives of their own, which give meaning to our lives as people. So, I think a lot about how people create ideas and how ideas grow. And in particular, what sorts of tools create a healthy environment for ideas to grow.

I’ve spent a lot of time over the years making creative tools, using creative tools, thinking about them a lot, and here’s something I’ve come to believe: Creators need an immediate connection to what they’re creating. That’s my principle. Creators need an immediate connection to what they create. And what I mean by that is when you’re making something, if you make a change, or you make a decision, you need to see the effect of that immediately. There can’t be a delay, and there can’t be anything hidden.”

– Bret Victor (Inventing on Principle)

MIDI Controller In UE5
Back in 2012 at Lucas Arts, we created our own interface to be able to live tune various settings by binding the value ranges to various midi controls. The person I worked on this with, @John Ballantyne (Cloud Chamber), currently works over in the other building at 2K as a Technology Directory but after we made this he went to go work for Epic and while there, helped to develop this functionality into Unreal directly.It’s somewhat buried in the UE5 documents but is now available in beta for anyone designers/artists/engineers/directors that want to try it out.

Is it actually useful?
The obvious use would be to use it to control something like meta-sounds or totune VFX live; and our lighting director in particular loved having the ability to run the game and live-tune each room one-at-a-time using a lighting board as the directors did their play throughs back when we first implemented a differed lighting solution in UE3 that let us have 1000s of lights in a scene, but it was also really useful for our systems/weapons/gameplay design and balance.

At one point we even got it working on a live 16-player server for our multi-player project First Assault (the internally developed Battlefront we were working at at Lucas Arts before the Disney buy-out and studio closure) and could use it to live adjust things like move speed, FoV, weapon/camera parameters, etc. during our daily playtests for our multiplayer balance testing. We would play a few rounds, do feedback and immediately adjust, then play again while we silently honed in on the specific set of variables we were A/B testing each day.

We also had this tech on my last project at Hangar 13 that used Fusion (the Mafia III engine) and nobody knew anything about it, but I was able to make a local build with the config for my particular MIDI controller that i have on my desk, an AKAI APC-20, however, only if I run with an old non-IMGui console via a special command line parameter. I used it frequently for various AI tuning and balance and will be setting it up at some point in UE5, including adding an additional Wacom Express Remote for even more convenient micro-tuning.

How do I try it?
If you are someone who frequently has to stop, change values and relaunch to test changes (I found myself doing it hundreds of times a day at one point), then I highly recommend giving it a try. You can get a pretty cheap MIDI controller for under $100 on amazon with various knobs, faders, buttons and toggles (keyboards are cheap but not quite as useful).

You can find the documentation on it here:
https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/MIDIDeviceManager/
https://docs.unrealengine.com/5.0/en-US/BlueprintAPI/MIDIDeviceOutputController/
https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/MIDIDeviceInputController/

Basic setup
There are videos online of how to set it up and I recommend making a template for each thing you want to tune (weapon, camera, enemy) and then keeping it always running/plugged in in case you need to make any adjustments while testing changes.

Once you get used to having it as a tool in your arsenal, you’ll wonder how you ever worked with out it (or at least I did 10 years ago and have been using it since).

Here is a video of us using it to tune the weapons and camera back at Lucas Arts when we were working on Star Wars: 1313.