GDC 2012

I am finally back from GDC 2012 and I feel exhausted. Exhausted in a way that only GDC's unique blend of information overload, loud parties, late night drinking, and presentation stress can achieve. But, I would do it again in a heartbeat if given the chance.

I got a lot of personal feedback from my talk and it was all positive ranging from "it was quite informative" to "you kicked ass". My talk was even brought up by people several times during the AI Roundtable sessions to help make their point. That was a surreal experience.

My GDC experience started Monday, where I got a chance to watch all of the AI Summit speakers give their talks. This gave me an opportunity to get a feel for how things worked and how people gave their presentations. Using this information, as well suggestions given to me by Dave Mark and a couple of co-workers, I spent Monday night adjusting my slide deck. Time well spent.

Tuesday was much of the same, except that in the back of my mind I knew I would be on the stage later that day. Oddly enough, though, I wasn't nearly as nervous as I thought I would be. It wasn't until Chris Jurney (one of the two other people my presentation was grouped with) had finished his presentation on Double Fine Happy Action Theater that I had a fleeting bout of nerves, but I knew that if I just got up and started talking I would get some momentum and be fine.

The four neural networks
My talk started out by going over the the basics like what we using the neural networks for, what a neural network is, and so on. This included a video of the neural networks in action followed by me going over what decisions in the video were actually controlled by the neural networks.

How data is fed into the neural net
The second section of my presentation dealt with how platoons actually used the neural networks. I described the various inputs we gathered and how they were converted to deltas to be fed into the neural network. I also described how the resulting outputs from the neural network corresponded to actions the platoon could take. I also told the audience what an ACU was and its importance, which I mentioned would come into play later.

Next, I went over how the neural network was trained. First, I went over the benefits of using a neural network in regards to training. I then went on to describe how the neural network learns what actions to take given various circumstances, which lead us to the fitness function.
My favorite slide

The next section of my talk dealt with describing what a fitness function is and how it is used to train a neural network. I then continued to describe the learning process, which included my favorite slide of the presentation. Finally, I described how long training took and what actions could cause you to have to retrain your neural networks.

Next came debugging. I asked the question: how do you know if your neural networks are working? The answer; visual debugging. I showed a short clip of the game running in debug mode with platoon debugging turned on. I explained that this allowed me to see what the platoon was doing and why. I also explained that this was very important in determining whether or not odd unit behavior is being caused by behavior code or by bad neural network output. Or, it could be caused by something far more sinister: good output.

Its not optimal
I then referred the audience back to my comments about the ACU being a very important unit in Supreme Commander 2. How destroying your enemy's ACU would win you the game. I then let them know that I left something out; what happens when the ACU dies. I then described how the ACU goes up in a nuclear explosion when it dies and how this played hell with the neural network.

Log output
So, I described to the audience my agony over this situation. I described how the AI would send massive platoons of units at a player, only to run away when the ACU showed itself. Then, I described my epiphany: don't try to fix the behavior in a traditional sense, try to teach the neural network the right thing to do instead. I told the audience that changing over to this mindset helped me come up with a solution. I described how I modified the fitness function to teach neural networks that killing the ACU was a good thing, no matter the cost. I went on to describe the beneficial behavior that resulted as I started a final video. I reached a preset point in my description where I said "as you can see" when an ACU explosion filled the screen and I finished with "they have gotten quite good at it."

For the final part of my presentation I left the audience with a final thought that boils down to this: Using neural networks where they make sense can have awesome results.

For anyone interested, the full PDF of my slide presentation can be found here: Presentation PDF

It wont be the same as being there, but I can't help that unless you have GDC Vault access.

If you look at the slides bear in mind two things:
1) Each bullet point was brought up individually as I talked, not all at once.
2) The pictures on slides 4, 20, and 27 are actually videos and the text on 4 appeared after the video.

13 comments:

Luke Degner said... / March 12, 2012 at 4:05 PM  

Thanks for the post! I wish that I could see your talk, but the slides will have to do. I am not doing AI myself, but I did enter that Google ants AI challenge and that made me appreciate AI programmers even more than I already did.

What do you think are the best applications for using neural networks, and where do you think that they are incorrectly used? They sound very interesting, but I don't know if I will ever get to use them (or things like them).

As always, keep up the great job and keep enjoying your work!

Emory said... / March 12, 2012 at 6:51 PM  

Great talk! Hopefully your presentation went a long way to improving the reputation of ANNs for Game AI.

You'd mentioned arriving at the number of hidden layers/nodes by following some advice found online. Did this work immediately?

I've long been interested in determining topology via a GA, something like NEAT - would this be overkill?

Michael Robbins said... / March 12, 2012 at 7:56 PM  

@Luke: The best applications for neural networks in games that I am aware of is in discreet situations where you want the most optimal solution, or at least the ability to rank solutions.

@Emory: The neural networks performed beautifully so I never bothered to change the number of hidden layers or nodes.

No idea if it would be overkill or not. I have not experimented with GAs (though I want to) so I have no idea.

posts said... / March 13, 2012 at 12:09 AM  

very interesting read, does anyone know where i can find a video of this to watch? i might not know a thing about programing or AI's, but i always find stuff like that facinating

Michael Robbins said... / March 13, 2012 at 9:47 AM  

@posts: It should be on the GDC Vault in about a month.

Tobias Fong C.W. said... / March 17, 2012 at 8:53 PM  

Wow, this looks amazing.

I suppose all the stuff you've learned will be applied to Supreme Commander 3 soon, right? ;)

CasualSax said... / March 21, 2012 at 10:19 PM  

Nice to see you posting again! Fascinating read as usual. Does this mean I need to start bugging you to post regularly again as well?

Michael Robbins said... / March 22, 2012 at 11:47 AM  

@CasualSax: Hopefully that wont be required.

dublindan said... / September 5, 2012 at 1:55 PM  

When was all of this added to Supcom 2? Because last time I played the game, maybe a year ago or so, the AI was shockingly bad and I ended up quitting the game over it. If its since been updated and improved, however, I'd love to give it another try.

dublindan said... / September 5, 2012 at 2:02 PM  

Ahhh nevermind! I see now that this is an AI mod :) Awesome, looks like I'm giving Supcom 2 another chance after all! Thank you.

Michael Robbins said... / September 5, 2012 at 2:29 PM  

Are you sure you are talking about SupCom 2?

dublindan said... / September 5, 2012 at 7:38 PM  

Err yeah, sorry, I was being fed conflicting information it seems. So, am I right that this was an official patch to Supcom 2? If I install the steam version - will I have this?

Michael Robbins said... / September 5, 2012 at 7:45 PM  

The AI fixes and improvements were added via a series of patches post release.

Yes, if you install the Steam version it will patch to the latest version, which will have the fixes.

Post a Comment