Three weeks later

I find it fitting that I am sitting down to write this on the day of Thanksgiving. Looking back over the last year I realize that I have a lot to be thankful for, not the least of which is the opportunity to work amongst such a talented group of individuals as those at Uber Entertainment. Planetary Annihilation is a very different project for me, obviously not due to the genre, but do to the fact that I have never had the opportunity to work on a project in such an early stage of development.

I have spent the last few weeks working on some really core level engine items such as creating a wrapper class for semaphore objects (to hide the multi-platform details further down), creating a thread pool object, a thread safe task queue, a condition variable wrapper; you get the idea. It has all been a fun and it has all been an interesting challenge.

The past week has mostly been spent working on replacing our resource system with something a bit more robust. This includes having support for re-loading changed items on the fly (which was fun to see the first time it worked, by the way) to make development iteration easier. I have the basics of it working and am now just working out the full implementation details, trying to see all the problems with the way it works currently and adjusting the implementation to compensate. I imagine I will spend some time this weekend fixing up the last identified flaw and start working on support for more than just textures. After that, it will probably be time for ripping out the old system and replacing it with the new one.

Uber has such a different and interesting culture than I am used to. They are far more open about the inner workings of the company and I find that refreshing. I also like that fact that I can go to the forums and see more than me interacting with the community. Posts from the forums are cited quite often in meetings and conversations.

Speaking of listening to the community, Planetary Annihilation pre-orders are now available. It seems odd to be offering them so early on in development, but that is what many of the community asked for, so Uber is delivering. If you are interested in pre-ordering Planetary Annihilation, you can do so here: http://store.uberent.com

Also, the book that I wrote an article for is now listed on Amazon and has an awesome cover: http://amzn.to/R4cllh I am fairly certain the listed publication date is incorrect, but we will see. I can't wait to get my copy.

Further down the rabbit hole

A while back I mentioned that I started doing some research into game theory to see what I could learn that would translate into game AI. That initial research led me to realize how interested I am in psychology and human behavior in general. As Dave Mark (President - Intrinsic Algorithm) is fond of saying; the behavioral model for AI is life.

More recently this rabbit hole of human behavior has led me into researching micro expressions (thanks to Netflix). Now, how is researching micro expressions going to help be a better AI programmer? At the moment, it probably wont, but I became fascinated with the subject. The fact that there are several core facial expressions that are the same for everyone, no matter their country of origin, ethnicity, or background, it is really interesting.

As a bonus, I have become more adept at noticing these expressions. It is amazing what you can pick up on. Does this mean I can always tell when people are lying? No, not at all. However, if I am in a conversation with someone (like my daughter trying to get herself out of trouble, or in a meeting with my boss) and things don't look right, I can take what is being said with a bigger grain of salt. Heck, it might even make me a better poker player, haven't had the opportunity to try that out. I wonder what topic I will dig into next.

In a little over two weeks I get to start work on Planetary Annihilation! I am super excited (if you didn't get that from the previous sentence). I imagine it will be a little while before I get to start work on the AI. There are a lot of moving parts that will need to be in place before that can happen, but I am looking forward to doing some real base level engine programming. I can't wait to get started.

Change is good

A little over 3 years ago Gas Powered Games took a chance on me and welcomed me to the GPG family. No other company would give me the time of day. Over the past 3 years GPG continued to take chances; first by having me take over the Supreme Commander 2 AI and again by allowing me to create the AI system for their current project from scratch. There aren't many companies that would give that level of responsibility to such a junior engineer.

However, all good things must come to and end.Yesterday I gave notice that as of November 1st I would cease to be an employee at Gas Powered Games. In three weeks I will no longer be working for the employer I spent 2 1/2 years of my life hounding for a job. I wish all the best for GPG and all of its employees in all of their future endeavors and I am very thankful for the opportunities I was given to grow as an engineer.

November 1st starts a new chapter in my career as I join the ranks of Uber Entertainment. I am very excited to have the opportunity to work an such an epic project as Planetary Annihilation. With the amount of talent located in Uber's office this project has huge potential.

I will continue to update my blog as development progresses on Planetary Annihilation, so keep checking back as we go for awesome!

$2.33 million

     The Kickstarter for Planetary Annihilation ended Friday September 14th and received $2.33 million of support between the Kickstarter and Paypal donations. The guys at Uber were kind enough to invite me to celebrate with them at their office while they live-streamed the last couple of hours before the Kickstarter ended, and they know how to throw a party. I can't wait for alpha access to start. My congratulations to the Uber crew on a very successful Kickstarter.
     I just submitted what I hope is the last revision of my article for the next AI Game Programming Wisdom book. It has been a far more involved process than I thought it would be. I can't imagine having to write a whole book. Of course, I didn't know I would be this busy when I agreed to write the article. Between work and home life I am amazed it got written at all.
     Work is still super busy. I currently get to spend half of my dev time working on AI related things while the other half is usually spent working on UI. This ratio is changing bit by bit to be more AI focused. It is interesting to see the AI behavior change and evolve over time from being non-existent to being able to make more sensible decisions. A couple of more sprints and it should be in good shape.

Been too long

Wow, I can't believe it has been almost 6 months since my last post. I really need to start keeping this more up to date.

So, where to begin. Back in May I moved off of the Age of Empires: Online project onto a new, still unannounced, project. I am heading up the AI development on this new project, so that is really exciting. This is going to be the first time I have had to write an AI system from the ground up, and it is coming along really nicely. The AI is already able to play the game, just not very intelligently. That is changing little by little.I can't wait for this game to finally be announced so I can share more details.

I have also spent the past couple of months working on an article for the next AI Game Programming Wisdom book. Man, has that been an experience. I thought, hey, I wrote a big long paper to prepare for my GDC presentation, ho hard can it be to turn that into an article for a book? Not as easy as I thought. Fortunately, the section editors were awesome and gave me some great feedback and support to get this article done. I am super excited to get my copy of the book.

In other news, if you have been living under a rock somewhere you may not have heard about the Kickstarter some friends of mine over at Uber Entertainment have going for game called Planetary Annihilation. At the moment they have already met their goal, but they have some pretty interesting stretch goals revealed and I have it on good authority that they have a couple even more interesting ones on the way.

If you have not checked out their Kickstarter page, give it a look: http://kck.st/SoF9jf. If you have already backed it, as I have, awesome! I am really excited to see this game released.

I will try to keep this blog more up to date in the future, regardless of how busy things are.

GDC 2012

I am finally back from GDC 2012 and I feel exhausted. Exhausted in a way that only GDC's unique blend of information overload, loud parties, late night drinking, and presentation stress can achieve. But, I would do it again in a heartbeat if given the chance.

I got a lot of personal feedback from my talk and it was all positive ranging from "it was quite informative" to "you kicked ass". My talk was even brought up by people several times during the AI Roundtable sessions to help make their point. That was a surreal experience.

My GDC experience started Monday, where I got a chance to watch all of the AI Summit speakers give their talks. This gave me an opportunity to get a feel for how things worked and how people gave their presentations. Using this information, as well suggestions given to me by Dave Mark and a couple of co-workers, I spent Monday night adjusting my slide deck. Time well spent.

Tuesday was much of the same, except that in the back of my mind I knew I would be on the stage later that day. Oddly enough, though, I wasn't nearly as nervous as I thought I would be. It wasn't until Chris Jurney (one of the two other people my presentation was grouped with) had finished his presentation on Double Fine Happy Action Theater that I had a fleeting bout of nerves, but I knew that if I just got up and started talking I would get some momentum and be fine.

The four neural networks
My talk started out by going over the the basics like what we using the neural networks for, what a neural network is, and so on. This included a video of the neural networks in action followed by me going over what decisions in the video were actually controlled by the neural networks.

How data is fed into the neural net
The second section of my presentation dealt with how platoons actually used the neural networks. I described the various inputs we gathered and how they were converted to deltas to be fed into the neural network. I also described how the resulting outputs from the neural network corresponded to actions the platoon could take. I also told the audience what an ACU was and its importance, which I mentioned would come into play later.

Next, I went over how the neural network was trained. First, I went over the benefits of using a neural network in regards to training. I then went on to describe how the neural network learns what actions to take given various circumstances, which lead us to the fitness function.
My favorite slide

The next section of my talk dealt with describing what a fitness function is and how it is used to train a neural network. I then continued to describe the learning process, which included my favorite slide of the presentation. Finally, I described how long training took and what actions could cause you to have to retrain your neural networks.

Next came debugging. I asked the question: how do you know if your neural networks are working? The answer; visual debugging. I showed a short clip of the game running in debug mode with platoon debugging turned on. I explained that this allowed me to see what the platoon was doing and why. I also explained that this was very important in determining whether or not odd unit behavior is being caused by behavior code or by bad neural network output. Or, it could be caused by something far more sinister: good output.

Its not optimal
I then referred the audience back to my comments about the ACU being a very important unit in Supreme Commander 2. How destroying your enemy's ACU would win you the game. I then let them know that I left something out; what happens when the ACU dies. I then described how the ACU goes up in a nuclear explosion when it dies and how this played hell with the neural network.

Log output
So, I described to the audience my agony over this situation. I described how the AI would send massive platoons of units at a player, only to run away when the ACU showed itself. Then, I described my epiphany: don't try to fix the behavior in a traditional sense, try to teach the neural network the right thing to do instead. I told the audience that changing over to this mindset helped me come up with a solution. I described how I modified the fitness function to teach neural networks that killing the ACU was a good thing, no matter the cost. I went on to describe the beneficial behavior that resulted as I started a final video. I reached a preset point in my description where I said "as you can see" when an ACU explosion filled the screen and I finished with "they have gotten quite good at it."

For the final part of my presentation I left the audience with a final thought that boils down to this: Using neural networks where they make sense can have awesome results.

For anyone interested, the full PDF of my slide presentation can be found here: Presentation PDF

It wont be the same as being there, but I can't help that unless you have GDC Vault access.

If you look at the slides bear in mind two things:
1) Each bullet point was brought up individually as I talked, not all at once.
2) The pictures on slides 4, 20, and 27 are actually videos and the text on 4 appeared after the video.

Coming up for air

Finally, I get some time to write on my blog again. I have been busy with family medical issues and constant Age of Empires: Online deadlines since October. Things on the home front are finally calming down, which gives me some free time for this.

Thankfully, (depending on how you look at it) there really hasn't been much to post about regarding Age of Empires: Online. I think the only thing that has happened is Skirmish Hall has been released. That was an interesting feature to work on. Lots of UI work and lots of figuring out how to get data from the UI to the scripting system. It also gave us in engineering a chance to really hammer out how we are going to do UI from now on.

Things are getting far more interesting lately. I am currently working on a new feature for Age that trumps anything feature for this game I have worked on previously. It is huge. Another engineer and I spent 3 days just going over what data we needed to store, where it needed to be stored, and what needed to be sent to the client. It will be really cool when it is all finished.

GDC prep has also been keeping me busy lately. If you haven't already heard, I am going to be speaking during the AI Summit and GDC 2012. Details can be found here: Off the Beaten Path: Non-Traditional Uses of AI. If you are going to be at GDC this year and happen to see, stop and say hi.

More cool stuff coming down the pipe. I hope to have more interesting things to write about here soon.