New Neural Networks Coming to PA

Last Friday I got the crazy idea to update the neural networks used by the AI. Why, you ask? Well, during work Thursday I watched a live stream by Alex Champandard of AIGameDev.com that went over modern neural network techniques. There were a couple of things mentioned during the live stream that caught my attention. One was that neural networks are making a comeback. The second was a new (at least to me) activation function that has gained popularity.

The first point made me laugh because not very long ago I would get weird looks and questioning faces when I mentioned using neural networks for Supreme Commander 2. The second point highlighted the fact that my already limited knowledge of neural networks may be getting out of date. So, I took last Friday off of work and spent the majority of the weekend reading research papers and fiddling with the neural networks in PA, looking to see if I could improve them.

The first step was to update my neural network class to be more robust. The class I had in place was pretty rigid and unsuitable for what I had planned. The first thing I did was add support for having more than one hidden layer. It seems like a simple change, but one that was, up until now, completely unnecessary. The research I read backs that up as well. The second thing I added was support for specifying different activation functions per layer type. Sure, I could make them per layer, but for now per layer type is fine. These two changes opened up a lot of new possibilities.

Currently the live build PA uses a trio of 3 layer multi-layer perceptrons, one each for land, fighter, and bomber/gunship platoons. The hidden and output layers of these neural networks utilize a sigmoid activation function to squash the input values it receives from the previous layer. The reason just adding multiple hidden layers isn't an immediate bonus is, as I understand it, due to the sigmoid activation function itself, or rather its derivative. To train a neural network you need to be able to calculate an error amount for each output in the neural network. PA uses the gradient descent method to achieve this. The problem this poses is as you propagate the error value further up the stack of layers this gradient gets tinier and tinier causing the layers nearer to the input layer (top) to have a harder time learning than the layers closer to the output layer (bottom).

To get around this people are using what is called a rectified linear activation function. The benefits of using a rectified linear activation function are three-fold. One, the rectified linear activation function is cheaper to calculate (as is its derivative) which allows you to have more hidden nodes. Two, it allows for sparsity in the neural network, where a portion of the nodes in each layer will be inactive for a given set of inputs. Three, because the full value neurons inputs are utilized (as long as it is a positive value), rather than squashed, it allows for effective gradient descent in the upper layers.

All in all this allows for faster training times and potentially more effective neural networks. This also allows neural networks to grow upwards instead of outwards, which is more efficient to calculate. So far, I have been quite happy with the results. Once the patch that includes them goes live, I hope you will be too. 

15 weeks at Uber

The last few months have been spent getting the many of the gameplay systems in place. The first step in that process was getting things like units and features (tress, rocks, wreckage, and the like) represented on the server and client. The client representations started as squares. Little squares for the tanks and big squares for the factories. These little squares eventually changed over to being full on models sliding around the screen and instead of just tanks and factories we had commanders, power generators, extractors, and other units.

The next thing I worked on was the economy system. My first implementation of the economy system was a TA style flow rate economy. After getting that working Jon had me change over to a new economy style. It is still a flow rate economy, but we are hoping it will be a little easier for new players to grasp while still staying true to the TA style economy fans enjoy. I was really excited to get the economy up and running because it felt like a real game for the first time.

Next on the long list of things to do was unit tasks like move, attack, and patrol. A task is not the same thing as an order, however. An order is what the player (or AI) tells a unit to do while a task is what actually makes the unit perform a function. That doesn't mean there will be a 1:1 ratio between orders and tasks. In fact, I expect there to be more than one task per order for most order types. For example, ordering a unit to build a metal extractor will start a build task. If the unit is not in range to build the extractor it will initiate a move task to get it within build range. Once the move task has finished moving the unit to within range the build task will take control again and start the build process. Patrol and assist orders can get quite a bit more complicated.

Three weeks later

I find it fitting that I am sitting down to write this on the day of Thanksgiving. Looking back over the last year I realize that I have a lot to be thankful for, not the least of which is the opportunity to work amongst such a talented group of individuals as those at Uber Entertainment. Planetary Annihilation is a very different project for me, obviously not due to the genre, but do to the fact that I have never had the opportunity to work on a project in such an early stage of development.

I have spent the last few weeks working on some really core level engine items such as creating a wrapper class for semaphore objects (to hide the multi-platform details further down), creating a thread pool object, a thread safe task queue, a condition variable wrapper; you get the idea. It has all been a fun and it has all been an interesting challenge.

The past week has mostly been spent working on replacing our resource system with something a bit more robust. This includes having support for re-loading changed items on the fly (which was fun to see the first time it worked, by the way) to make development iteration easier. I have the basics of it working and am now just working out the full implementation details, trying to see all the problems with the way it works currently and adjusting the implementation to compensate. I imagine I will spend some time this weekend fixing up the last identified flaw and start working on support for more than just textures. After that, it will probably be time for ripping out the old system and replacing it with the new one.

Uber has such a different and interesting culture than I am used to. They are far more open about the inner workings of the company and I find that refreshing. I also like that fact that I can go to the forums and see more than me interacting with the community. Posts from the forums are cited quite often in meetings and conversations.

Speaking of listening to the community, Planetary Annihilation pre-orders are now available. It seems odd to be offering them so early on in development, but that is what many of the community asked for, so Uber is delivering. If you are interested in pre-ordering Planetary Annihilation, you can do so here: http://store.uberent.com

Also, the book that I wrote an article for is now listed on Amazon and has an awesome cover: http://amzn.to/R4cllh I am fairly certain the listed publication date is incorrect, but we will see. I can't wait to get my copy.

Further down the rabbit hole

A while back I mentioned that I started doing some research into game theory to see what I could learn that would translate into game AI. That initial research led me to realize how interested I am in psychology and human behavior in general. As Dave Mark (President - Intrinsic Algorithm) is fond of saying; the behavioral model for AI is life.

More recently this rabbit hole of human behavior has led me into researching micro expressions (thanks to Netflix). Now, how is researching micro expressions going to help be a better AI programmer? At the moment, it probably wont, but I became fascinated with the subject. The fact that there are several core facial expressions that are the same for everyone, no matter their country of origin, ethnicity, or background, it is really interesting.

As a bonus, I have become more adept at noticing these expressions. It is amazing what you can pick up on. Does this mean I can always tell when people are lying? No, not at all. However, if I am in a conversation with someone (like my daughter trying to get herself out of trouble, or in a meeting with my boss) and things don't look right, I can take what is being said with a bigger grain of salt. Heck, it might even make me a better poker player, haven't had the opportunity to try that out. I wonder what topic I will dig into next.

In a little over two weeks I get to start work on Planetary Annihilation! I am super excited (if you didn't get that from the previous sentence). I imagine it will be a little while before I get to start work on the AI. There are a lot of moving parts that will need to be in place before that can happen, but I am looking forward to doing some real base level engine programming. I can't wait to get started.

Change is good

A little over 3 years ago Gas Powered Games took a chance on me and welcomed me to the GPG family. No other company would give me the time of day. Over the past 3 years GPG continued to take chances; first by having me take over the Supreme Commander 2 AI and again by allowing me to create the AI system for their current project from scratch. There aren't many companies that would give that level of responsibility to such a junior engineer.

However, all good things must come to and end.Yesterday I gave notice that as of November 1st I would cease to be an employee at Gas Powered Games. In three weeks I will no longer be working for the employer I spent 2 1/2 years of my life hounding for a job. I wish all the best for GPG and all of its employees in all of their future endeavors and I am very thankful for the opportunities I was given to grow as an engineer.

November 1st starts a new chapter in my career as I join the ranks of Uber Entertainment. I am very excited to have the opportunity to work an such an epic project as Planetary Annihilation. With the amount of talent located in Uber's office this project has huge potential.

I will continue to update my blog as development progresses on Planetary Annihilation, so keep checking back as we go for awesome!