I have a habit of keeping a notepad next to my keyboard at work and using it to take notes, write out debug info I need, or making a list of things I need to get done. Not too long ago I went through my first full pad. Instead of throwing it away I decided to hang onto it. Judging from the last couple of pages it appears I finished with the pad back in April or May, since it has references to the work I did on the Protip notification system in Age of Empires: Online.

First page of notepad

It is interesting looking back and seeing where it all started, or at least close to where it all started. The first page of the notepad has engineering tasks for Supreme Commander 2 on it. Tasks that included engineer repairing while idle, making their auto repair functionality prioritize by need and veterancy, and a bug where engineers would keep moving along their last vector when told to reclaim. There are even scribbles where I was trying to figure out why the engineers were behaving the way they were.
Land AI improvement

Seven pages in you can see where GPG decided to give me a shot at fixing the AI. I gave each AI type a page where I outlined some of the things I wanted to fix specifically for that AI. As you can see for the Land AI the research needed to be re-worked, it needed reactionary platoons, and they needed help woth platoon retreating (although, that was more of an all around issue). There are pages of notes specific to AI.

Then, of course, many more pages in is my first written down idea for the AI Team Manager. The Team Manager for the Supreme Commander 2 AI was not originally planned to even be in Supreme Commander 2. I came up with the idea while brainstorming for the Kings and Castles AI. Since Kings and Castles did not have a functioning AI at the time, I decided to implement my idea in Supreme Commander 2. It turned out so well and ended up being such a nice feature that I was allowed to release it for Supreme Commander 2.
KnC prototyping
More KnC ideas

I also wrote out other things I had planned for the Kings and Castles AI. Lots of other things that I still have yet to implement. Things like the platoon manager, the frontline manager, improvements to the AIs influence mapping system. I even wanted to increase the AIs use of neural networks.

I still hope to get to implement them at some point. Whether it is on another Supreme Commander game, a complete overhaul of the Age of Empires: Online AI, or on an entirely new IP, it doesn't matter to me.

What matters is that AI is fun, challenging, and just plain awesome.

Two weeks later

Age of Empires: Online has been out for two weeks now, but it doesn't feel like we have released at all. We are still working like we are still in production. We are working on the next civilization, the next booster, as well as working on improving features and bug fixes. The pace of work hasn't changed a bit.

AoE:O has been a huge success so far. The number of people playing the game is staggering. We had an office pool going for how many people would play in the first 24 hours after launch. Microsoft gave us their estimate and most guesses in the office were within that range. Nobody was even close.

The launch went very smoothly. There haven't been any huge game breaking bugs. There are, of course, some minor issues, but we are tackling those as quickly as possible. I wish I could share more detail about what is coming.

This project is quite different than Supreme Commander 2. Sure, we worked on it after launch, but not to this level. We are planning things and working on things you guys wont see for months. Even though the game has launched it still seems like we are in the production phase. Will take some getting used to.

Also, tomorrow is my two year anniversary at Gas Powered Games!

Getting close

In a little over a month Age of Empires: Online will be released to the world. This will mark the second shipped game to go on my resume in as many years. Many game developers spend that long, or longer, just getting one shipped game, so I feel very fortunate have had the opportunities I have had thus far.

The game is looking better and better every day. We have managed to clean up a lot of the code base which is making our tasks easier as time goes on. I haven't had much game play work to do on this project, which has been frustrating, but understandable given the nature of the project. There have been glimpses of my work on the few video blogs that are out, such as the Defense of Crete UI and the player trading UI. It is amazing the amount of work that goes into seemingly simple tasks.

Work has already started on the design and art side for the next civilization, while the programmers (such as myself) continue to work on bug fixing and new feature work. Fixing bugs in this engine has proven to be complicated and tiring, but that is probably because the engine was created by another company and is organized quite differently than the SupCom engine. It has taken quite a while to get used to.

There is a light at the end of the tunnel, for this phase at least. With a game like this there really isn't an end date. Patches will continue to be released at regular intervals as well as new civilizations and other content. This project wont be truly over for some time yet.

Game Theory

As I mentioned on Twitter (and probably Facebook), I have been studying game theory lately in the hopes of finding a use for game theory concepts in AI.I picked up a couple of books from my local library on the subject of game theory. The first book wasn't very useful at all. It would have been helpful if I had no idea what game theory was, but since I had a basic understanding of it the entire book was basically a review of things I already knew. I didn't get very far into the second book before another book I had ordered from Amazon arrived at the office.

After mentioning my interest in game theory as it relates to AI on twitter Dave Mark (@IADaveMark - President and Lead Designer, Intrinsic Algorithms) said I would find a lot of useful information. He suggested that I view his slides from his GDC talk last year and to see if I could gain access to the GDC vault to view his presentation. I was able to view the slides, but was unable to find anyone I knew with access to the GDC vault (No matter, I got the gist of it from the slides). He also suggested I grab a copy of his book. Now, I know how that looks, but seriously, his is the only book I can find that covers the topic of game theory in AI.

The book arrived last Monday and I just finished this afternoon and I have to say, I am glad someone else out there gets it. His entire book is dedicated to to creating dynamic behaviors in game AI where the AI reacts to situations and events as humanly as possible. Most, if not all, of the other AI books I have looked at simply go over the various tools you can use to get a functioning AI (decision trees, state machines, and such), but fail to mention how to get the AI to behave in a believable fashion.

All of that being said, I'm not sure I am 100% sold on some of the methodology. One of Dave's comments to me was that after reading the info I may decide that neural networks are no longer worth the trouble. And, while I do see the appeal, I would have to disagree. Dave and I are out to achieve the same thing; feed the AI with various info so that it can make an informed intelligent decision. Dave achieves this by creating a series of complex formulas designed to assign priorities to a number of choices the AI can make. The problem with this approach is that you have to create the formulas. With neural networks those associations are made for you. Once you have neural network code that works it is relatively simple to create new networks that take in available information and returns an action to take.

I am not saying my ideas on AI design are better than Dave's or even that his ideas are better than mine, in fact, a good mix of both is probably the best solution. One thing is for certain, though, I certainly have a lot more ideas about how to create even better AI systems.

For those interested, Dave Mark's book can be found here:

Back in the thick of it

Things have been very busy lately. Microsoft handed Age of Empires: Online development to us not too long ago, which meant that are current production schedule went out the window. Most of my work has still been in the UI and the backend support for the UI. Thankfully, I worked out a better work flow with Steve, so it is a lot easier on me and allows us to both be working on the UI at the same time. Previously, I would work out the initial layout, do all the supporting code, then send it to Steve. It would then come back to me to update the code for whatever changes Steve had made to my layout. It was quite frustrating. That and UI is not my favorite thing in the world, but the backend stuff is interesting.

This project has been quite interesting on the whole. I have had an opportunity to learn a lot more on this project than I have on the Supreme Commander 2 project. I think the reason for that has been because, unlike the SupCom 2 project, I am creating parts of the game from scratch and I own that part of the engine. This also means my code gets reviewed more often and more closely, which gives me the opportunity to learn. Thankfully, I haven't done anything that has made James want to throw me out a window, yet.

It looks like Microsoft is in the process of expanding the beta. Not sure if still has keys left, but if you aren't in the beta yet head on over there and sign up.

One last piece of info. Servo has made a post over on the GPG forums asking people to submit any known game exploits to him so we can compile a list. No guarantees, of course, but do it anyway. His post can be found here:

Been a while

Wow, it has been a while since I have posted anything. My apologies for that, but, honestly, there has not been a whole lot to write about until recently. Let me get you up to speed.

Shortly after my last blog entry I entered the Google AI contest. I didn't get to put a ton of time into it, but winning the contest really wasn't my goal. I had two goals going into the contest. One goal was to beat the bot created by Alex Champandard. The other goal was simply to create a bot that functioned and did relatively well. I did and up beating the bot (he placed 633, I placed 431), so goal number one complete.

As far as the second goal, my bot functioned well and placed in the top 10% of the contest, so I consider that a good showing for the relatively small amount of work I put into it. But, it was more than that. This was the first time I coded something like this from scratch. Most of the work I have done up until that point felt more like modding to me. Sure, I created the AI Overlord, but that wasn't really all that big and really not that hard at all. No, I really wanted to challenge myself to make something from, essentially, nothing. No existing code to look at. Nothing to cut and paste from. Just pure, out of my head, code. I even had a co-worker look at it for me when the contest was over to get a code review. I know, it sounds a little paranoid (for lack of a better term), but I really needed to see if I could do it.

The rest of October was spent porting code from SupCom 2 over to Kings and Castles and getting it to the point that it didn't cause the game to crash. Then came the new project. Most of November and December were spent doing lots and lots of documentation and investigation. That is why I said there really wasn't much to write about. Who wants to hear about me having to investigate code and write documentation? Heck, who wants to be the person writing all the documentation? I learned I don't like to be that guy. That was also a part of why I entered the contest. I needed a project to work on and it came at a good time.

More recently, I have started getting to code things again. For now, I have mostly been working on UI stuff, which has been interesting. At least I am getting to work on code again. I think my upcoming project assignments will put me back onto the game play side of things again, and I truly look forward to that. I can't wait to be able to share more detailed info.