Seven Questions With Bungie's Max Dyckhoff

[image:78171 align=right hspace=5 vspace=5 border=0 size=thumbnail] Bungie AI programmer Max Dyckhoff last month mentioned a long-standing bug in Halo's driving code he'd found and eliminated. He was later kind enough to go into a bit more detail about how Halo's AI figures out how to handle driving and riding in vehicles.

Q: You're one of several active community members who went on to actually work at Bungie. How many Halo games have you worked on now, and what were your primary contributions to each?

A: I started at Bungie at the beginning of production on Halo 3, where I worked with Damian Isla on the AI systems. I spent some time adding a few AI features for ODST as well, such as Firefight, before moving on to Reach, where I have again been working on the AI team. For Reach I've added a bunch of great things like new flying behaviors, better vehicle driving, Elite combat, and so on. The AI team is typically very small, just 2-3 for most of Reach, but we get a lot done.

Q: You recently mentioned a revision to the AI driving routines in Reach that should provide a noticeable improvement. What was the problem that caused driver issues in previous games, and how is it being remedied?

A: There were a few things that I fixed actually, the first actually happened by chance. I was driving the Warthog myself, and I needed to back up to get out of a sticky situation. I turned while backing up, and noticed that I didn't turn in a circle while I did; instead I turned like a forklift, with the front wheels swinging around. Do it yourself in Halo 3 and you'll see what I mean.

The AI assumes that all of our turns are going to be circular; we give them a turning circle and they use this to plot various possible turns, and then they check them to see if they are going to lead them into walls or off cliffs. Because of the way the Warthog backs up (essentially in a straight line, with the front sweeping around), our "backwards" turns were always being plotted wrongly, and would often lead us off cliffs or into walls, when the AI assumed it was going to be clear.

I talked with the physics guys, and the Warthog turns like this because it's more fun for the player, which is fair enough. Visually, the rear wheels steer, but the physics says they don't do any steering. The fix was to make the rear wheels steer when the AI is driving. Various other things were done to make this not drastically affect the turning circle of the 'Hog when driven by AI, but that's basically it. Check it out in Reach; when an AI backs up, he will do so in a circle.

The second major fix was also based on the physics. If you take a Warthog and try making a hard turn with the throttle held down, your rear wheels will quickly lose traction and the vehicle will start slipping sideways across the ground. This is super fun, and the player can control it pretty intuitively, but the way the AI were modeling turns didn't accommodate enough for this. To fix this, I added some code which modulates the Warthog throttle when an AI is cornering; he'll turn into the corner, release the throttle a little and let the wheels gain traction, and then slam on the throttle again. It looks great, and has resulted in AI taking corners in a much more controlled fashion.

[image:78174 align=left hspace=5 vspace=5 border=0 size=thumbnail]Q: Marines seem to drive into things a lot. An improvement that helps them get unstuck is great, but why would a marine drive straight into a rock in the first place?

There are many reasons, the answer to the above question being one of them. The AI has a big problem to solve when driving; continuous fast movement through a very complicated space. The way the AI represents the world is through a massive mesh of "sectors" that specify where they can move. Sometimes these sectors can misrepresent a space, indicating that a rock is smaller than you might expect or something similar. We've done a lot of work to improve the pathfinding representation, and I am very happy with the end results.

Another reason would be poor understanding of how the physics works. As humans we can understand the physics of the Warthog very easily, but the AI has limited computing power to model how the Warthog is going to react to various controls. The throttling back as we go around corners is one improvement we made, and we tweaked various other ways that we model the physical behavior of the Warthog.

In my opinion, it's this sort of thing that makes Halo such an awesome game. We could just hardcode the AI into the vehicle and make them follow a path completely accurately, but instead we have all these systems which are grounded in reality, and we make them work together. The physics guys do an awesome job of simulating how a vehicle moves, and I make the AI understand how to drive the vehicle given the physical constraints.

Q: How does an AI driver in a Halo game "see" the playing field, and how does it compare to what a human player sees?

The environment in a Halo game is simplified into a "pathfinding mesh" for the purposes of AI navigation. This is a pretty standard structure used in video games, I'm sure you can go and Google it if you'd like to know more.

On top of this we have a layer of what are called "firing points"; these are essentially points in space that the AI will consider occupying when he wants to move. When an AI wants to move, he will evaluate a few hundred nearby firing points to identify the most optimal, and then he'll move there. This is identical for drivers and infantry, although flying vehicles use a separate system entirely unrelated to firing points (new in Reach!)

For the most part, the AI ignores the actual geometry of the level. When doing line of sight checks he will use it, but otherwise he sees himself as standing on the pathfinding mesh, and his target as standing somewhere else on that mesh. This allows the designer to specify areas as "non pathfindable" if they don't want an AI wandering in there.

[image:78173 align=right hspace=5 vspace=5 border=0 size=thumbnail] Q: How do AI characters decide whether to drive, gun, or ride shotgun for each vehicle type? I know I love trying to get rocket marines into a passenger hog on The Storm in Halo 3, but those guys sure do love their mongeese.

A: Hah! I believe that the rocket marines on The Storm were hard scripted by the level designer to get into the Mongeese, because that's how he wanted the encounter to play out. We very rarely do things like that, but occasionally it's a good idea.

The seat occupancy for AI characters is pretty easy to understand. If a character notices a vehicle is nearby they will "attach" to it, and then calculate their preference score for each of the available seats. Other nearby characters will do the same thing, and essentially whoever has the highest score, wins. The score includes checks like "do I have a rocket launcher?" (if so, a passenger seat is preferable), "am I a hero character (a member of Noble team, or the Arbiter in Halo 3, for example)?" (if so, give me a huge preference boost over a lowly marine). There are then a bunch of behaviors which make them get in or out of the seat based on these preference scores.

While being very simply implemented, it is actually a somewhat hard problem to get right. Essentially, we are trying to model what the player is expecting. Because there are very limited user inputs to say whom I want to go where, we need to be very intelligent and rigorous about who goes where. If the player gets out of a vehicle, should the marine passengers get out too? What if the player is just getting out to switch his weapon for a rocket launcher, something that an AI character can't possibly know he is about to do? If the player pulls an AI character out of a seat, is it because he wants that seat, or because he doesn't want me in the vehicle?

We basically had to come to a consensus on what various player actions "meant", and then wrote the code to support this. It's largely the same as the Halo 3 system with a few tweaks to make other Noble Team members more highly preferred. I'm actually very happy with the end results, and I hope all our fans are too.

Q: I think a lot of Halo players tend to list the levels focused on outdoor action including vehicles among their favorites: Silent Cartographer, Assault on the Control Room, the end of Outskirts, Metropolis, Delta Halo, The Ark, The Covenant. Halo seems to be at its best-- and its worst-- when the player is a passenger for certain sequences, especially on higher difficulties. (I'm thinking here of the outdoor theater in Metropolis where the Gauss hog wants a gunner, but on Legendary the hog gets killed very quickly.) Would you expect this to change for the better in Reach?

I have successfully been a passenger in every vehicle section in Reach on Heroic difficulty; and I believe our test team has done the same on Legendary. Various "survivability" behaviors were tweaked so that the AI driver will try and keep his player passenger alive better, and the new handling of the vehicle makes it much more robust at navigating around a combat encounter.

[image:78167 align=left hspace=5 vspace=5 border=0 size=thumbnail] Q: I once wandered away from a Warthog at the start of Delta Halo and came back to find it full of Elites who refused to get out or drive. It was hilarious-- but I'd have loved it if the Elites came after me with the hog! Do enemy AI have to be specially set up to take advantage of vehicles on the playfield, human or Covenant?

Hah! I actually saw this just the other day in a level on Reach; I got out of a Warthog to complete some objective inside a building, and I come out to find a pair of Elites getting into it. Fortunately I managed to destroy them all with a rocket launcher before they took it and started driving doughnuts around me.

The answer to your question is basically "yes". The firing points I mentioned before are actually filtered based on vehicle or infantry. We want infantry firing points inside, and on a higher resolution than vehicle points. If a designer hasn't placed any vehicle firing points in an area, then the AI will be unable to drive vehicles in that encounter. Most encounters that you can approach in a vehicle will have Covenant vehicles opposing you though, which means that they will be able to drive just fine.



Very cool article! I'm glad the AI can drive a whole lot better in Reach. :)

Good Interview! thx

Interesting read, even though I've not been following Reach's development closely. I'm glad you asked him some good nitty-gritty questions. Your comment "Marines seem to drive into things a lot" was comically diplomatic I thought. If I'd been doing the interviewing, I might've said something more like "Sir, I put it to you that the Marines are dang fools behind the wheel! Explain yourself!". But it sounds like things could be significantly better in Reach. Maybe even good enough to risk clambering aboard? Guess we'll find out soon enough. Wish I was in AI programming; I think it would be very interesting. I'm sure it's some pretty tricky stuff, so although the Marines have never been great drivers or had much sense when it comes to taking you into combat situations, I've always felt inclined to cut the programmers some slack.

On the matter of AOTCR, that's certainly my favourite level but not because of the vehicles. I can't even remember the last time I tackled it with vehicles. I like going on foot. You seem to hint at a liking for passenger play when you say "Halo seems to be at its best-- and its worst-- when the player is a passenger for certain sequences, especially on higher difficulties.". That surprised me, and in H3 I had the distinct impression that it would be less and less advisable to be a passenger, the higher the difficulty level . Not sure about ODST; I just didn't play it that much in the end. Was that any better in regard to Marine driving intelligence?

There are some encounters that I think were built around the idea of having the player man a gun instead of drive. Certainly the AI does have problems with both roles at times.

The concert bowl in the Halo 2 Metropolis level, after you emerge from the tunnel, seems built around introducing you to the Gauss cannon and its wonderful gun. However, on Legendary I find that the sheer number of snipers means that you can be left a sitting duck without a driver even if you don't miss a shot. Even if you make it through that area, the next one-- with a Wraith and waves of ghosts-- can be murder on Legendary if the driver gets hung up on an obstruction, which is very, very likely.

You can, of course, drive those sections and just contend with the AI's odd target selection, but I think the encounter was designed around gunning.

To be honest, I did not notice that ODST was much better, but there aren't really any areas there where I'd want to gun instead of drive. Uplift Reserve seems designed around the player driving, and on the highway, Buck refuses to drive so it's a moot point.

For whatever reason, the AI seems even worse at driving the Scorpion. Speed doesn't seem to be an issue, but navigating around obstructions is. I've noticed that in H3 and in ODST.

Rampant for over se7en years.

this interview was very insightful.thank you. i cant wait to be driven around by the friendly AI now! it usually always made me mad in halo2 and3