There are two schools of thought with navigation for bots within games. One is that the level designer or artist should mark the the usable areas and cover used by the computer controlled characters. The other method is for those items to be calculated by the computer.
As I am both the coder and the level designer one way or another I had to do some coding. Rather than programme methods in to the editor to let me manually lay out the mesh I decided to get the computer to calculate the navigation details.
A navigation mesh is a connected set of closed shapes representing the area of the map that can be navigated by the non-player characters. There are loads of articles and presentations on the Internet explaining the advantages of this type of design over others.
I have spent the last month, evenings and weekends, working on my version. My first attempts were to use edge detection and I came up with a very nice outline of my map but I was unable to come up with a satisfactory solution for turning that outline in to the closed convex shapes needed for pathfinding.
My eventual solution was to fill the map with the largest rectangles I could. They get smaller as necessary to fill all the areas of the level that a character can move to.
I call the rectangles rooms but as you can see from the picture they are not rooms as we would envisage them. Just open spaces within the level. Any edge of a rectangular room that touches another rectangle I call a doorway. Again not a real doorway but a space that a bot can move between to get from one rectangle to another.
I am pleased with the way this fits round obstacles while still letting bots pass through quite narrow gaps.
This is all calculated by the editor at design time. The grid size used is 4x wider and 4x taller than the in-game grid. This gives 16x more precision than the run time terrain grid. Not all of this information is needed for pathfinding. Only the room numbers and the doorway locations need to be saved and then loaded and used in-game at run time for pathfinding.
The paths calculate relatively quickly. With my previous pathfinding solution I used a grid based A-star (A*) method. On my test map this was slow mainly because the level would have over 16 thousand nodes. The new solution on my small test area has only 64 nodes and I expect a full map to have less than 200 nodes. A factor of about 100 smaller. In addition the new navigation mesh doorways are more accurately positioned than a simple fixed size grid. The A-star algorithm is nearly identical but is working on a much small sample set and it starts with only open nodes. On my development PC the path calculation appears instant.
The information I am now storing lets me include ceiling heights and path widths. The new methods prevent large entities trying to move through gaps that are too small.
The last feature I added was to pre-calculate cover points. There are two types but I do not differentiate between them at run time. Cover that can be hidden behind and shot over and cover next to a corner that can be hidden behind.
The small orange squares shown in the pictures indicate where a bot might possibly find cover. This is not guaranteed cover because the target will be moving and the bots size is unknown at design time. At run time the artificial intelligence (AI) will try each suggested cover point in order and check if it provides cover for the size of entity trying to hide and if that spot enables the ability to shoot over the cover at the target. Only a suitable spot will be selected by the AI for the bot.
My next task is to write the AI that will use the paths. I already have a state machine AI solution but I found it is not flexible enough for my expectations. I am now looking to write a goal based AI solution. We'll have to wait and see how I get on with that.