Importing In To Blender

I have purchased a few models to help make the levels a bit more interesting.  It is supposed to save me time so I don't have to create them all myself but it has taken me a while to find a good method to get the models in to Blender.

The sets of models come with loads of different file types for each model.  The most useful being Autodesk FBX, Collada DAE, Wavefront OBJ and Lightwave LWO.  The trouble is none of them are fully compatible with the Blender importers for them! 

 

 

Some of them just cause errors and will not import, some import without their textures but by far the biggest problem is that the models import as one object with multiple textures.  The latter looks OK but the moment you edit them the textures get combined on to just one of the supplied textures spoiling the look of the model!

I have a solution for most of the issues with static models but I have not got anything reliable for the armature and animations in animated models.

Getting the model in to Blender

I found the best solution for static, non-animated, models was to use the Autodesk FBX converter (v2013.2).  Take the FBX file as the source and output to an OBJ file.  The result is compatible with the OBJ importer in Blender (2.63a).

It's very easy to use, just open the source FBX shown on the left, select OBJ on the right hand side and press convert. 

Animated Models

As mentioned above I do not have a reliable solution for these.  I would welcome hearing from anyone who does.  Luckily the animated models I have purchased, so far, have always had a version of Collada compatible with Blender supplied with them.  I can fix the other problems with the imported files using the techniques that follow.

I have tested converting FBX files to Collada using the Autodesk FBX Converter mentioned above and the results have failed to open in Blender!

As far as I know OBJ files do not contain any animation information.  You can convert the mesh to OBJ but the armature and the animations are lost.

I cannot find a way to scale animations and keep them working!

Textures

Blender can find the textures more reliably if they are all in the same folder as the blend file.  The first job I do at the start is to find all the images from the originating folders and copy them in to whichever folder I have created the blend file in.

Some source files contain the paths used by the author on their machine.  That is not very helpful.  You can use a text editor to fix this but I find that converting the FBX file as described above usually fixes that problem at the same time.

Importing

No need for instructions for this bit.  Its just select import from the file menu, the import type, Wavefront (.obj) or Collada (.dae) and select the file.

A word of warning.  Animated models can take a very long while to load.  If Blender appears to hang, give it a chance.  For one model I had to wait nearly 5 minutes but it did load eventually.

Rotation

If you've used the importer on the model you should have it in Blender looking correct but it might still have some problems.  Most common is that lots of other 3D modelling editors use the Y axis for up but Blender uses Z for up.

When imported in to Blender an Object rotation might have been added so the model looks the correct way up.  Check what the rotation is in the Transform window.

Alternately, the model appears laying on it's back but the Object rotation is set to zero.

My preference is to end up with no rotation in Object mode and apply any rotation needed in Edit mode.   That is required if you want to use the model in XNA.

Blender has a simple function to sort this out but as is common with Blender it is lost within all the other features.  See the apply menu which pops up when you press Ctrl+A.

First get the model looking the right way up in Object mode.  If the model is laying on its back usually all that is needed is to manualy type in 90 in to the X axis field of the rotation transformation.

Once the model appears the correct way up and if it has a transformation you can apply that transformation to remove the Object mode settings and apply that to the vertices and armature. (Ctrl+A)

In Object mode select the model that needs rotating.  It the model has an armature also select that at the same time.  I usually select the mesh first then the armature.  I don't know if it makes any difference.

Make sure the cursor (or whatever point is used for rotating and scaling round) is at exactly the same point as the origin of your model. 

Move the model and the cursor to zero first or probably better still use the pivot centre that rotates and scales round the model.

With the rotation point in the correct position, from the Object menu, Apply, Rotation.

The model will stay the same but the transform settings will change to nothing.

If the model is animated, now is a good time to check that the actions still look correct. 


Scale and Location

I would also recommend removing any scale and location before you start using the model.  This is Just so you have a clean slate when you start.  It is almost essential to do this if you want to use the model with XNA.

In object mode you can just edit the fields to a scale of one and location of zero.  There are similar methods to that described for rotation to apply location and scale. 

If you want the model to stay still and just the origin move apply the location (Ctrl+A L).  I find that useful for laying out levels.  If you want the whole model to move to zero then manually edit the location.

I have had mixed results with scale but mainly because the results can be dramatically affected by the rotation point.  Make sure that you know where whatever rotation point you are using is.  That rotation point affects what the model will look like after applying the scale.  If all three scales are already the same yiou can just set them all to one and adjust the size of the model in Edit mode.

Separate The Textures

Although at this stage the model should look correct, if you were to try to manipulate it in Edit mode you would find that the UV mapping would get messed up.  For some reason unknown to me the model is imported as a single object with multiple textures.  I have no idea how to deal with a model like that in Blender.

I find the simplest solution is to split the model by its component textures before it gets muddled up.

It must be a common thing because Blender has a simple method to do this called Separate.

Select the model, go to EDIT mode and press P to separate.  From the drop down list on the popup menu select 'By Material'.  Job done.  Now you have one object per texture.  Much easier to keep the model looking as it should.

That's it.  A few steps to get there but the model is in a usable form within Blender.

Texture Names

I'm doing more and more of my level design within Blender.  The process is much quicker.  I can import nearly any model type and line it up as I want then export to a single FBX file for use in the game's pipeline.

I slightly changed my XNA code to sort models at the mesh level rather than the model level so I can load FBX files containing multiple models and multiple meshes.  So far it works well.

In the process of doing this I notices that most models use a suffix to the texture name to indicate if the image is the diffuse texture or a normal map or a specular map etc.  I had been using a prefix.  I don't know if it will be useful in the future but I decided to do the same as everyone else before I had gone too far.

When renaming the UV map in Blender you need to be aware that the name of the texture on disk and internal name used by Blender are not necessarily the same.

 

When you replace the image you can see the old name used which will not have been renamed.

To ensure that the name used in the FBX export and the name on disk and the name within Blender are all the same I do the following:

• Open the blend file before changing anything.

• Rename the files on disk.

• Rename the UV maps in Blender.

You can just type over the old name with the new name and hit return.
The final crucial bit is to...

• replace the file used in Blender with the renamed one.

I use the same file name as the name shown in Blender.

In my opinion having the same name throughout saves a lot of confusion when transfering the exported FBX file and the textures for use in the XNA pipeline.

Balancing Act

Over the last couple of weeks I've been designing and modelling a set of 3D structures to use as the alien interiors.  I wanted them to be consistent, sci-fi looking but also slightly alien rather than appearing like they were something a human would construct.  I'm getting there but my inspiration is limited somewhat by my technical and artistic ability.

Being consistent they can all share a very few number of textures.  This has the great advantage that if they are created as one model it only needs one draw call to the graphics card to display them.  The Xbox 360 can handle huge numbers of triangles but gets much slower if it needs to make loads of draw calls.

I got a bit too carried away trying to get the minimum number of draw calls...

After having returned from the Olympic 3 Day Eventing final on Tuesday I started work on laying out another section of what will become the first level of the game.

Olympics
My wife is the horsey person which is why when the chance to get tickets to the 3 Day Eventing came up we took it.  It was a good day with a fantastic atmosphere.  Team GB got a silver medal.  The stands shook with excitement.  I was very impressed with how the whole thing was organised.  There were lots of people but still everything flowed, we quickly passed through security in to the grounds of Greenwish Park and to our seats. 

The view from the stands was perfect and even with the few minutes of rain we had from time to time the brollies did not block the view.

My wife has some pictures and more info. on her blog:
http://tomandhenry.blogspot.co.uk/2012/08/greenwich-park-equestrian-eventing.html

I use Blender to create sections of the level and then import them in to the game and move them in to their final position using my own editor.
http://blog.diabolicalgame.co.uk/search/label/Editor


I've had to redo a lot of the work I did in Blender on Tuesday night.  Perhaps the excitement of the day got to me!

I had joined all the models together too early on and moved vertices in Blender to layout a large sections of the level.  I did this with the intention of making the draw calls more efficient but that was a mistake.  The single model is now too complex to separate and adjust sections and I had forgotten to allow for shadowing!

The Diabolical engine does not self-shadow.  This is where the triangles of a single model cast shadows on to other triangles in the same model.  I have other posts on the unsightly effects caused by self-shadowing so I won't repeat that here:
http://blog.diabolicalgame.co.uk/search/label/Shadows

3D Modelling for the Best Game Performance
Keep all models as separate component Objects so they can easily be picked up and moved about while laying out.  Combine them [Ctrl-J in Blender Object mode] at the last minute in to a smaller number of meshes that use the same texture to reduce the number of draw calls required by the game engine.

Remember to keep a copy of the source model file before the Objects are merged to make future adjustments easier.

3D Model Meshes
There is no advantage combining Objects that use different textures.  Each material will automaticaly separate to another mesh in the exported file because typically shader effect files only use one diffuse texture.

3D Modelling for Shadows
In the current Diabolical engine there is no self shadowing and this may apply to other engines.  When combining the Objects look at which Objects might cast shadows on other objects and make them separate models.

For my game engine I need to balance the performance gain from having less draw calls for the Xbox 360 against the visual quality of having shadows cast correctly on to more surfaces.

Performance Solved

I have just had the oddest result by fixing one problem I have solved another.  As my wife just pointed out normally it works the other way round.  I normally solve one problem and create two other problems!  Not this time. :-)

To monitor performance while I play the game I use some graphs to avoid creating garbage.  These I display at the sides of the screen to show me memory allocations and CPU usage.  The memory bar goes up and eventually I can see garbage being collected with the momentary slow down on the Xbox.

On the other side I have a performance monitor that measures the time that hardware thread one Update and Draw methods take to complete and a third bar that measures the time taken for the Update that runs on hardware thread four.  These are my primary methods.  The graph is in milliseconds (ms) up to 17 and the bar changes colour to red when any result exceeds 16ms (one frame). 

For a long time I have known that looking in some directions causes a noticeable slowdown and the third bar shows red and off the top of the chart!  Visually the screen would stutter because the position of the view was not being updated quickly enough!  It was probably as annoying as the glitch caused by garbage collection.

As I had run out of things to try to get rid of my remaining garbage I decided it was time that I needed to deal with the performance related hiccup to make it feel nicer to play.

After a bit of fiddling I decided I could not guess where the problem was I needed some way to measure it.  I created myself a simple timer.

///   /// A static timer which displays the duration of a method in the trace output.  /// This is NOT thread safe and cannot be nested.  ///   public class MethodTimer  {  public static System.Diagnostics.Stopwatch Clock = new Stopwatch();  public static TimeSpan StartTime = TimeSpan.Zero;    ///   /// Mark the starting time to measure from.  /// This also starts the timer if it is not already running.  ///   [Conditional("DEBUG")]  public static void Start()  {  if (!Clock.IsRunning)  {  Clock.Start();  }  StartTime = Clock.Elapsed;  }    ///   /// Output to the trace window if the duration measured is greater than that specified.  ///   [Conditional("DEBUG")]  public static void Stop(string logName, TimeSpan logDurationOver)  {  if (Clock.IsRunning)  {  TimeSpan duration = Clock.Elapsed;  duration -= StartTime;  if (duration > logDurationOver)  {  System.Diagnostics.Debug.WriteLine("Method: " + logName + " took " + duration.TotalMilliseconds.ToString() + " milliseconds!");  }  return;  }  System.Diagnostics.Debug.WriteLine("The clock was not running when the Stop method was called!");  }    }

I put it either side of a method and get it to log a result if the time that method takes is longer than I expect.

I was surprised how quickly I got it down to one method causing my problem.  I won't go in to detail about the specific method.  It just simply attempted to do far to much every frame to get more accuracy than I needed.  I produced a much quicker but less accurate version.

Now for the great news.  When I tested that on the Xbox not only had my performance problem completely gone but so had ALL of my memory allocations, no garbage collection!  Two in one.

It's not entirely true that all my memory allocations have gone, but as long as you don't make any new sounds (XACT) or respawn there are none and the tiny allocations that do happen from those methods take a very long time before they would trigger the garbage collection :-)

==

Downloads:
'Performance Tools' including the graphs mentioned above:
http://code.google.com/p/xna-game-menu/downloads/list

Update: 27 July 2012
I have just come across an article written a few years ago  suggesting the same method as above:
http://blogs.msdn.com/b/shawnhar/archive/2009/07/07/profiling-with-stopwatch.aspx

Memory Allocations

A couple of weeks ago I mentioned that I had managed to work out how to run the CLR Profiler with XNA 4.  I now understand enough to work out what the results mean.  At least enough to track back to the parts of my code that are allocating memory during the game and therefore creating garbage to be cleaned up!

I have not finished getting rid of all the allocations yet but I was able to run through some testing on the Xbox 360 without the Garbage Collector triggering too frequently.

I found the CLR Profiler daunting at first but once you know which bits to ignore it's not so bad.

For my purposes all I need is the Allocation Graph button.

There's a tiny bit to know before you'll get anything at all and that is to make sure the 'Allocations' Profile: is ticked before unticking 'Profiling active'.  The 'Calls' profile collects a lot more data and as far as I can tell does not help with finding memory allocations.

At the point in the game where you want to start logging from simply Alt-Tab out of your game and in to the CLR Profiler window to tick the 'Profiling active' box.  Then Alt-Tab back to the game.  If you do it the other way round and try and tick the Profile: Allocations after starting the application then nothing is logged!

You could log everything from the start of the game but I found the graphs just far too confusing to read if I tried to do that.

When you have played enough of the game to get some data, skip back to the CLR Profiler window and 'Kill Application'.

For my purposes I can now ignore everything except the 'Allocation Graph'.  Open that up and skip to the RIGHT hand end.  This is the biggest tip I can give.  You work backwards from the right hand end towards the left hand end.

Start at the largest allocations at the top right.

Sometimes you can easily follow the lines but when they get lost in the others, either increase the scale or just click on any one of the boxes in the line and the lines change to a cross hatching either side to make them easier to follow.  You can also double click a method to see the connecting methods either side.
Just work backwards from method to method until you recognise a method in your code.

So far I have found the changes necessary to avoid the garbage have been quite minor.  Most of my own allocations have been caused by List<T>.AddRange(...) where <T> is one of my own classes.  This is easy to fix by using a loop instead of the built in AddRange().

I have had one where it was caused by an internal XNA framework method but luckily I did not need to be doing that during the game, so I simply paused its update during game and restarted whenever the menus open.

I'm now left with a few tiny allocations that I have to go through one at a time to eliminate them.  Then I can enjoy testing again on the Xbox 360 to have a garbage collection free game, I hope.

==

EDIT:
Just came across a tutorial that explains how to use the timeline to only look at allocations over a given period.  This saves having to activate the profiling mid game:
http://spacedjase.com/post/2010/07/02/How-to-eliminate-frame-by-frame-Garbage-Generation-using-CLR-Profiler.aspx

Vertex Information Missing

When loading my new character I have been getting intermittent errors or warnings.  The model works fine in game but the pipeline either throws an exception or puts up a warning about not all vertices having UV mapping information.

I had been working on the assumption that I have a vertex missing from the UV map.  I did some tests in Blender but I could not get any test model to have un-mapped faces or vertices.  However after some hunting through the text of the FBX file I eventually found a PolygonVertexIndex group which was formed of only two points.

Faces need at least three points because Blender uses either quads or triangles.  So two points must be wrong.

The clever ones among you will immediately think that this is an edge.  Well done you'd be right.  Edges cannot be UV mapped on their own and therefore they cause the warning when loading the file!

Finding Orphan Vertices or Edges

Now the tricky bit!  How do I find one lone edge out of over 8000 edges!

With a bit of trial and error I was able to select all (A) the faces in face select mode and hide (H) all the faces.

I swapped to edge select mode in order to find the edge.  I had to select all (A) several times before I could even see the tiny speck of orange flash on and off.

Once found I added it to its own vertex group so I could easily select it again.  I can now delete this edge and export the FBX again.


==

I've seen reference to a Clean Meshes tool with a Fix Edges option which does the above but I cannot find it for Blender 2.5.  If anyone knows if its available please post a comment.  Thanks

==

An orphan edge or vertex is one that does not form part of a face.

Fixed Some 3D Models

I have confirmed that the odd lighting problem I was having was mainly caused by my modelling skills not my coding skills!

I fixed several models.  It was as easy as applying the Blender 'Edge Split' modifier.  In most cases using the default 30 degree setting but for one model I used 50 degrees to get rounded corners with the minimum of faces.

The 'Edge Split' modifier duplicates vertices where faces join so that they do not share vertices.  This is useful because when rendering, where vertices are shared, the normals are interpolated based on the adjoining vertices.  That makes corners look rounded rather than square.  With separate vertices for each face the corner looks square.