Tuesday 4 October 2011

XBox HLSL Peculiarity

While fiddling with my shadow code to try and get more reliable performance on the Xbox I changed my HLSL shader code so I could quickly make changes to a pair of nested loops that calculated the weighted average.

Most of the testing was done on the PC and it all worked fine.  When I tried it on the Xbox the shadows had completely gone!

It took a while to find but it was the simplest code possible that did not work on the Xbox but worked perfectly on the PC.




 float shadowTerm = 0.0f;
 int sampleCount = 0;
 for (float y = 0; y <= sampleRadius; y++)
 {
   for (float x = -sampleRadius; x <= sampleRadius; x++)
   {
     float sample = something();     
     shadowTerm += sample;
     sampleCount++;
   }                      
 }    
 // Average by the number of values
 shadowTerm /= sampleCount;
  


That code does NOT work on the Xbox.
Note the sampleCount immediately following the shadowTerm in the middle of the loop.
What could be simpler code.

Take it out and replace with a sum at the end after the loop and...
the following code does work on the Xbox!


  float shadowTerm = 0.0f;
  for (float y = 0; y <= sampleRadius; y++)
  {
    for (float x = -sampleRadius; x <= sampleRadius; x++)
    {
      float sample = something(); 
      shadowTerm += sample;
    } 
  } 
  // Average by the number of values
  shadowTerm /= 
        (sampleRadius + 1) * 
        ((sampleRadius *2) + 1);
  


If anyone has a reason other than the compilers are different, please let me know.

No comments: