Part of the reason my game takes so long is that I keep having to change it. Often because performance forces me to find a better way but sometimes just because I had a better idea.
I am changing my rendering loop because now that I have chunks of terrain instead of one big terrain I can use that to store other information as a way to cull which models are in view quicker than my current method.
It's not a big change to add a list of structures in each chunk but it's still a few hours work. The advantage will be that the terrain will be more scalable.
I also have some ideas for more accurate collision but I am forcing myself to ignore those until I have some finished models in a nice scene to show off what I have done so far!