I too am having problems with build times and the second hut.Running version .03 for Mac.They built the bath great with pop. ranging 10-12 and it being the only thing they worked on.
They started on the rubble without direction, I just noticed scaffolding there with 1% at some point, so I let them work there. When my pop. maxed out at 17, I started all builders on the hut. When I have the game open, they make progress on the hut, only occasionally wandering to the rubble and then I drag them back.
When the game is closed they make no progress on the hut or the rubble, and when I open they are wandering in a large group back to the hut from the rubble, even though it has not progressed either.
This occurs in both my nature game.
In my magic game they are going to the bath instead of the hut, because it is not finished yet. The rubble isn't started. Again, neither progressing unless I have the game open.
The bath in my nature game was able to progress while closed, I think because they only had 1 task going. That may be the issue here.