My main suggestion about destroying the idol is to keep your game on the normal or slow speed. I was trying to do it on the fast speed and I couldn't control the heathens. They were coming out too fast and rebuilding too fast and too often. Once I changed it to the normal speed I was able to get my builders to complete the task at hand.