When I leave the game running in the background on my iPhone, I sometimes come back to find that the construction my villagers were working on has reduced percentage. Am I doing something wrong? Or does that happen to other people?

What specific construction are you referring to? In VV4, there is a project that "counts" its' progress backwards - which could be what you are seeing.

Well I understand that the stream goes backwards and gets reclogged with debris, but it seems like every time I go to do something else, my villagers abandon their work and let debris fill up. And the other construction projects stop too.

They do tend to wander off if they aren't trainee level.

Likewise, if they are Masters and if you have a lot of construction projects going at once, they will sometimes go from one project to another.

That's not it because they are still on the project when I get back, but nothing has advanced.

Are you pausing the game when you go do something else?

No. I leave it going.

How long would you say that you are leaving the game to do other things?

When leaving the game, are you pressing the home button to access another app, or are you turning the device off?

I'm probably away for anywhere from a few minutes to a few hours.
And it can be either one.