No, it's not just your game. Only the mayor (and the adopted child) age. I don't adopt, I just wait until my mayor retires then the game let's me pick a new one like at the very beginning except they 'inherit' the old mayor's house, tools, clothes, etc.
But it's not just your game.