Which countries did you have in mind? The countries with the highest population decline are not the countries with the lowest birth rate. People emigrate from the former, and choose not to reproduce in the latter.
The places with above-replacement fertility are seeing theirs decline too.
We convert them to childlessness faster than they can replace the children we're not having. Less than 25 years away from universal below-replacement fertility.
Immigration is zero sum and soon everyone will be competing for the same pool of people.