I've had exactly this issue. I was working remotely a couple of years ago and the project manager couldn't effectively communicate (we were all remote in different parts of the US).

The result was a disaster and the project manager playing the blame game. I eventually left.