I have been unable to reproduce your problem. I’ve done a full direct restore from backup files a few times without issue.
I am still not sure what is different about your particular setup, but it definitely seems unreliable. Are you still using the same destination target and protocol as before?