since a short time Wasabi offers a new endpoint in Europe (eu-central-1). Could you please include it into Duplicati’s options? If I try to use a custom server URL (s3.eu-central-1.wasabisys.com) I only get this error message:
Failed to connect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
If I only set the endpoint to eu-central-1 and keep the standard server URL (s3.wasabisys.com) Duplicati connects, but I don’t know if it really uses the european server.
Afaik, Wasabi is S3 compatible. So it shouldn’t require any changes. I’ve used rclone a lot with it. As far as I see, the Duplicati S3 already supports everything required to use Wasabi eu-central-1. If the UI doesn’t set everything required, you can always add extra parameters.
Check out: duplicati-cli help s3
Edit: Did you check that the bucket you’re trying to use is created on the right endpoint? You can’t use wrong end point which isn’t where the bucket resides.
Yes, Wasabi is S3 compatible. But why were the other endpoints integrated, although this could have also been solved with a custom S3 URL?
Besides that you don’t need to create the buckets on the European server. I can access all buckets using FileZilla Pro without any problems. I only had to configure the new URL. The first time you connect to the new server, you will be asked if you accept the certificate. There you will also find the correct server URL. Thus it is clear that you are actually connected to the European server.
Thanks for trying to help. I just tried it again. Doesn’t work for me. I only get the same error message. But it shouldn’t be a problem that the bucket was created on the standard (us-east-1) endpoint. I think I used the web interface to create the buckets at that time.
Well, enough speculation, I had to test it and check the facts.
The error message you gave, is clear indication about misconfiguration. With EU buckets you must use EU endpoint. You can’t use US endpoint with EU buckets.
I just tried this with duplicati and it all works perfectly when correctly configured. If I misconfigure the system on purpose then I’ll get the error message:
Operation List with file attempt 1 of 5 failed with message: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. => The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
With EU buckets you must use s3.eu-central-1.wasabi.com server. No need to modify the connection URL at all.
Anyway, wasabi seems to work very well and is fast in EU. Heads up Wasabi Cloud Storage Team.
This is great news, now all I need to do is figure out how to move my data from US to EU - I’m assuming Duplicati won’t care as long as I point it to the new bucket correctly after replicating the data and it will resume from where it was before.
Hi Folks - thanks for the feedback & interest in our newest storage region. Some details on using this region are provided in the article below. One of the users in this thread was asking about movement of data from US to EU. We don’t our own method doing this with Wasabi right now but we are working on it (it’s discussed a little bit in the article below). We thank the Duplicati community for bring up this topic and we welcome any questions/comments you may have. Thanks, Jim
You can always use per hour cheap vps with unlimited bandwidth and rclone or so. So you’re “re-uploading” everything, but at 1 Gbit/s (full-duplex). And that hopefully won’t take too long, unless you’ve got really a lot of data.
@wasabi-jim Having multiple buckets under single billing account with sub-accounts technically works, but the access right management (IAM) totally sucks. It’s about as user hostile as it can be. The old system was much better, but keeps nagging about using IAM all the time. - There should be totally trivial way of setting up rights, without need to study it for days and tuning with JSON. Sure, I can do that, but I don’t have any kind of interest doing it. I’ve already used about 2 hours, and it didn’t work in that time, so I gave up. And I’m pretty sure, I tried harder than 99% of users will be trying. This will only lead to buckets which are world readable / writable, because it’s much easier than doing things right with user-hostile system.
I’m afraid I concur, I tried to set up two accounts that could only access and see a single bucket at two different regions, and no matter what I did both users could still see all the buckets but only access one. Annoying as the tool I was using to browse would throw up errors all the time. The JSON is truly awful.
I have not tried to do this with Wasabi, but it certainly sucks for AWS.
I have yet to find a menu/wizard that says “this user can read but not write this specific bucket”. I managed to get it working myself, but concluded that most users would simply give up, so I put in special handling for AWS S3 that asks the user if it should set it up correctly, to avoid users just running with the root credentials.
If the IAM on Wasabi is similar to that of AWS, maybe we can add such a feature to Wasabi as well?
Duplicati just needs to:
Check if the account is root
Check if the account can create new users
Create a new user account with no privileges
Assign a policy document to the new user, granting it read/write to a particular bucket
Looking at the policy documents, they look very similar to AWS, so I think it is doable:
Hi Folks - I’m not sure of the specific question here but I can say that Wasabi does follow the AWS IAM model from the perspective of having root & sub-users and using the AWS policy constructs. For example, you can use the AWS policy generator (AWS Policy Generator) to create your policies and paste those into Wasabi’s policy editor. Our choice of this approach wasn’t because we thought that the AWS policy model was the best approach ever but more driven by the fact that we had to pick some model to use and given our embrace of the S3 API and other aspects of the AWS IAM model, we felt our users would be familiar with the AWS policy approach (even if that familiarity is of the ‘dislike’ nature). I do agree that it can be complicated to work with.
Going forward, for the region-to-region bucket migration use case (as one might do when trying to move your storage from Wasabi us-east to Wasabi eu-central), our future Wasabi Bucket Replication feature would allow you to do this:
Setup a replication job between your us-east and eu-central bucket
After the replication completes, you could delete your us-east bucket (or keep it in place if you wanted the added redundancy of storing your data in 2 different regions.
This feature will work in the same way as AWS CRR (cross-region replication) except that we don’t require you to have bucket versioning turned on and we won’t charge you for the data transfer between regions. It is our goal to have this feature available by the end of 2q19. Until then, you have to do this migration through other methods like reuploading again or using a third-party tool for the job (sorry).
For those interested, for Windows I could only get Cloudberry Explorer to properly connect to both regions with the same account and still be able to copy from one to the other. The rest either would not connect to Wasabi at all, or simply threw up meaningless errors when trying to upload the files.
So right now I’m slowly copying my data across US->EU.
BTW, Wasabi doesn’t seem to care that I don’t prepend my username to the bucket name, so I wish the GUI would stop asking.
