Alibaba Cloud Object Storage (OSS) with S3 Compatible throw Error?阿里云对象存储报错

Support
1

When adding Alibaba Cloud Object Storage (OSS) with S3 Compatible, an error is reported:
Alibaba oss error 阿里云对象存储报错 · Issue #4994 · duplicati/duplicati (github.com)
用阿里云对象存储报错,测试连接能正常,但是无法上传文件到oss,密钥和key都是正常使用的

2

Alibaba Support:The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint

3

Hello

I don’t know if the message you get from support matches the error message. Supports says is is an access rights problem, message hints at a missing feature in Duplicati.

If you are very adventurous and wish to help the project, I could bake up especially for you a Duplicati-next experimental that could - possibly - solve the multichunk issue (if it is really the problem as said in the error message that is, if it is an access issue as the support said, maybe explicitly setting a region or something could be the fix). Warning: experimental features are as said on the tin, experimental, and can be not included in next Duplicati if they are problematic otherwise.

4

I’m glad to receive your reply. May I know where I can obtain the Duplicati-next experimental version? I hope it can help resolve this issue, as it would be great to have all my backups in one place.

Do you have any additional workload with this experimental version? If possible, I’m willing to give it a try.

5

Uh. Now looking back at your issue, I see that you are using Docker. Unfortunately it is a pain to do experimental stuff with Docker, it’s meant to provide turnkey software.

I have pushed a zip file with the binaries and you would have to somehow import them in your container, but it seems very unwieldy and I doubt that anyone would try that. That would be unzipping the file into the container. To remove it, just rebuild the container.

Thanks if you take the time to test it, but be wary of using it for real because it contains a number of changes that are not all well tested. In the unlikely case you are actually trying it, remember that the change only concerns the minio driver, so you would have to edit your job to replace the Amazon driver by the minio one. Also remember to set the --disable-streaming-transfers option.

6

I apologize for the 404 error on the page. I will continue to monitor the page until it becomes accessible. In the meantime, I will also be careful and cautious during the testing process.

7

it’s back, sorry for the silly mistake.

8

I’m sorry to hear about your experience. I understand that you downloaded the repository I provided and tried running the exe file by double-clicking on it. Unfortunately, even after setting the --disable-streaming-transfers option, the upload still failed with the same error as before. You also tried adding the Aliyun backend service yourself, but you found it difficult to understand C# (only familiar with Java). You hope that someone kind-hearted will continue to optimize it in the future. Thank you again for your help.

9

I believe that the error message means that Alibaba does not support MultiChunkedEncoding, but we try to use it.

UseChunkEncoding is set to true by default in AWS PutObjectRequest. I am not sure about the Minio client, it might be similar to the Amazon behavior, but from what I can tell it does not have this configuration option. Maybe just setting it to false will work, but that would need to be an extra option.

@wodepig
Did you try Minio SDK in the S3 options?

If you can compile the C#, you can also try to modify S3AwsClient.cs (line 136) and use AWS SDK:

public virtual async Task AddFileStreamAsync(string bucketName, string keyName, System.IO.Stream source,
            CancellationToken cancelToken)
        {
            var objectAddRequest = new PutObjectRequest
            {
                BucketName = bucketName,
                Key = keyName,
                InputStream = source,
// Add this:
                UseChunkEncoding = false
// ------------
            };
            if (!string.IsNullOrWhiteSpace(m_storageClass))
                objectAddRequest.StorageClass = new S3StorageClass(m_storageClass);
10

good idea,i will try it on late,ths

11

I’m very happy that I can now successfully upload files to Aliyun OSS using the method you provided. It was indeed due to the UseChunkEncoding = false setting. I am unable to create an add-on through the front-end page, so I can only package it into a DLL file using Visual Studio and use the “docker cp” command to replace the file inside the container. I am still testing its issues and hope someone can submit a pull request to the code repository to fix this problem. I have written a detailed replacement tutorial, hoping it can help more people.