SIA (blockchain) storage network

@kenkendk, now that the redundancy issue has apparently been addressed (per this reddit post), is it possible we could get a new canary build sometime soon to test? There doesn’t seem to be any further activity either in this thread or in the github thread regarding this issue, and I’d love to continue testing with a more usable build.

It is up now: Releases · duplicati/duplicati · GitHub

1 Like

what do you fill in for the username and password fields? I do not see anywhere to set those in sia-ui

what fields do you mean exactly? i don’t have “username”, but if you mean “host”, then you should probably put “localhost”, and for “server password”, you should put your sia seed (as far as i know - i don’t believe it’s wallet password)

edit: actually, it looks like the redundancy stuff is working differently now… i’ll keep testing.

FYI, there seems to be a bug somewhere - maybe my fault or something else, but it’s needing me to run a repair currently – since, even though i ran a test backup successfully earlier this afternoon, Duplicati now claims there are “11 files missing from remote storage”… :confused:

That means that the file listing from Sia is not returning all the expected filenames. Is it intermittent or does it keep saying the same number of missing files?

I attempted to run “Repair” as it suggested and it immediately failed; then I ran a “Recreate”, which took hours, seemed to finish successfully (or at least relatively successfully) in the end, except i still get “6 missing files” errors when trying to run a backup OR a regular repair. It’s a bit frustrating that there seems to be no way to repair the existing data without wiping it out and starting over :frowning:

The following was the result of the recreate operation (sorry for lack of linebreaks, it seems to copy that way from the output window):

Summary

MainOperation: Repair RecreateDatabaseResults: MainOperation: Repair ParsedResult: Success EndTime: 2017-09-15 3:43:15 BeginTime: 2017-09-15 1:44:59 Duration: 01:58:16.3093090 BackendStatistics: RemoteCalls: 107 BytesUploaded: 0 BytesDownloaded: 21658466 FilesUploaded: 0 FilesDownloaded: 106 FilesDeleted: 0 FoldersCreated: 0 RetryAttempts: 0 UnknownFileSize: 0 UnknownFileCount: 0 KnownFileCount: 0 KnownFileSize: 0 LastBackupDate: 0001-01-01 0:00:00 BackupListCount: 0 TotalQuotaSpace: 0 FreeQuotaSpace: 0 AssignedQuotaSpace: 0 ParsedResult: Success ParsedResult: Error EndTime: 2017-09-15 3:43:16 BeginTime: 2017-09-15 1:44:59 Duration: 01:58:16.4302504 Messages: [ Rebuild database started, downloading 4 filelists, Filelists restored, downloading 102 index files, Recreate completed, verifying the database consistency, Recreate completed, and consistency checks completed, marking database as complete ] Warnings: Errors: [ Remote file referenced as duplicati-b8425e68d1f364d1aad51408dd2e75847.dblock.zip.aes, but not found in list, registering a missing remote file, Remote file referenced as duplicati-b1c9d05f9784c463cbcff169b130d181f.dblock.zip.aes, but not found in list, registering a missing remote file, Remote file referenced as duplicati-b0f7bff76f85d4ca4afc3aaa85955d490.dblock.zip.aes, but not found in list, registering a missing remote file, Remote file referenced as duplicati-b478e8d9b93ba4387b533abdac243c5d4.dblock.zip.aes, but not found in list, registering a missing remote file, Remote file referenced as duplicati-bc28d75edca594220aff0220e84c52f53.dblock.zip.aes, but not found in list, registering a missing remote file, … ]

On a completely unrelated note: the SIA “server password” setting seems to be completely irrelevant for some reason - I tried my SIA Seed at first, and that worked… then I tried my wallet password, which also worked… and now i’ve set it to “1234” and it’s still working. (My password isn’t “1234”.)

My duplicati screen looks different than yours. It is similar in that it has the folder path but I have a username and password field. I believe I am using the latest off git.

Can I get a screenshot of the screen you’re talking about? FYI i’m just using the latest canary build.

Edit: ok, so this is weird… I’m running the canary version on both my home PC as well as my work laptop, but the SIA config screen is different on both.

This is the screen i get when I try it on my work laptop:

Screenshot

Note: the Sia option is in a sub-group called “Others” in this instance.

Now this is the screen I get on my home PC install (this one seems more correct to me):

Screenshot

Note: in this version, the Sia option is in the “Proprietary” sub-group, near the bottom…

Both of these show what I get if i click “Add Backup” to add a new one. I would note that the version on my home PC is the one that I had trouble updating yesterday and needed me to reboot my PC a few times, if that could make any possible difference.

@sszretter - try saving it with the “Username” and “Password” boxes blank.

Ok, I think I figured it out - I did a HARD refresh on my work laptop’s browser (CTRL + F5), and when I went to the “Add Backup” screen again, I now get a setup form consistent with what I see on my home system:

seems correct now...

I am now on 2.0.2.5 - my screen is the same but interestingly even though there is no username and password filled in it appears to be working and in sia I noticed it was transferring a file!

1 Like

No matter what I do I now keep getting stuck with the error of 6 missing dblock files (i still have no idea what happened to them - all that happened was I restarted my system a few times, including restarting Sia GUI). I just tried deleting all of the remote index files and running “repair” - it ran all the way through and recreated the remote index files, but at the end I still get the error regarding 6 missing dblock files.

Luckily this was just a test run but is there really no way to recover from missing or corrupt dblock files? Say it’s a huge backup, shouldn’t there be a way to recover the data available in the ones that are not missing?

Maybe we could add something for the paranoid that allows for creation of “parity dblocks”.

Seriously, as we start scraping the bottom of the barrel in terms of cloud providers we might start running into issues that look like Duplicati problems but are actually crappy provider problems.

1 Like

or PEBKAC problems (such as may be in this case) - “oops i accidentally lost one of the dblocks on my local backup drive”, etc.

You can look in the local database, it can be opened by any SQLite browser tool. Inside it there is a table “RemoteOperation” that shows everything that has been done with the remote (Sia in this case). Not sure it reveals everything, but it shows what Duplicati thinks has happened.

You can use the operations “list-broken-files” and “purge-broken-files” (only in the commandline area for now) to see what files are affected, and then remove all references to these files.

Restoring is possible from a broken destination, you can get the “best effort” meaning that it will restore as much as possible given the data that is accessible.

1 Like

Wow - I’m really THREE YEARS late thinking of this? Ouch. :blush:

2 Likes

Hello, I have a nice article about block-chain, check it please)