Softint
September 6, 2017, 9:26am
1
From looking at the database and also testing an restore, this is not saved nor restored.
Is this something planned to be added, without that the backup for an server is kinda useless, its quite important those are restored too.
As a sidenote on the restore webscreen clicking an folder it always takes arround 30s to show the subfolders and clicking one again like 30s, this is with 1 gb sqlite database and arround 1.3 million files.
Softint:
From looking at the database and also testing an restore, this is not saved nor restored.
Is this something planned to be added, without that the backup for an server is kinda useless, its quite important those are restored too.
Duplicati does store UID/GID, but you need to apply the setting “Restore permissions” on the last page. The reason for this design is that it could potentially restore files that you do not have permission to edit/delete afterwards.
There is also a bug with permissions on folders:
opened 02:03PM - 30 May 17 UTC
closed 05:15PM - 24 Apr 18 UTC
bug
pending user feedback
core logic
<!-- This is a bug report template. By following the instructions below and fill… ing out the sections with your information, you will help the developers to get all the necessary data to fix your issue.
You can also preview your report before submitting it. You may remove sections that aren't relevant to your particular case.
Let's begin with a checklist: replace the empty checkbox [ ] below with a checked one [x] if you already searched for duplicate bugs -->
I have:
- [x] searched open and closed issues for duplicates
----------------------------------------
### Version info
**Duplicati Version:** 2.0.1.60_canary_2017-05-23
**Operating System:** Arch Linux on kernel 4.9.27-1-lts
**Backend:** Amazon Cloud Drive
### Bug description
When restoring files, across multiple folders, the setting "Restore read/write permissions" applies properly to the files recovered, but not to the folders they are in. All folders in the given filepaths are set to root, not to their original user.
### Steps to reproduce
- Select to "Restore Files".
- Select files across *different* folders, to force the creation of the containing folders.
- Select to "Pick location", so that the original folders are not used.
- Check the option to "Restore read/write permissions"
- Attempt to restore the files.
**Actual result:**
The files will be created with proper permissions. The folders won't, they will be given to root.
**Expected result:**
Everything, including the folders, should have the original permissions.
Yes, I am aware of the issue. The fix requires a rewrite of how the paths are stored in the database:
opened 04:03AM - 09 Feb 15 UTC
closed 09:44AM - 11 May 18 UTC
enhancement
Using Duplicati 2.0.80, I have a ~75 GB backup set, but the local database is ta… king up 1.999 GB itself.
From poking at `%APPDATA%\Duplicati\LRELCBKGXP.sqlite` with sqlite3, it appears that a great deal of that space is used storing the full path for every file in the database. Running `SELECT sum(length(Path)) FROM File;` reports that the actual path strings occupy ~572 MB not counting overhead. Running `DROP INDEX FilePath;` and comparing the `.sqlite` file size before and after indicates that the index uses ~730 M, so the total space cost of the path strings is probably closer to 600 MB in the main table and then the same amount again in the index.
I did a bit more analysis of the paths using python:
``` python
tot, paths = 0, 0
for name in db.execute('SELECT Path FROM File'):
tot += name[0].rindex('\\')
if name[0].endswith('\\'): # this File is for a directory
paths += len(name[0])
print 'Total redundant path components: %d, total unique path components %d' % (tot, paths)
```
It tells me that the leading path components contribute ~418 MB of data for only ~36 MB of unique leading paths; the 418 MB is then doubled by the FilePath index.
So, I suggest optimizing the storage of file paths. One simple way is to partially dematerialize the full path into `dirname` and `basename` parts:
``` sql
CREATE TABLE "Dirname" (
"ID" INTEGER PRIMARY KEY,
-- this creates an implicit index, but the UNIQUE constraint is important
"Dirname" TEXT UNIQUE NOT NULL
);
CREATE TABLE "File" (
(
"ID" INTEGER PRIMARY KEY,
"Dirname" INTEGER NOT NULL,
"Filename" TEXT NOT NULL,
"BlocksetID" INTEGER NOT NULL,
"MetadataID" INTEGER NOT NULL
);
-- If the UNIQUE constraint is really only (Dirname, Filename), it would be nice to take the other
-- columns out of the index. But if not, this is still a lot smaller than before.
CREATE UNIQUE INDEX "FilePath" ON "File" ("Dirname", "Filename", "BlocksetID", "MetadataID");
```
This fairly simple change will probably shrink the local database by 50+%.
##
<bountysource-plugin>
---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/8410154-local-database-should-store-file-paths-in-a-space-optimized-fashion?utm_campaign=plugin&utm_content=tracker%2F4870652&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F4870652&utm_medium=issues&utm_source=github).
</bountysource-plugin>
Softint
September 6, 2017, 12:21pm
3
I did use that setting and its not working proper.
The left side is current and the right side is the restored backup, it gives read permissions to group and all which wasnt there before, this will break any stuff using the key since its now allowed that anyone expect the owner got permissions to the key file, but even if so it would create serious security problems.
Even if that would work it would also need to assign the uid and guid, otherwise a restore will break stuff.
