Dupliact in FreeNAS Jail - User/Permissions

Another thought. I looked at the folders I created in /mnt. These folder are what I mount the FreeNas folders to so I can segment the backup into different processes.
root can see these folders, but duplicati cannot. All duplicati can see is the /mnt with nothing in it.
I created these folders as root.

What do I need to change so that the user duplicati can see these folders, and then maybe duplicati will see the files below that

?

I tried chmod -v 755 * (also directory names) and get Operation not permitted
using chown to give ownership to duplicati means I can see the folders but nothing in them

Sigh

bummer… sorry, I have very little experience with FreeNAS jails. Maybe another user with more experience can speak up.

That may be part of it, but Duplicati still needs to read files within those mount points. So if any don’t have at least read access to “other” users, Duplicati won’t be able to back them up.

What are the current permissions of the mount points within /mnt ? Can you do an ls -l /mnt and show the output?

Thank you for your help

The result is that all three subfolders of /mnt are drwxrwx—, but I have been messing around with chown to no avail

Instead of messing with chmod or chown inside the FreeNAS jail, try looking at the mapped folder settings in the FreeNAS Web UI instead. I don’t recall if there was any ability there to customize access to the mapped folders for the jailed application.

If I give access to everyone in the FreeNAS settings then Duplicati has access. However I consider giving everyone access, which also effects any Windows (CIFS/Samba) shares a s a bad thing, and walk over any form of security.

The duplicati user is not defined in FreeNas, only in the Jail so I have no apparent way to assign privaleges to duplicati

Sean

Personally I would just get Duplicati to run as root in the freenas jail, instead of changing permissions on your nas shares.

When I mentioned the freenas web UI I was talking about management of the Duplicati jail. There may be a way to get Duplicati to run as root there. If you are still stuck I can try bringing up a freenas vm to experiment

I tried messing with a load of settinsg for the Jail. No effect.

I agree with you that running duplicati as root will solve the issue. Just can’t figure out how. I am not an “ix” expert by any stretch of imagination (even mine)

Sean

Give me some time to mess around with it, I’ll see if I can figure it out…

I attempted to change the user that Duplicati runs as by issuing these commands (from within the jail shell):

# sysrc duplicati_user="root"
# sysrc duplicati_group="wheel"

I believe that is the correct method instead of editing the /usr/local/etc/rc.conf file directly.

I also changed ownership of the Duplicati config files:

# chown -R root:wheel /config

I restarted the jail and noticed the same thing you did - Duplicati doesn’t start. But what’s weird is you can manually start it by doing this from the jail shell:

# /usr/local/etc/rc.d/duplicati start

It seems to work and it is running as root:

I don’t understand why it no longer auto-starts after issuing those sysrc commands to change the user it runs as. I’ll try to do more digging when I have time, but I wanted to share my progress so far.

I also have had some luck.
Speaking to a freind of mine he came up with an a idea.
Given that the jail file structure is part of the FreeNAS file structure. He suggested creating a user with the same UID as the UID that was in the Jail passwd file (1001)

So we created a user in FreeNAs with a UID of 1001 to match the 1001 inside the Jail and gave that use all rights over the Dataset that we were trying to backup.

And it worked - the duplicati user can now see the files in the mounts
and back them up
and restore them (even more important than backing them up). caveat - I am getting some restore errors about failing to apply metadata to some files but the files are there and working.
[I am testing with a bunch of MP3’s so testing the files is quite easy]

An example below
2020-04-28 21:15:17 +01 - [Warning-Duplicati.Library.Main.Operation.RestoreHandler-MetadataWriteFailed]: Failed to apply metadata to file: “/mnt/Music/Atomic Kitten/Right Now (2001)/12 - Strangers.mp3”, message: Unterminated string. Expected delimiter: ". Path ‘[‘unix-ext:user.DosStream.01APIC_00.jpg:$DATA’]’, line 1, position 102397.

BUT the specific file mentioned is back and readable. I think this is a different issue (currently), also permissions, but different

See [How-To] Giving Plugins Write Permissions to Your Data | iXsystems Community for useful information which I hadn’t spotted

Sean

Following the post you linked would be the intended course of action. You will find similar posts if you look for information around resilio sync or other plugins that you want to interact with your datapools. Plugins use the freebsd jails construct. Jails are isolated to themselves by design and require a bit of tweaking to get to interact with the host system.

Hello
I’m a newbee with truenas and I testing it at this time…
I have made install of the plugin Duplicati under truenas but I cannot see the directory under /MNT/

I have read here it needeed to create a user in truenas with the UID in the jail duplicati…
But how to know the UID of duplicati ?

Many thanks for your help

I’m also a newbee and have run into similar problems. But I have a different suggestion for a possible solution.

I created 5 directories, and Duplicati’s file browser only sees two. Both have 775 permissions. Two of the other three have 770, and one has 070. The key here is the middle (group) permission: 7.

What I propose is to look up the group for each of the three unseen directories and then add the Duplicati user to those groups.

The only fly in the ointment is that Duplicati is only a user in the jail; it’s not in the base FreeNAS. So one would have to find the group IDs of the directories in FreeNAS, outside the jail, and then:

  1. Check to see if the group id is assigned in the jail. If so, just add Duplicati to the group.
  2. If not, create the group inside the jail and add Duplicati to it.

Does anyone with more experience see any reason why this wouldn’t work? If it works, it seems to be an easier, more elegant solution rather than running Duplicati as root or creating a separate user with the same uid as Duplicati.

ADDED AFTER DEMONSTRATING THE PROOF OF CONCEPT (SEE BELOW)

This approach works quite well and has some major advantages over NugentS’s suggestion of creating a user in FreeNAS with the same UID as Duplicati has in the jail.

By working only in the jail to assign the Duplicati user to the necessary groups, all customization remains in the jail and specific only to Duplicati. If instead you find the Duplicati user’s UID in the jail and then create a Duplicati user with the same UID in FreeNAS or TrueNAS, what happens when you run into the same problem with another plugin, call it FooBar, and it has a FooBar user with the exact same UID? An individual user can have only one UID but many GIDs. Once you assign Duplicati’s UID to the FreeNAS Duplicati user, FreeNAS user FooBar would be out of luck. In contrast, because a user can have multiple GIDs, leave the FreeNAS user list alone and modify the group list in the Duplicati jail.

As a general rule, when customizing jailed plugins it’s best to do the customizations in the jail itself. This makes it much less likely that the customizations will interfere with other plugins or FreeNAS/TrueNAS itself.

I went ahead and tried my proposal, and it works quite well. Here’s a step-by-step.

  1. Identify the groups to which the directories you want to backup belong.

    FreeNAS > Storage > {dataset you want to backup} > : (3 dots) > {Edit Permissions or Edit ACL}. Copy (write down) the group name from the Group field in the file information.

  2. Identify the GIDs of these groups.

    FreeNAS > Accounts > Groups (inspect the groups you identified in Step 1, and write down their GID numbers.

  3. Enter the jail for the plugin (duplicati in this case).

    FreeNAS > Jails > duplicati > " >" > “SHELL”

  4. Inspect the jail’s group file.

    cat /etc/group

  5. In the output from Step #4, look for any of the groups you identified in Step #1. There will be four possibilities:

    a) Most likely, the group names and GIDs are not assigned to any groups in the jail. If so, add them as described in Steps #6 & 7, below.

    b) There’s a group in the jail with the same name and GID as you found in Steps #1 & 2. If so, you don’t need to do Step #6 for such groups.

    c) There’s a group in the jail with the same GID as you found in Step #2 but a different name. If so, don’t bother with Step #6 and assign duplicati to this group when you get to Step #7 below.

    d) There’s a group in the jail with the same name as you found in Step #1 but a different GID. If so and you didn’t find another group with this GID in Step #5.c, in Step #6 create a group with a unique name and the GID you found in Step #2. In Step #7, assign duplicati to this new group.

  6. Add the missing groups to the jail.

    pw groupadd -n GROUP -g GID

where GROUP is the case-sensitive name of the group you identified in Step #1, and GID is the corresponding group ID you identified in Step #2. Do this for every group you identified in Steps #1, 2, & 5.

  1. Add duplicati to each of these groups.

    pw groupmod GROUP -m duplicati

You might want to check that duplicati is an actual user in the jail. (cat /etc/passwd).

Here GROUP is one of the group names you identified in Step #1, bearing in mind the details in Step #5.

See Joshua Parker Reuhlig’s post, “[How-To] Giving Plugins Write Permission to Your Data” for more on adding groups and users in a jail.

  1. Restart the jail.

    FreeNAS > Jails > duplicati (Duplicati jail name. Usually it’s “duplicti”.) Check the checkbox.

Select UPDATE.

  1. If you were in the Duplicati web interface.

    Exit Duplicati and log into it again.

Hi titou43,

A few points:

  1. I believe you mean /mnt/. TrueNAS is based on BSDUnix, and all Unix systems are case sensitive. So /mnt/ and /MNT/ are two different things.

  2. To learn duplicati’s UID, you would start a shell in duplicati’s jail. Then from the command line you could enter:

     cat /etc/passwd | grep duplicati
    

This would print out the duplicati entry in the shell’s /etc/passwd file. Read the UID from the output.

  1. BUT DON’T DO IT THIS WAY! See my two posts above.

Hello Monkey_Demon

The solution proposed by Sean function very well for me.

I will try your solution for compare (and learn Truenas too) with the solution of Sean for to prenvent if in the future with another plugin it needeed to do other configuration

Many thanks to you for this idea, may be i will return to you…

And many thanks for Sean who help me very well

Titou43 (claude)

Hi titou43,

For now, both solutions should work equally well. Assigning Duplicati to groups within the Duplicati jail has several advantages that may affect you in the future.

  1. With creating a new Duplicati user in FreeNAS or TrueNAS, if you ever need to use the same UID for other purposes, Duplicoti will be hogging the UID.This won’t happen if you just assign Duplicati to groups in the jail.
  2. With creating a new Duplicati user in FreeNAS or TrueNAS, if you ever switch from Duplicati to another plugin for backups, you’ll have to remember to go back and clean up the UID among FreeNAS/TrueNAS users. If instead you just make Duplicati a member of the appropriate groups in the jail, you’ll just have to delete the jail to clean everything up.

So in less words what you have said - just to get this straight in my mind.

  1. wheel owns all the relevant datasets
  2. wheel is GID 0
    3/4 in group file wheel is wheel:*:0:root
  3. Option b so skip 6
  4. Skip
  5. Add duplicati to wheel
  6. Restart

[Precis - add duplicati to wheel in jail]

I have now added (in my head) duplicati to wheel and wheel is some sort of super group - so doesn’t this leave duplicati with too many permissions. For example it now has access to all datasets on TruesNAS, even the ones I don’t want to backup (iSCSI and NFS datasets for example.)

Using “my” process (made up of several other people’s ideas), whilst more complex, only gives duplicati access to what I explicitly grant access to rather than everything as wheel by default owns every dataset - at least it does on my NAS.

I would point out that my “objection” above is in theory only. For my use case, the group theory would work just fine and I wouldn’t care about the excessive rights. However if this was a more commercial operation I would be a bit twitchy as others might have access to Duplicati

I guess you could change the group that owns the datasets and add duplicati to that group - I wonder what the implications of changing the dataset owner would be (might try that out when I get home, and have time):
Create new group backup_gid. GID 1000
Create new dataset /mnt/SMB/Test. Change dataset group owner. Add some files to Test
Add group in jail with GID 1000
Add duplicati to backup_gid in jail
See if it works

Sean,

I’ll try to address your concerns. Sorry this became so long, but your profile says you’re a basic user. I’m a relative newbee to FreeNAS too, but I’ve been using and administering computers since the 1960s. I think this gives me some helpful perspective, and from you comments I surmise that a view from 20,000 feet would be helpful. So here goes.

History: Invention of the Wheel

When I first used Unix in the late 1970s, files and directories had 3 kinds of permissions: owner, group, and other. Although I had administered IBM mainframes, back then I was only a Unix end-user, and AFAIK, files could be associated with only one group. Then, somewhere along the line, Access Control Lists (ACL) were introduced, at least in FreeBSD, which allow much more fine-grained access control. In particular, multiple groups can be associated with a single file or directory, which in Free/TrueNAS carries over to datasets.

Unix itself had evolved from other operating systems (the name itself was a reaction to “Multics” – an experimental project at MIT). Early Unix systems often ran on Digital Equipment Corporation’s PDP computers, which commonly shipped with the TOPS operating system. As a security measure, TOPS distinguished between super- and and ordinary users. Instead of everyone with the superuser (or root) password having access to superuser status, such potentially dangerous privileges were reserved for the “big wheels” in the organization. Hence, the “Wheel group.”

Who Needs Wheels?

The question is, do you really want your datasets to belong to the wheel group, especially in the jail? Your datasets are “Install,” “Music,” and “Data.” Now I don’t know how you use them or intend to use them. But I’m working with five datasets, which are somewhat similar: Documents, FTP, Media, TimeMachine_dataset, and Users. By my conscious decisions, none of them has wheel as a group owner. Documents is in the webdav group, FTP is in shared_ftp, Media is in media, TimeMachine_dataset is in TimeMachine, and Users is in users.

By making such distinctions, I can better control access to the datasets. For example, I’m the only member of the webdav group because I use WebDAV to maintain a large library of pdf files, but if a file allows “other” users to read it, anyone with WebDAV access to the dataset can read it (but not change or delete it). So you may want to reconsider if you really want your three datasets to be in the wheel group at the FreeNAS level.

But even if you do, this doesn’t mean they have to be in the wheel group within the jail. See this thread and the ACL documentation for FreeBSD. The question you need to ask is if Duplicati needs superuser (“big wheel”) privileges in the jail, or if simple read/write/execute access to the datasets you’re backing up would be sufficient. I suspect the latter is the case. And if so, you can use groups other than wheel within the jail and still leave wheel as a group owner outside of it.

Baby You Can’t Drive My Car

I think you’re on the right track at the end of your post. But instead of a generic group, backup_gid, I would suggest two things.

First, a minor point: all groups have GID’s, so adding “_gid” to a group name is redundant. But then again, every dataset you access from the Duplicati jail is likely to be there because you’re backing it up. So if this is the only place you care about the backup group, using “backup” as a name would also be redundant.

This brings up the second, more useful point. There’s a ton of information about not only Free/TrueNAS, but also general *nix system administration. (One free [2011] handbook for *nix system administration is 1344 pages long!) But the basic idea of groups is to allow users to collaborate or who have similar needs to share data, while excluding those who don’t have such needs.

So, go back to your original three datasets and consider them in this light. Even if you’re currently the only user on your system, imagine that someday you’ll have many users using it. (Maybe because you use it for a business that grows, your family grows and someday includes hacker children, you make new friends and share music with them, etc.) Now consider your datasets and decide if they need identical access permissions or are logically different from each other. (Remember that as the system administrator you can give yourself sufficient access that such differences become almost invisible to you.) Then decide what groups you need, what datasets should be associated with which groups, and which users should be in which group.

You may decide, as I did, that each dataset is (potentially) associated with unique needs. For example, you may want to allow some friends to listen to your music but not to access your data. If you associate Music and Data with different group owners, making such distinctions become trivial. OTOH, if you associate them both with a single “backup” group because they share the characteristic that they’re backed up (as most of your files should be), then you may have overlooked other important distinctions between the two datasets and their potential users.

In Short

This all boils down to three simple principles:

  1. Think through your ultimate and potential use of datasets, and create groups and assign group ownership accordingly.
  2. When customizing access from a jail, as much as possible try to confine the customizations only to the jail.
  3. Because security is always a concern, don’t overdo usage of “owner” or “other” permissions – the former may give away too much power, while the latter may give power to the wrong users; instead, use groups judiciously to group subsets of users with similar needs and to exclude users without such needs.

I disagree with nothing you have said there - I think you took what I said and expanded (considerably) on it. Also I use Active Directory to define user rights over the files - I don’t have ix type clients so don’t have to worry about them. The fact that wheel owns all the files / datasets doesn’t actually mean anything from outside the NAS

[Basic User is I think in reference to Duplicati]
I am a retired network engineer - with a limited knowledge of Linux/FreeBSD/Unix