I have tried to figure this out on my own through the forum, Google and the Docs, but I am unable to find a solution to this:
For organization purposes I have a few folders on my Windows 10 machine that is named with an @ (at sign) at the start as this makes it appear at the top when sorting by name in Windows File Explorer. ( They are i.e. @ Archive, @ Backup, @ In, @ Out, etc). But when I try to add these folders as a Folder path in Duplicati it doesnât seem to find it and asks me to create the folder.
I have tried other folders that has a simpler naming convention and they work fine. I have also tried to escape the @ with backslash and tried to put the path in double quotes in various ways. I am very reluctant to change this folder structure as Iâve had it for almost 30 years.
Does anybody have any suggestion for what I could try as a solution?
Iâm confused. You say you have existing folders that you donât want to rename so I assume that you want to add them as source for a backup, but in this case Duplicati should never try to create them.
I tried to create 2 folders with names prefixed with â@â and they backed up without problem (test done under Windows)
If what you want is to create a target directory prefixed with â@â, I did not try this. But even if it does not work, why would you want that anyway ?
suggests this is on that field in the Destination screen, and the message is like Test connection does.
Regardless, I can reproduce the popup (but canât explain it). An @ sign later in folder names does it too.
This confuses me. If weâre talking about a destination, it needs an empty folder. Old ones might not be.
One weirdness is that if I watch file accesses in Process Monitor, I donât see C:\@ Sync hitting the OS.
Another is that I havenât found a syntax to get Duplicati.CommandLine.BackendTool.exe list fail.
Uri Class and other mechanisms try to interpret the URI, so maybe the left part is getting dropped here.
As evidence of that, Test connection for folder C:\later @ Sync shows an access attempted to the C:\ProgramData\Duplicati\duplicati-2.0.6.104_canary_2022-06-15\RUN\ Sync folder, which
could mean that the first part got dropped and the rest became a relative path into my current directory.
If somehow it still makes sense to keep this convention, you might be able to hide it from Duplicati with mklink /d so things look more normal to Duplicati while you still keep your favorite view. I havenât tested.
EDIT:
Other supporting evidence for this was seeing the web browser POST a file: URL to server, specifically:
Not that I really agree with such naming practices as they always seem to end up conflicting with something a some point.
Side note/flashback: Any one remember using ALT+0255 to make a âhidden folderâ in DOS?
I digress, it looks like you should be able to use the ~ (tilde symbol to the left of the 1/! key) in the same manner as the @ symbol. It will keep folders with it at the top of the list and Duplicati doesnât seem to be affected.
I tried a few simple backup/restores and it seemed to be fine. Iâd suggest testing it on your end to verify it does truly work for you but it looks promising to me.
I think the point of this is not if you agree with my naming practices or not, nor if you wonder why I would want to do this. I have good reasons for it and it has served me well for a long time and Iâm not gonna change it, neither for the tilde sign or anything else. As long as a character is allowed by the OS to be used in the filename, any decent software dealing with said file system should be able to handle that.
I really thought this was a bug since every other backup or sync tool I have ever used, browser based or otherwise, paid or free have had no problem whatsoever to read or write from or to these folders. Syncthing, Goodsync, UrBackup, Dropbox, Google Drive, OneDrive, Acronis, well, every other similar software have no problem with this.
ts678:
I can see that the use of the @ can get interpreted as an URI in some cases, but I think software that is used for backup solutions on the file system should be able to make the distinctions for when to treat this as a folder path and when to see it as an URI.
Someone else may find the ~ suggestion helpful as an alternative that provides the exact same functionality as the @ symbol without causing any conflicts.
I donât and I donât, itâs about anyone else that reads this thread, knowing that it is a bad idea to use characters like @ in file or folder names regardless of what OS or software they use, even if theyâve done it forever and âeverything else works fineââŠ
A quick internet search for âcharacters to avoid in filenamesâ shows that many other people aside from myself who think that even though you can use certain characters (like an @ symbol) maybe, just maybe, you shouldnât because every once in a while (often at the worst time possible) you will find out it that it can mess things up and all you had to do to avoid it was use a slightly different character. Oddly, I did not find the ~ symbol on any of those lists of characters to avoid, strange stuffâŠ
If everything else is obviously so much more capable than Duplicati, clearly you should be using those tools and not wasting your time around here with this seemingly incapable software and itâs crappy workarounds.
When configuring a backup, the storage URL parameter determines which storage provider will be used. Each storage provider has a different URL format which can be used along with the documented options below.
but Local folder or drive does have a confusing number of different ways shown, and some unshown.
The additional complication is that both the C# code in server and JavaScript in GUI try to deal with it.
is probably what the CLI is using, and \\?\C:\later @ Sync\ or file://C:/later @ Sync/ work, however trying to put them into GUI gets fields broken out by the JavaScript, and not as I would wish:
While thereâs seemingly a bug in here somewhere, finding and fixing it without breakage may be hard because regular expressions are hard, and because Duplicati has other URI uses that must still work.
The forum is not a bug tracker, but you are welcome to open an Issue. Also see if any is already filed, because typically ones that affect a lot of people and have no workarounds are good ones to work on.
Please also keep in mind that Duplicati is created by people who volunteer personal time to try to help. Volunteer developers are needed. Forum is doing better, but your help and understanding are needed.
Having rejected some proposed workarounds, what about this one mentioned before, and now tested?
C:\>mklink /?
Creates a symbolic link.
MKLINK [[/D] | [/H] | [/J]] Link Target
/D Creates a directory symbolic link. Default is a file
symbolic link.
/H Creates a hard link instead of a symbolic link.
/J Creates a Directory Junction.
Link Specifies the new symbolic link name.
Target Specifies the path (relative or absolute) that the new link
refers to.
C:\>mklink /d "C:\Duplicati Backup" "C:\@ Backup"
symbolic link created for C:\Duplicati Backup <<===>> C:\@ Backup
C:\>
Note that the icon shows that itâs a link, however Test connection follows that to @ Backup at top.
Adding the synonym lets you get view you like, while also giving Duplicati something it handles well.
Ideas are probably all that we can give you instantly. If you wish, file an Issue, to get in the long line.
Done.
EDIT:
So hereâs my test backup in a folder whose naming you favor:
JimoboJones is correct to an extent. Eg Android 12 on my phone it itself wonât allow use of ? in the folder names for file creation but works fine for folder creations (when I tested it in app code). The system file explorer app wonât even allow ? in the folder name.
On desktops, various applications have seen issues with symbols in paths as problems people have seen. This stuff happens. You only happened to have not seen it until now lol.
It is however something that programmers working on paths should think about checking but its understandable to not think about it or test every variation given how much time one wants to spend on testing and can think of.
It is likely Duplicati with a bug though. Although, the .net version or whatever could see it too and be the reason. But, its expected to be the application.
Duplicati can backup and restore a source path that starts with, contains or ends with an @ symbol without issue.
The issue only presents when using an @ symbol in the destination path.
Symptoms:
Any part of the specified path prior to the @ symbol is removed. The job will save (which it shouldnât) and run but then goes to âVerifying backend files.â almost instantly then a minute or so later errors out complaining about an invalid path along with error message in the GUI showing the specified path contents after the @ symbol.
Work Arounds:
Use a different character in the destination path.
Create a symbolic link and use the linked path in Duplicati.
Fix:
A quick look at that Regex says itâs not directly responsible for removing the @ symbol, likely one of the .replace() methods a bit further down is stripping it out. Probably just need another if to check if itâs a local destination or URI and if so run a slightly different sanitizing on the string. Probably quite low on the priority list but itâs also probably a fairly easy fix.
A third workaround (but probably more unappealing to GUI users) is to use CLI. These one-line
regular expressions are hard to read. Thanks to @JimboJones for persevering with code reads.
I got as far as finding by experiment a CLI URI syntax that worked, then GUI JavaScript choked.
In addition to another one-line regex there, one would have to read JavaScript more than I doâŠ
I wasnât willing and able to either put a debugger onto the program or write test code for parsing,
given how there are a couple of workarounds. I did take the C# regex and try to study multi-lines.
organized somewhat around what gets grabbed via (?<name> and whatâs optional (? after something). Syntax section of Uniform Resource Identifier Wikipedia article has a diagram, and links to references.
An optional userinfo subcomponent followed by an at symbol (@), that may consist of a user name and an optional password preceded by a colon (:).
was my theory of where the portion before the at symbol was disappearing into. It IS sometimes used. Storage Providers shows some examples of what looks like the userinfo part. Iâm not sure if a folder accepts it or not. It wouldnât apply for truly local, but could SMB credentials maybe be given that way?
Someone could, if willing, continue my use of the Backup destination three-dot menu to put URIs in.
I kept getting unwanted pieces landing in the Username and Password fields, but I donât do JavaScript.
Upon a deeper dive, it looks like the regex is excluding (not capturing) the @ symbol when dealing with file:// URI paths.
It works correctly for capturing usernames in http:// URIs such as user.name@example.com but excludes the :// and @ parts when capturing file:// URIs which in turn creates too many slashes in the path and the capture groups end up capturing the wrong matches and populating the other fields with slashes or a slash and the drive letter.
If file://./ is used to specify the host it blows up when an @ is added to the path, group 2 becomes ./driveletter which doesnât work, using file:/// causes a null in group 4 that which seems fine until an @ is added and it falls apart, group 2 becomes /driveletter.
I didnât notice anything about passing SMB credentials over file:// I expect that to be the OSes job to be deal with but Iâll have to do some more reading to be sure.
As always Windows requires some special cases to work around like supporting dual or quad slashes (currently does not seem to support either) but thatâs a side note and it just means a few more optional ? capture groups and an even longer expression.
Looks like the expression needs a bit of love, Iâll see what I can come up with.
Good point, but if they have to go to the OS, Duplicati needs to be able to pass them, not kill them.
The on-screen Username and Password fields are (we think) for SMB use. You can see remarks at Local disk credentials issue where someone thought it was to change the user, for NTFS ACL use.
Thank you for trying to help, unlike others around here.
As Iâve said before, the naming convention has a long history for me and is used in many software solutions and is used in many of my Python and AutoHotkey script. It is therefor an enormous task to undergo to change all this, which I have neither the time nor desire to do for a change in backup tool.
Maybe I will try this software at a later stage as it seems to be a good option, and I do appreciate all the voluntary done by the people behind it. It just doesnât seem to be for me right now.
Sorry to barge in your research, but Iâm always interested in understanding why this is this way. While I did not look at the code, I am reminded of the @ in sftp uri and thinking that maybe it was the motivation behind this special handling. It may be that the code is using a generic regex to handle all sorts of backends uri, but that writing such a generic regex may be too complicated, and that the way forward could be to handle the whole problem differently - a specific handling for some backends possibly.
Thatâs mere speculation so feel free to ignore my ramblings.
I havenât tried this to see what would actually happen with Duplicati, but a warning to anyone who might attempt it with any sort of UNIX/Linux: â~xxxxxxâ generally translates automatically to âpath to user xxxxxxâs home directoryâ (Like â/home/xxxxxxxâ). While it might work in Duplicati (if neither Duplicati, nor any system libraries it uses, parses it that way), anyone trying to access such a folder/file from a UNIX/Linux shell is going to, at least, have to escape the â~â to reference it. I know this thread isnât about alternatives, and this is not a suggestion to the OP since it wonât help them. Underscore (â_â) is about the only special character Iâd ever use in a file/folder name. Itâs harmless in both UNIX/Linux and Windows.
