I receive this error message - what can I do? “Der versuchte Vorgang ist unzulässig, weil er den vom Administrator erzwungenen Listenansicht-Schwellenwert überschreitet.”
Interesting, what were you doing at the time this error appeared?
I’m starting a backup. Some minutes after start I receive this error. Backup is aborted without any other message.
Kind regards
Andree
What version of Duplicati are you using? What OS is installed on the machine running Duplicati? If not Windows, what version of mono?
Also what back end target type are you using?
Hi Rod!
I’m using Duplicati 2.0.5.1_beta_2020-01-18 on a Windows Server 2016 standard. My target is Microsoft OneDrive für Business.
I try to backup round about 5 Million files = nearly 5 Terabyte. The target ist not out of space: It is filled nearly 70 percent and there is enough space on it. Local Harddrive is also not out of
space. There is space of many Gigabytes.
I try to translate this message: „The operation you’ve tried is not allowed because it exceeds the list view threshold enforced by the administrator“.
At the target (Onedrive) I find 5546 elements. Log data at the administration page is empty.
How you can see there is an other operation on the same server: „Alle…“ is working correctly. It backups to the same target – only to an other subdirectory. The second operation („Benutzer…“) leads
to the same …threshold error. That’s why it is deactivated until I’m able to solve this problem (hopefully!!).
Kind regards
Andree
Ok I’m not a OneDrive user but I believe it has a limit of ~5000 files. One solution may be to increase the “remote volume size” in Duplicati from the default of 50MiB to something larger. This results in fewer files on the back end, but it has its downsides too. So you don’t want to pick too large of a value, either. See Choosing Sizes in Duplicati - Duplicati 2 User's Manual for more information.
Also, if you do increase the volume size, then with OneDrive I believe you may also need to increase the --http-operation-timeout value from the default (100 seconds, if I recall) to something longer. It needs to be long enough to allow for a larger remote volume size to be downloaded or uploaded within that timeframe.
Finally, note that changing the remote volume size only affects future volumes that are uploaded. So you may need to erase your backup data on the back end and start over to fix this.
File server backup suggested another tuning (and more) for a 12 TB server. Your 5 TB might benefit.
The working backup seems like it must already have an oversize Remote volume size per job stats showing 898 GB on the remote. Because each dblock has a dindex, 2500 dblocks hold only 125 GB.
Viewing the log files of a backup job to find BackendStatistics KnownFileCount should either be under 5000 due to larger remote volume size, or somehow the Alle backup is working differently than others.
For the error ones, does it upload for a long time (depending on upload speed), then fail towards end? Uploading does not continuously read the directory listing, but file listing is checked after uploads end.
If you search for words onedrive list view 5000 you’ll see this is a notorious limit on file listings… There are some signs that the new Microsoft Graph API helps it, though I haven’t seen it documented.
Storage Providers on page right side Table of Contents shows which storage types use the Graph API. Generally I’d think Microsoft SharePoint v2 would be your best fit, but I can’t detail how to convert.
What storage type are you using now? OneDrive for Business uses SharePoint technologies, I believe.
Here is some talk about the 5000 list view limit. Results from these aren’t clear, but links from them are.
No backup without “–no-backend-verification=true” also gives some migration hints if you want to try…
Issue using Onedrive for Business
Various positive + negative feedback also talked about German servers being different. I know no more.
Thank you very much for your information. It helps a lot!
I changed remote volume size (2 GB) and http-operation-timeout (120 minutes) and everything runs fine!
Glad you got it working. 2GiB is pretty large for a remote volume size. Note that restoring any file, even tiny ones, will require the download of at least one volume. That’s the downside of using really large volume sizes. If you have a fast connection and/or restore infrequently, maybe it’s not an issue.