OneDrive full - delete unsuccessful

Support
1

Hi,
first of all and once again thank you for this great piece of software. I had my issues when I started but I’ve been using Duplicati as my main backup solution for over two years now and it never let me down. Plus, I love how flexible you can do so many things via the command line (and not cluttering the UI with it)!

Now my problem. I’m using OneDrive on a business license with 1 TB. After almost two years, it now ran pretty much full (998 of 1000 GB used), I saw the warnings and wanted to free up some space with the delete command (here with options --version=13-18). That threw me a long error message. I repeated it with a smaller delete job (–version=18) and it, again, did not work. Here’s the error message from the log:

30. Nov. 2023 15:38: Die Operation Delete ist mit folgenden Fehler fehlgeschlagen: Error converting value "full" to type 'System.Nullable`1[Duplicati.Library.Backend.MicrosoftGraph.QuotaState]'. Path 'quota.state', line 1, position 769.
{"ClassName":"Newtonsoft.Json.JsonSerializationException","Message":"Error converting value \"full\" to type 'System.Nullable`1[Duplicati.Library.Backend.MicrosoftGraph.QuotaState]'. Path 'quota.state', line 1, position 769.","Data":null,"InnerException":{"ClassName":"System.ArgumentException","Message":"Requested value 'full' was not found.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":"  at Newtonsoft.Json.Utilities.EnumUtils.ParseEnum (System.Type enumType, Newtonsoft.Json.Serialization.NamingStrategy namingStrategy, System.String value, System.Boolean disallowNumber) [0x001f9] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Converters.StringEnumConverter.ReadJson (Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Object existingValue, Newtonsoft.Json.JsonSerializer serializer) [0x0007b] in <f5c4580481df43759ebf446d5ffb33c1>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147024809,"Source":"Newtonsoft.Json","ParamName":null},"HelpURL":null,"StackTraceString":"  at Newtonsoft.Json.Converters.StringEnumConverter.ReadJson (Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Object existingValue, Newtonsoft.Json.JsonSerializer serializer) [0x000f1] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable (Newtonsoft.Json.JsonConverter converter, Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Object existingValue) [0x00055] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00052] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00280] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00065] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00280] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Boolean checkAdditionalContent) [0x000db] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00054] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.JsonSerializer.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Newtonsoft.Json.JsonSerializer.Deserialize[T] (Newtonsoft.Json.JsonReader reader) [0x00000] in <f5c4580481df43759ebf446d5ffb33c1>:0 \n  at Duplicati.Library.Backend.MicrosoftGraphBackend.ParseResponse[T] (System.Net.HttpWebResponse response) [0x0001c] in <31240f90f26d4e5da68af2ea32d1c2b6>:0 \n  at Duplicati.Library.Backend.MicrosoftGraphBackend.SendRequest[T] (System.Net.Http.HttpMethod method, System.String url) [0x00043] in <31240f90f26d4e5da68af2ea32d1c2b6>:0 \n  at Duplicati.Library.Backend.MicrosoftGraphBackend.Get[T] (System.String url) [0x00006] in <31240f90f26d4e5da68af2ea32d1c2b6>:0 \n  at Duplicati.Library.Backend.MicrosoftGraphBackend.get_Quota () [0x00007] in <31240f90f26d4e5da68af2ea32d1c2b6>:0 \n  at Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.Collections.Generic.IEnumerable`1[T] protectedFiles) [0x00291] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.Collections.Generic.IEnumerable`1[T] protectedFiles) [0x00000] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter backendWriter, System.Boolean latestVolumesOnly, System.Data.IDbTransaction transaction) [0x00019] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Operation.DeleteHandler.DoRun (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Data.IDbTransaction& transaction, System.Boolean hasVerifiedBackend, System.Boolean forceCompact, Duplicati.Library.Main.BackendManager sharedManager) [0x00043] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Operation.DeleteHandler.Run () [0x00075] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Controller.<Delete>b__17_0 (Duplicati.Library.Main.DeleteResults result) [0x00012] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.Action`1[T] method) [0x00009] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.Library.Main.Controller.Delete () [0x00006] in <9cab5b8f1b4f49ec980acae8e278968b>:0 \n  at Duplicati.CommandLine.Commands.Delete (System.IO.TextWriter outwriter, System.Action`1[T] setup, System.Collections.Generic.List`1[T] args, System.Collections.Generic.Dictionary`2[TKey,TValue] options, Duplicati.Library.Utility.IFilter filter) [0x00084] in <81fea8e154eb4fd9aa8b06aac66d311a>:0 \n  at (wrapper delegate-invoke) System.Func`6[System.IO.TextWriter,System.Action`1[Duplicati.Library.Main.Controller],System.Collections.Generic.List`1[System.String],System.Collections.Generic.Dictionary`2[System.String,System.String],Duplicati.Library.Utility.IFilter,System.Int32].invoke_TResult_T1_T2_T3_T4_T5(System.IO.TextWriter,System.Action`1<Duplicati.Library.Main.Controller>,System.Collections.Generic.List`1<string>,System.Collections.Generic.Dictionary`2<string, string>,Duplicati.Library.Utility.IFilter)\n  at Duplicati.CommandLine.Program.ParseCommandLine (System.IO.TextWriter outwriter, System.Action`1[T] setup, System.Boolean& verboseErrors, System.String[] args) [0x00342] in <81fea8e154eb4fd9aa8b06aac66d311a>:0 \n  at Duplicati.CommandLine.Program.RunCommandLine (System.IO.TextWriter outwriter, System.IO.TextWriter errwriter, System.Action`1[T] setup, System.String[] args) [0x00002] in <81fea8e154eb4fd9aa8b06aac66d311a>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":"Newtonsoft.Json"}

I can’t really get a grip on it. Does anyone how I can safe the backup and/or free up some space manually on the OneDrive? I could simply access the OneDrive via the UI and start deleting old packages but something tells me I should check here first :wink:

Thanks for any ideas!

2

oh and maybe one addition:

It does work (deleting old versions) in general, when sending the same command to another backup destination with the same domain but a different OneDrive account.

3

On first glance, it looks like the quota reports “full” instead of a percentage or numeric value, so the parser can’t handle it. Can you free up some space in other ways?

Don’t do that. The backup data in old files is most likely still necessary for new versions, except for list files (named by the date). But if you delete those, it might fail to run your delete command because the backend is not as expected.

1 Like
4

@degudejung

Try to add the no-backend-verification option to your delete operation.

1 Like
5

Is this all for one Duplicati job? You might need to free space a little bit at a time, even if temporarily.

OneDrive personal seems to reduce the GUI’s storage “used” value if I put items in the Recycle Bin.
Taking them back out puts the value back where it was. Ideally, test this with some less valued stuff.

One challenge is that even if the delete works, space isn’t freed until a compact runs, and a compact requires a little bit of free space because it uploads the new remote volume before deleting old ones.

What happened to your final 2 GB anyway? Does the GUI now show it completely full? Due to what? When scrambling around trying to free a bit of space, we don’t need something quickly filling it again.

1 Like
6

Is this all for one Duplicati job? You might need to free space a little bit at a time, even if temporarily.

Yes, indeed. Unfortunately, this OneDrive only has the data from this backup job so without touching the backups, I see no way how to free up space.

OneDrive personal seems to reduce the GUI’s storage “used” value if I put items in the Recycle Bin.
Taking them back out puts the value back where it was. Ideally, test this with some less valued stuff.

Worth a try as a measure of last resort, thanks

What happened to your final 2 GB anyway? Does the GUI now show it completely full? Due to what? When scrambling around trying to free a bit of space, we don’t need something quickly filling it again.

Nothing changes about the last 2 GB. New backups fail (obviously) and since there is nothing else going on but the Duplicati backups, it’s a pretty stable mess currently :joy:

7

add the no-backend-verification option to your delete operation

That did the trick! It took me a minute to find the full parameter is
--version=x --no-backend-verification=true
but then it went trough.

After that, I had to manually entry the trash of the OneDrive. And after that empty the “eternal trash” (or however that translates from German), which I find kind of funny that you now have multiple layers of trash cans…

So far my Duplicati GUI stills shows 998 GB / 2 GB free but that should get fixed with a compact run or even a new backup.

Thank you everyone. I’ll tag this as solved.

1 Like
8

This is not necessarily true in all cases. If one deletes an old version, yes. However when filling out my OneDrive (all the 5 GB of it), I carefully created new files. When deleting the last version, Duplicati immediately deleted all the most recent remote files uploaded with the most recent data. When directly connected to OneDrive, the reported size was updated and the drive was not full anymore.

9

Delete can run the compact:

and the last version has the benefit of being able to fully delete files instead of having to repackage.
So if one is completely full, starting with the last version is probably a safer way to avoid having the repackaging compact fail due to lack of space to upload a new volume before deleting old volumes.

or so the theory goes. You can look at an Information level log to see if you’re getting lines like this:

2023-11-08 06:40:20 -05 - [Information-Duplicati.Library.Main.Database.LocalDeleteDatabase-CompactReason]: Compacting because there are 3 fully deletable volume(s)

and verbose log will show even more. I’m still thinking one needs a compact, but it can be hidden.
It can also be done for various reasons and in various ways, as its trigger looks at many factors…
I didn’t read the code long enough to see if complete deletions require another trigger beforehand.

Starting at the latest version is probably a good way to start freeing space if that is extremely tight, because with changes only uploaded as needed, making them not-needed “should” allow all to go.