Maybe, if not too revealing about problems with current scheme. Or maybe post it somewhere.
The issue captures the main need, which is how to deal with the additional challenges it raises.
These do go somewhat hand-in-hand, as people will want to know the usefulness of more pain.
Hopefully without compromising security too much – and as you noted there are some tradeoffs.
Thanks for making plans to improve the UX.
ah that’s it
old Duplicati-server.sqlite is in indeed in ~/.config/Duplicati/
this version Duplicati-server.sqlite is in ‘~/Library/Application Support/Duplicati/’
its not a huge issue for this machine to start a new backup,
but is there a way to fix this?
cool, got an other intel mac, running 12.6 (Monterey) that I can update if we have the database location sorted out.
when updated, I will add some screenshots and more details to the issue
====== more test results info added =====
updated my mac to macos 15 Sequoia beta, no issues with Duplicati
and updated a Ubuntu machine,
Duplicati didn’t know what to do with the %HOME% folder that was set up in the backup, so unselected that.
apart from that, noticed that the first backup was a full one.
Hmm… It should have been fixed:
Did you run any versions in between 2.0.8.1 and 2.0.9.104? The logic checks for both an old and new. If a database on the new path is found, that database is found. If you had 2.0.9.100 installed, it would have created the new database, causing it to ignore the old one.
I have posted the description here.
To replicate this, you had:
%HOME% (checking the user folder)%HOME% folderIs that accurate? What version of Ubuntu is this?
i’d need to check that, on the ubuntu machine i found the duplicati version in the logs of the backups in the web interface, so i need access to the database to do that
is it as simple as rename the Duplicati-server.sqlite in the '~/Library/Application Support/Duplicati/’ folder and then copy the one from ~/.config/Duplicati/ in there to use the old database?
missing important details indeed
machine is running Ubuntu 22.04.4 LTS
old version Duplicati was 2.0.7.103 canary
Yes. The logic behind this change is the ~/.config is a *nix convention and Apple recommends storing configurations in ~/Library/Application Support.
If you move it to ~/Library/Application Support/Duplicati/ it should work for anything going forward, but it should also work for users upgrading from 2.0.8.1 with no manual intervention.
Thanks for the details. I have added an issue for tracking the problem with %HOME%.
copied the database files (server and backup), rebooted the machine and it picked up the new/old database
this machine was running 2.0.8.1 (2.0.8.1_beta_2024-05-07) before the update
checked the other (older) mac i can still update to this version of duplicati, that is currently running 2.0.8.1
and the database is also in .config
anything you want me to try on this machine?
used the opportunity to update this with some screenshots
Then it clearly did not detect the old version. I have re-opened the issue.
Can you wait a few days? Then I would like to try once again to fix the issue, and release a new version that correctly detects the old location.
sure, that’s why i asked
I tried to replicate this issue with my debugger attached on MacOS. It works exactly as I described above, picking up the ~/.config/Duplicati/Duplicati-server.sqlite file, until I place a file in ~/Library/Application Support/Duplicati/Duplicati-server.sqlite.
It may have been fixed in the latest source code as I consolidated some code that was looking for the right configuration folder in two different ways.
I will try to fix the Ubuntu %HOME% issue you found and then I will make a new canary build (2.0.9.105).
Before you install 2.0.9.105 on your other system, can you please check if you have these files prior to installing the new version:
~/.config/Duplicati/Duplicati-server.sqlite~/Library/Application Support/Duplicati/Duplicati-server.sqliteYou should have the first one and not have the second one if you have not tried a version after 2.0.8.1.
2.0.9.105 just came out, but I got an earlier look at a 2.0.9.104 issue that came up occasionally.
Bringing my Windows 10 PC out of sleep in the morning would sometimes require a login again:
http://localhost:8200/ngax/index.html
Watching in Edge developer tools, it looks like a refresh got a 401 error.
{“Error”:“Failed to refresh token”,“Code”:401}
I don’t know if it matters, but I’ve accumulated lots of cookies on localhost:
RefreshToken_8200
RefreshToken_8201
RefreshToken_8300
current-theme
default-theme
xsrf-token
EDIT:
is the view after wake. I clear the network log before sleep.
I also have Firefox (Windows, Linux), Chrome (Windows), and Chromium (Linux) trying to repro.
It could be the anti-theft logic that is somehow tripping it, where the server refreshes the cookie, but the request is lost, so the next request fails.
If you discover it again, can you see what the server/trayicon is logging as the error? It would reveal why the token is considered invalid.
Does not matter, the server only looks at the cookie that matches the port number.
Fortunately I’ve been starting trayicon from a bat file, and added a pause at end to keep it open.
Nothing is timestamped, terminal can’t be cleared, but below is likely since sometime yesterday:
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
An unhandled exception has occurred while executing the request.
Duplicati.WebserverCore.Exceptions.ServerErrorException: Failed to import backup: Missing passphrase
at Duplicati.WebserverCore.Endpoints.V1.Backups.ExecuteImport(Connection connection, Boolean cmdline, Boolean import_metadata, Boolean direct, String passphrase, String tempfile)
at Duplicati.WebserverCore.Endpoints.V1.Backups.<>c.<Map>b__0_2(ImportBackupInputDto input, IJWTTokenProvider jWTTokenProvider, Connection connection, IHttpContextAccessor httpContextAccessor)
at lambda_method237(Closure, EndpointFilterInvocationContext)
at Duplicati.WebserverCore.Middlewares.HostnameFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Duplicati.WebserverCore.Middlewares.LanguageFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|129_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<>c__DisplayClass102_2.<<HandleRequestBodyAndCompileRequestDelegateForJson>b__2>d.MoveNext()
--- End of stack trace from previous location ---
at Duplicati.WebserverCore.Middlewares.WebsocketExtensions.<>c__DisplayClass0_0.<<UseNotifications>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)
fail: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[3]
Exception occurred while processing message.
Duplicati.WebserverCore.Exceptions.UnauthorizedException: Token family not found
at Duplicati.WebserverCore.Services.TokenFamilyStore.GetTokenFamily(String userId, String familyId, CancellationToken ct)
at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ValidateAccessToken(TokenValidatedContext context, ITokenFamilyStore store)
at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
fail: Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HN6300HGFDUU", Request id "0HN6300HGFDUU:00000004": An unhandled exception was thrown by the application.
Duplicati.WebserverCore.Exceptions.UnauthorizedException: Token family not found
at Duplicati.WebserverCore.Services.TokenFamilyStore.GetTokenFamily(String userId, String familyId, CancellationToken ct)
at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ValidateAccessToken(TokenValidatedContext context, ITokenFamilyStore store)
at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.AuthenticateAsync()
at Microsoft.AspNetCore.Authentication.AuthenticationService.AuthenticateAsync(HttpContext context, String scheme)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
An unhandled exception has occurred while executing the request.
Duplicati.WebserverCore.Exceptions.UnauthorizedException: Failed to refresh token
at Duplicati.WebserverCore.Endpoints.V1.Auth.<>c.<<Map>b__3_0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<TaskOfTToValueTaskOfObject>g__ExecuteAwaited|92_0[T](Task`1 task)
at Duplicati.WebserverCore.Middlewares.HostnameFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Duplicati.WebserverCore.Middlewares.LanguageFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|129_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
at Duplicati.WebserverCore.Middlewares.WebsocketExtensions.<>c__DisplayClass0_0.<<UseNotifications>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)
[WebSocket] Client closed connection prematurely.
[WebSocket] Client closed connection prematurely.
[WebSocket] Client closed connection prematurely.
[WebSocket] Client closed connection prematurely.
[WebSocket] Client closed connection prematurely.
[WebSocket] Client closed connection prematurely.
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
An unhandled exception has occurred while executing the request.
Duplicati.WebserverCore.Exceptions.UnauthorizedException: Failed to refresh token
at Duplicati.WebserverCore.Endpoints.V1.Auth.<>c.<<Map>b__3_0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<TaskOfTToValueTaskOfObject>g__ExecuteAwaited|92_0[T](Task`1 task)
at Duplicati.WebserverCore.Middlewares.HostnameFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Duplicati.WebserverCore.Middlewares.LanguageFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|129_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
at Duplicati.WebserverCore.Middlewares.WebsocketExtensions.<>c__DisplayClass0_0.<<UseNotifications>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
An unhandled exception has occurred while executing the request.
Duplicati.WebserverCore.Exceptions.UnauthorizedException: Failed to refresh token
at Duplicati.WebserverCore.Endpoints.V1.Auth.<>c.<<Map>b__3_0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<TaskOfTToValueTaskOfObject>g__ExecuteAwaited|92_0[T](Task`1 task)
at Duplicati.WebserverCore.Middlewares.HostnameFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Duplicati.WebserverCore.Middlewares.LanguageFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|129_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
at Duplicati.WebserverCore.Middlewares.WebsocketExtensions.<>c__DisplayClass0_0.<<UseNotifications>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
An unhandled exception has occurred while executing the request.
Duplicati.WebserverCore.Exceptions.UnauthorizedException: Failed to refresh token
at Duplicati.WebserverCore.Endpoints.V1.Auth.<>c.<<Map>b__3_0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<TaskOfTToValueTaskOfObject>g__ExecuteAwaited|92_0[T](Task`1 task)
at Duplicati.WebserverCore.Middlewares.HostnameFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Duplicati.WebserverCore.Middlewares.LanguageFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|129_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
at Duplicati.WebserverCore.Middlewares.WebsocketExtensions.<>c__DisplayClass0_0.<<UseNotifications>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task)
[WebSocket] Client closed connection prematurely.
[WebSocket] Client closed connection prematurely.
[WebSocket] Client closed connection prematurely.
Excellent!
The message Token family not found indicates that for some reason the token has been revoked. This can either happen with logout, expiration, or re-use.
We can rule out that you performed the logout, and if it was re-use, there should be messages like eitherInvalid refresh token counter or Token family re-use detected, but these are not present.
The refresh token is valid for 30 days, so we should not trigger any expirations either.
I will try to replicate with a VM where I can play with the clock. I have added an issue tracking the logout.