Is there a way to run partial / limited compact? If not, what’s the recommended way to dealing with huge backup sets where standard compaction is taking ages? My suggestion is separating the compaction filter level (what’s being compacted) and compaction trigger (when compaction is started). This…

For now the best you could do is increase the default value for --threshold, but that’s just the start trigger - there is no “only compact until 15% or less is wasted” end trigger. --threshold As files are changed, some data stored at the remote destination may not be required. This option contro…

Yes, I think --threshold should do what you want. The threshold parameter is used to determine if a volume is considered “wasteful”, so if you set it high, like 95, it should not download the dblock unless it holds 95% unused data: duplicati/duplicati/blob/master/Duplicati/Library/Main/Da…

If --threshold=100 is set, does that effectively disable compacting - even if a manual compact is run?

Ok, treshold=100 practically disables compacting for non-fully deletable volumes. After this deleting fully deletable volumes is quick. But it disables threshold based partial compacting, which then needs to be run separately. How does Duplicati behave in a situation where I run compact separately,…

[image] Sami_Lehtinen: How does Duplicati behave in a situation where I run compact separately, and just kill it based on timer? The worst that SHOULD happen is you have some leftover local temp files that may not have gotten cleaned up correctly. The compacting process basically: download…

Well, next run should clean-up the temp files. Of course there’s no way to deal with temp files if process is killed. But next run should take care of left-overs, if program is working correctly. But back to the compaction process. I just tested it, it seems that the auto-compact also compacts ALL …

[image] Sami_Lehtinen: it seems that the auto-compact also compacts ALL files that could be compacted, even a little bit. I always assumed it only compacted files that met the wasted space threshold, resulting in (eventually) enough small files for them to be re-compacted into fewer files.

[image] Sami_Lehtinen: But back to the compaction process. I just tested it, it seems that the auto-compact also compacts ALL files that could be compacted, even a little bit. If you still recall after some time has passed, could you please describe the test method that indicated this? Testin…

[image] ts678: If you still recall after some time has passed, could you please describe the test method that indicated this? I didn’t make any specific test set, I just observed operation. But actually when I encountered that other thread it made me think about that dblock size thread, it co…

@JonMikelV If I can go back to what you initially misunderstood in tghis discussion I actually have a question related to database compacting: I have a daily backup of our company storage (about 1M files) running 6 days / week with 1 year retention. The sqlite database has grown over time to abou…

Duplicati

Compact - Limited / Partial

Features

show post in topic

Home
Categories
Guidelines
Terms of Service
Privacy Policy

Powered by Discourse, best viewed with JavaScript enabled