--exclude not working as expected

[blueray453]

I have checked https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-patterns

borg create --exclude 'pp:/media/ismail/8TBRaid0/lost+found' --stats --progress --show-rc --show-version /media/ismail/WDPurple8TB/8TBRaid0_Backup::{hostname}-{now:%Y-%m-%d_%H:%M:%S} /media/ismail/8TBRaid0

The above command shows

borgbackup version 1.1.15
/media/ismail/8TBRaid0/lost+found: scandir: [Errno 13] Permission denied: '/media/ismail/8TBRaid0/lost+found'

so it did not exclude the directory.

What is the issue here?

How to exclude lost+found with relative path instead of absolute path?

[ThomasWaldmann]

BTW, you were reading the wrong docs, for 1.1.15 you need to read:

https://borgbackup.readthedocs.io/en/1.1.15/usage/help.html

Can you try without the leftmost /?

[blueray453]

Thank you very much.

[blueray453]

Sorry for bothering you again. My script looks like:

export BORG_LOGGING_CONF="/home/ismail/.config/borg/logging-8TBRaid0.conf";
export BORG_CACHE_DIR=/media/ismail/WDPurple8TB/Borg_Cache;
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes;

borg create --pattern=!/media/ismail/8TBRaid0/lost+found --pattern=!/media/ismail/8TBRaid0/_QEMU_Backup --stats --progress --show-rc --show-version /media/ismail/WDPurple8TB/8TBRaid0_Backup::{hostname}-{now:%Y-%m-%d_%H:%M:%S} /media/ismail/8TBRaid0;

It works.

But when I replace --pattern=!/media/ismail/8TBRaid0/lost+found --pattern=!/media/ismail/8TBRaid0/_QEMU_Backup with --patterns-from /media/ismail/WDPurple8TB/include-exclude-pattern.txt where include-exclude-pattern.txt looks like:

! /media/ismail/8TBRaid0/lost+found
! /media/ismail/8TBRaid0/_QEMU_Backup

it does not work. To be more specific, i think it does not even find the include-exclude-pattern.txt file. What might be the reason here?

[ThomasWaldmann]

Did you try without the leftmost "/" as I suggested?

[blueray453]

yes i tried --exclude 'pp:media/ismail/8TBRaid0/lost+found' and it worked.

Then i tried

export BORG_CACHE_DIR=/media/ismail/WDPurple8TB/Borg_Cache;
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes;

borg create --pattern=!/media/ismail/8TBRaid0/lost+found --pattern=!/media/ismail/8TBRaid0/_QEMU_Backup --stats --progress --show-rc --show-version /media/ismail/WDPurple8TB/8TBRaid0_Backup::{hostname}-{now:%Y-%m-%d_%H:%M:%S} /media/ismail/8TBRaid0;

it also worked.

but when i try

export BORG_CACHE_DIR=/media/ismail/WDPurple8TB/Borg_Cache;
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes;

borg create --patterns-from /media/ismail/WDPurple8TB/include-exclude-pattern.txt --stats --progress --show-rc --show-version /media/ismail/WDPurple8TB/8TBRaid0_Backup::{hostname}-{now:%Y-%m-%d_%H:%M:%S};

where /media/ismail/WDPurple8TB/include-exclude-pattern.txt looks like:

+ media/ismail/8TBRaid0
! media/ismail/8TBRaid0/lost+found
! media/ismail/8TBRaid0/_QEMU_Backup

or,

+ /media/ismail/8TBRaid0
! /media/ismail/8TBRaid0/lost+found
! /media/ismail/8TBRaid0/_QEMU_Backup

it does not work.

[ThomasWaldmann]

BTW, you know that WD purple series is for surveillance purposes (recording video - constant bitrate is more important than correcting bit errors) and not recommended for data backup purposes?

[blueray453]

When i went to purchase, wd-red was not available. So, i purchased two wd-purples, one is 6TB another 8TB (this one). My current data is in raid0 (two 4tb wd).

Once I used to have no backup. Then my HDD went belly up. To recover the data i had to spend the price of four 8TB wd-red, so now i keep multiple backups.

[ThomasWaldmann]

Yes, it is likely better than no backup. But be aware that this hdd is made to rather let bit errors go through uncorrected than try an error correction that would take a bit more time. It's made for recording/replaying video, where an error in one frame does not matter much, but stuttering video due to time consuming error correction would be a problem.

For backups, you need the opposite.

[blueray453]

Yes i know. I created a vm (40GB) in one of these HDD and it kept breaking due to faulty sata cable? Anyways, which HDD do you recommend?

[ThomasWaldmann]

Considering that HDD manufacturer sell a lot of SMR HDDs these days (often without even declaring it clearly as such), it is hard to recommend anything. Sometimes the implementation of SMR is crappy to the extend that it produces more than only performance problems (seen that on 2.5" Seagate USB HDDs).

What I liked most recently were helium-filled server/datacenter HDDs from HGST / WD / Toshiba. These are quite silent (when not seeking) and run rather cool, consume little electricity. But they are made for 24/7 so not sure how well they would cope e.g. as desktop or external HDD.

But in any case, for data storage purposes avoid any HDD that is declared for video / surveillance purposes.

[blueray453]

Thank you very much for the guideline.

[ThomasWaldmann]

Closing this as outdated: it has seen no activity for a long time and predates the current borg releases.

If this still affects you on a current version - borg 1.4.x (stable) or the borg 2.0.0 beta - please open a new discussion with up-to-date details.