FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu User

 
 
LinkBack Thread Tools
 
Old 02-28-2009, 08:46 PM
Vincent Trouilliez
 
Default rsync : how to exclude specific files or folders ?

Hello,

I am using rsync currently, to backup my home directory every evening.
I would like to exclude a few large folders so as to speed up the backup
by an order (or two) of magnitude.

The problem is that although it should be simple.. it isn't.
I read the man page several times, and it's not very clear what the
syntax is, nor if individual folders to be exluced can be specified
directly at the command line, or if they need to be specified in a
separate "black list" file, and what the syntax in this file should be
like.

So does anyone know (for sure, from past or present
experience) how to word it so rsync understands ?

To be specific, I am trying to exlude my virtual machines (15GB) and
googlearth database (1.1GB) :

/home/vincent/.VirtualBox
/home/vincent/.googleearth


Regards,

--
Vince

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-28-2009, 09:22 PM
Hal Burgiss
 
Default rsync : how to exclude specific files or folders ?

On Sat, Feb 28, 2009 at 10:46:06PM +0100, Vincent Trouilliez wrote:
>
> The problem is that although it should be simple.. it isn't.
> I read the man page several times, and it's not very clear what the
> syntax is, nor if individual folders to be exluced can be specified
> directly at the command line, or if they need to be specified in a
> separate "black list" file, and what the syntax in this file should be
> like.
>
> So does anyone know (for sure, from past or present
> experience) how to word it so rsync understands ?
>
> To be specific, I am trying to exlude my virtual machines (15GB) and
> googlearth database (1.1GB) :
>
> /home/vincent/.VirtualBox
> /home/vincent/.googleearth

This is my solution. And yes the syntax is onery. Use rsync with the
-F option. It think will use a .rsync-filter file. I place this in /,
but I backup most of the system. By default everything is included, so
I do simple exclusions. Some examples, that work for me:

- /mnt/media

- /var/tmp/*
- /var/cache/*
- */_htdocs/
- */logs/*

The '*' is a wildcard and will ignore a directory named _htdocs
anywhere it finds one.

There are other ways to use filters, but I found this and it works. So
something like this should work:

- /home/vincent/.VirtualBox
- /home/vincent/.googleearth

--
Hal


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-28-2009, 10:07 PM
Vincent Trouilliez
 
Default rsync : how to exclude specific files or folders ?

On Sat, 28 Feb 2009 17:22:15 -0500
Hal Burgiss <hal@burgiss.net> wrote:

> This is my solution. And yes the syntax is onery. Use rsync with the
> -F option. It think will use a .rsync-filter file. I place this in /,
> but I backup most of the system. By default everything is included, so
> I do simple exclusions. Some examples, that work for me:
>
> - /mnt/media
>
> - /var/tmp/*
> - /var/cache/*
> - */_htdocs/
> - */logs/*
>
> The '*' is a wildcard and will ignore a directory named _htdocs
> anywhere it finds one.
>
> There are other ways to use filters, but I found this and it works. So
> something like this should work:
>
> - /home/vincent/.VirtualBox
> - /home/vincent/.googleearth


Oh ! After reading your mail, and re-re-re-reading the man page, I ventured into this:

$rsync <options> --exclude '*.VirtualBox' --exclude '*.googleearth' <source> <destination>

and it worked ! It didn't backup the directories, yeehaa !! :-)
So it was possible to do it directly from the command line then, without a filter file, cool :-)


--
Vince, happy chappy...

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-28-2009, 10:10 PM
NoOp
 
Default rsync : how to exclude specific files or folders ?

On 02/28/2009 01:46 PM, Vincent Trouilliez wrote:
> Hello,
>
> I am using rsync currently, to backup my home directory every evening.
> I would like to exclude a few large folders so as to speed up the backup
> by an order (or two) of magnitude.
>
> The problem is that although it should be simple.. it isn't.
> I read the man page several times, and it's not very clear what the
> syntax is, nor if individual folders to be exluced can be specified
> directly at the command line, or if they need to be specified in a
> separate "black list" file, and what the syntax in this file should be
> like.
>
> So does anyone know (for sure, from past or present
> experience) how to word it so rsync understands ?
>
> To be specific, I am trying to exlude my virtual machines (15GB) and
> googlearth database (1.1GB) :
>
> /home/vincent/.VirtualBox
> /home/vincent/.googleearth
>

You may want to give grsync a try - it's a gui frontend for rsync. In
grsync you can also easily run simulations (as you can from the command
line of course) to see first. I don't know about rsync from the command
line (man rsync would be your friend there), but in grsync I'd create an
exclude file:

/home/<user>/.grsync/exclude.txt
and the file would include:

.VirtualBox
.googleearth

Then in the Additional Options tab I' just put:
--exclude-from=/home/<user>/.grsync/exclude.txt

http://ubuntuforums.org/showthread.php?t=795668
[HowTo: Use Grsync and OpenSSH to sync your /home directory over a network]

For the command line you might find this helpful:
http://articles.slicehost.com/2007/10/10/rsync-exclude-files-and-folders
http://sial.org/howto/rsync/
http://www.ss64.com/bash/rsync.html


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-28-2009, 10:33 PM
"H.S."
 
Default rsync : how to exclude specific files or folders ?

Vincent Trouilliez wrote:
> Hello,
>
> I am using rsync currently, to backup my home directory every evening.
> I would like to exclude a few large folders so as to speed up the backup
> by an order (or two) of magnitude.
>
> The problem is that although it should be simple.. it isn't.
> I read the man page several times, and it's not very clear what the
> syntax is, nor if individual folders to be exluced can be specified
> directly at the command line, or if they need to be specified in a
> separate "black list" file, and what the syntax in this file should be
> like.
>
> So does anyone know (for sure, from past or present
> experience) how to word it so rsync understands ?
>
> To be specific, I am trying to exlude my virtual machines (15GB) and
> googlearth database (1.1GB) :
>
> /home/vincent/.VirtualBox
> /home/vincent/.googleearth
>

Fast solution, exclude using --exclude=. For example, in your case, use
the following additional flags and see what happens:
-n --exclude="/home/vincent/.VirtualBox"
--exclude="/home/vincent/.googleearth"

That "-n" will just do a dry run. Remove that flag when you are
satisfied with what will be transfered.

Alternative is to use an excludes file. Use the following options
assuming you have exclude patterns in /path/to/excludes.txt
--exclude-from=/path/to/excludes.txt

where excludes.txt has the lines:
#-----------------------------------------
#exclude some large unnecessary files/dirs
- .VirtualBox
- .googlearth
#-----------------------------------------

These two lines exclude any similarly named file or folder irrespective
of their paths.

Search google for more information about exclude and include files.

Good luck.



--

Please reply to this list only. I read this list on its corresponding
newsgroup on gmane.org. Replies sent to my email address are just
filtered to a folder in my mailbox and get periodically deleted without
ever having been read.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-28-2009, 11:48 PM
Matthew Flaschen
 
Default rsync : how to exclude specific files or folders ?

Vincent Trouilliez wrote:
> Oh ! After reading your mail, and re-re-re-reading the man page, I ventured into this:
>
> $rsync <options> --exclude '*.VirtualBox' --exclude '*.googleearth' <source> <destination>
>
> and it worked !

Yes, and if you want to switch from a chain of exclude parameters to a
file (e.g. if the command line starts getting long), you can just do:

rsync <options> --exclude-from=my_exclude_file <source> <destination>

And my_exclude_file is then just:

*.VirtualBox
*.googleearth

Exactly the same format.

Matt Flaschen

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 03-01-2009, 12:47 AM
Vincent Trouilliez
 
Default rsync : how to exclude specific files or folders ?

On Sat, 28 Feb 2009 19:48:33 -0500
Matthew Flaschen <matthew.flaschen@gatech.edu> wrote:

> Yes, and if you want to switch from a chain of exclude parameters to a
> file (e.g. if the command line starts getting long), you can just do:
>
> rsync <options> --exclude-from=my_exclude_file <source> <destination>
>
> And my_exclude_file is then just:
>
> *.VirtualBox
> *.googleearth
>
> Exactly the same format.

Ah ok, thanks, will keep that in mind.


--
Vince, sorted and happy.

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 03-01-2009, 01:19 AM
"H.S."
 
Default rsync : how to exclude specific files or folders ?

Matthew Flaschen wrote:
> Vincent Trouilliez wrote:
>> Oh ! After reading your mail, and re-re-re-reading the man page, I ventured into this:
>>
>> $rsync <options> --exclude '*.VirtualBox' --exclude '*.googleearth' <source> <destination>
>>
>> and it worked !
>
> Yes, and if you want to switch from a chain of exclude parameters to a
> file (e.g. if the command line starts getting long), you can just do:
>
> rsync <options> --exclude-from=my_exclude_file <source> <destination>
>
> And my_exclude_file is then just:
>
> *.VirtualBox
> *.googleearth


The asterisks are needed. ".foo" will match a file any file called .foo
in any path. See first rule in section EXCLUDE PATTERNS in rsync man page.

Regards.




--

Please reply to this list only. I read this list on its corresponding
newsgroup on gmane.org. Replies sent to my email address are just
filtered to a folder in my mailbox and get periodically deleted without
ever having been read.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 03-01-2009, 07:39 PM
Smoot Carl-Mitchell
 
Default rsync : how to exclude specific files or folders ?

On Sat, 2009-02-28 at 21:19 -0500, H.S. wrote:
> Matthew Flaschen wrote:
> > Vincent Trouilliez wrote:
> >> Oh ! After reading your mail, and re-re-re-reading the man page, I ventured into this:
> >>
> >> $rsync <options> --exclude '*.VirtualBox' --exclude '*.googleearth' <source> <destination>
> >>
> >> and it worked !
> >
> > Yes, and if you want to switch from a chain of exclude parameters to a
> > file (e.g. if the command line starts getting long), you can just do:
> >
> > rsync <options> --exclude-from=my_exclude_file <source> <destination>
> >
> > And my_exclude_file is then just:
> >
> > *.VirtualBox
> > *.googleearth
>
>
> The asterisks are needed. ".foo" will match a file any file called .foo
> in any path. See first rule in section EXCLUDE PATTERNS in rsync man page.

Not quite correct although it is a bit confusing. The above will match
any file which ends with .VirtualBox or .googleearth. If you start your
transfer from say, /home/user, and want to exclude the above directories
at that level in the hierarchy then the precise syntax would be:

/.VirtualBox
/.googleearth

The leading "/" says to match only at the root of the transfer. Any
instance of .VirtualBox which appears anywhere else will be included.

As a practical matter, it likely does not matter, since it is unlikely
you have another file or directory in your home directory
called .VirtualBox or .googleearth. As another poster indicated you can
see what rsync does this with the --dry-run and --verbose option.

For more complex filtering take a look at the --filter option. It
allows all kinds of sophisticated file inclusion/exclusion.
--
Smoot Carl-Mitchell
Computer Systems and
Network Consultant
smoot@tic.com
+1 480 922 7313
cell: +1 602 421 9005

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 03-01-2009, 11:56 PM
"H.S."
 
Default rsync : how to exclude specific files or folders ?

H.S. wrote:
> Matthew Flaschen wrote:
>> Vincent Trouilliez wrote:
>>> Oh ! After reading your mail, and re-re-re-reading the man page, I ventured into this:
>>>
>>> $rsync <options> --exclude '*.VirtualBox' --exclude '*.googleearth' <source> <destination>
>>>
>>> and it worked !
>> Yes, and if you want to switch from a chain of exclude parameters to a
>> file (e.g. if the command line starts getting long), you can just do:
>>
>> rsync <options> --exclude-from=my_exclude_file <source> <destination>
>>
>> And my_exclude_file is then just:
>>
>> *.VirtualBox
>> *.googleearth
>
>
> The asterisks are needed. ".foo" will match a file any file called .foo

I had meant to write they are *not* needed to match all such named
files. Sorry for the omission.




--

Please reply to this list only. I read this list on its corresponding
newsgroup on gmane.org. Replies sent to my email address are just
filtered to a folder in my mailbox and get periodically deleted without
ever having been read.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 

Thread Tools




All times are GMT. The time now is 01:40 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org