Unix Tip: Experimenting with rsync

January 23, 2008, 01:42 PM —  ITworld.com — 

Given the comments and questions I've received since last week's column on using rsync to move a file system from one disk to another, I thought we might run through some simple exercises to gain more insight into how rsync works. Two of the problems that I have seen people running into when using rsync to copy collections of files from one place to another are 1) failing to retain symbolic and hard links and 2) misaligning the source and destination directories such that the copies don't look as intended. Here we'll create a directory with a number of files, copy the directory with an assortment of rsync (and other) commands, take a closer look at some of rsync's command options and then use rsync to verify that our replicated directory is intact.

First, to create a sample directory to be copied, we're going to use a smattering of common Unix commands, taking care to include a variety of file types.

testDir=" rsyncTest"
mkdir $testDir
touch $testDir/emptyfile
cp /etc/motd $testDir/textfile
mkdir $testDir/dir
mknod $testDir/devfile c 11 11
ln -s /etc/hosts $testDir/symlink
ln $testDir/textfile $testDir/link2textfile

In the commands above, we've created a small directory containing some regular files (one empty, one containing text), a directory, a special device file, a symbolic link and a hard link. A long listing displays the directory's contents. Notice that I've included the -i option so that I can see the inode used for each of the files.

> ls -li $testDir
total 8
   7865105 crw-r--r--   1 root     other     11, 11 Jan 22 12:01 devfile
   7865103 drwxr-xr-x   2 root     other        512 Jan 22 12:01 dir
   7865089 -rw-r--r--   1 root     other          0 Jan 22 12:01 emptyfile
   7865090 -rw-r--r--   2 root     other        233 Jan 22 12:01 link2textfile
   7865107 lrwxrwxrwx   1 root     other         10 Jan 22 12:01 symlink -> /etc/hosts
   7865090 -rw-r--r--   2 root     other        233 Jan 22 12:01 textfile

Now, let's copy this directory to another location using an rsync command. Notice we are using the -av options. The -a indicates that we want archive (use recursion) mode and the -v is verbose. I'm omitting the -z (compress) option in this case since this isn't going to add value when copying simple directory. In fact, compression will probably only slow down local copies and is likely only of value when a lot of data will be sent over a network. In some tests I ran, the copy took approximately five times longer when the -z option was used for a local copy.

I like it!
Post a comment
The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
Resources
White Paper

Symantec Backup Exec 12 and Backup Exec System Recovery 8 deliver industry leading Windows data protection and system recovery. Download this whitepaper to find out the top reasons to upgrade and how to get continuous data protection and complete system recovery.

Webcast

Data and system loss — from a hard drive failure, malicious attack, natural disaster, or simple human error — can happen anytime. Don’t leave your business vulnerable. Make sure you have a secure recovery strategy in place. Symantec's latest backup and system recovery technology can efficiently restore critical applications, individual emails and documents and even restore your entire system in minutes in the event of a loss.

White Paper

Businesses face a growing challenge to ensure that the IT environment is properly protected. Backup Exec 12 integrates with other applications in the Symantec family of products, to complement your current data protection strategy, keep your data securely backed up and make it recoverable when you need it most.

Free stuff

Enterprise 2.0 Implementation
By Aaron C. Newman, Jeremy Thomas
Published by McGraw-Hill
Learn more!

Deploying Cisco Wide Area Application Services
By Zach Seils, Joel Christner
Published by Cisco Press
Learn more!

Featured Sponsor

AISO founders envisioned a Web hosting company that was environmentally friendly. While the company employed energy-efficient innovations like solar panels, its infrastructure produced unacceptable power and cooling requirements. Find out how AISO leveraged AMD technology to overcome their challenge in this case study white paper.

In this whitepaper, Scalar explores the opportunity to change the landscape with respect to mission critical databases built around Oracle. Leveraging technologies such as Linux, high-end commodity processing power and Oracle RAC technology to architect, design, build and maintain database infrastructure that delivers maximum availability, reliability and performance at a fraction of traditional cost.

On a typical day, weather.com, the Web site for The Weather Channel in Atlanta, serves up between 15 million and 20 million page views. But in September 2004, when back-to-back hurricanes ransacked Florida, the peak traffic on one day more than tripled: over 70 million page views by more than 7 million unique visitors. Read the full success story now.

More Resources