|
|
|
|
|
|
|
|
by Yusuf Nagree
This article describes backing up files on a Linux system. Two programs are described---tar and taper. The first program is available from the Free Software Foundation under the GNU license and is included with most distributions of Linux. The second program is written by the author of this article and provides a more user friendly interface. It is also available under the GNU license and thus is freely available. Note that this article is not meant to be a full reference for either package, but merely an introduction to get you started. For full details, see the documentation that comes with each package.
Nearly every form and clone of Unix (as well as other operating systems) comes with some version of tar. It is a standard program, and archives made on one machine should always be usable on other machines. The real problem with tar is that there is virtually no user-interface at all. All operations must be done via command line switches.
tar can make backups to a hard disk file or to a tape drive as well as over a NFS link (which we won't cover here). The files to be backed up can be compressed using GNU gzip (or compress).
To make a backup, the basic form is:
$ tar [options] files_to_backup_or_restore
The most commonly used options are:
Thus, to create a compressed backup of your /etc directory in a file called etc_backup.tar, you would do:
$ tar czf etc_backup.tar /etc
Note that all subdirectories under /etc will be backed up as well.
If you now want to add the contents of /usr/local/etc, you would do: $ tar rzf etc_backup.tar /usr/local/etc
Suppose that you have now made some changes to the files, but not all of them. You can do:
$ tar uzf etc_backup.tar /etc /usr/local/etc
and tar will go through and append to the archive only those files that have been changed since the archive was originally created.
The above examples apply to backing up to a file on the hard disk. Backing up to a tape drive simply involves giving the filename of the tape device, usually /dev/ftape for floppy tape drives and /dev/st0 for SCSI tape drives.
The two options that are relevant here are:
$ tar xzf etc_backup.tar
Note that tar does not put the files back where they came from, but rather creates a new tree based on the current directory. For example, if you were in the /usr/home/john directory when you issued the above command, you will find that a new subdirectory /usr/home/john/etc has been created and all the files are in that subdirectory. If you wish to restore the files whence they came:
$ cd /
$ tar xzf etc_backup.tar
Note that doing this is very dangerous, since old files are over-written without warning. This can have dire consequences if not used properly. It is often much safer to restore in your home directory or /tmp and then copy the files to their correct location after you have checked that nothing horrible will happen.
To restore an individual file or directory, simply specify the name after all the tar arguments. For example, to restore just the hosts and the passwd file:
$ tar xzf etc_backup.tar etc/hosts etc/passwd
Note that the full pathname (excluding the leading /, which tar explicitly does not store) needs to be specified.
Most people who write software for Linux and make it available to others distribute it via tar files. For example, say that you have downloaded a new game, best_game-1.3.tar.gz. To install (restore) this game in your home directory:
$ cd
$ tar xzf best_game-1.3.tar.gz
The game and all its applicable files will then be restored. If the author of the program has followed normal convention, all the files will be in a directory called best_game-1.3. Note that when restoring files from an unknown source, it is a very good idea to restore the files in your home directory, examine the files, and then when sure everything is correct, move them to the location suggested by the author. This way, you will avoid inadvertent file overwrites. It is also best to first use the t option to see whether the author has put the files in a subdirectory. If not, make a subdirectory and use it:
$ cd
$ mkdir worst_game-0.1
$ cd worst_game-0.1
$ tar xzf worst_game-0.1.tar.gz
Some of the other options that tar supports are:
$ date > last_backup
Although most backups are done on tape drives, it is sometimes useful to use your floppy drive (which can be very slow and frustrating!). To use your floppy, give tar the filename /dev/fd0 for the first floppy (drive A: in DOS parlance) and /dev/fd1 for the second floppy (drive B:). So, to make a backup of your /etc directory to ``drive A:''
$ tar czf /dev/fd0 /etc
Note that the existing contents of the floppy will be totally overwritten. When using floppy drives, the -M option is very handy since after one floppy is full, tar will automatically prompt for the next floppy.
If you use ftape, you cannot append files to an existing tar backup. Therefore, if you make a backup that occupies only 10 MB, you will have wasted the rest of the tape. Fortunately, there is a way of using the rest of the tape, using the mt program. After tar has written a backup to the tape, it writes two EOF marks on the tape. Schematically, your tape now looks like:
You can tell mt to advance to these EOF marks by
$ mt -f /dev/nftape fsf 1
Note that you must use the non-rewinding device (/dev/nftape), because if you don't, after mt has repositioned the tape, it will automatically be rewound and your repverbose mode positioning will be lost. The fsf 1 is an mt command that advances the tape to the first EOF mark.
Your tape will now be positioned at the end of the first backup, and you can create another backup at this position:
$ tar czf /dev/ftape files_for_backup_2
Because you used the rewinding device, the tape will rewind after the tar backup has been written. Your tape will look like this:
Should you wish to add a third backup, you can do the following:
$ mt -f /dev/nftape fsf 2<\ $ tar czf /dev/ftape files_for_backup_3
The fsf 2 advances past two EOF marks---hence, past the first two backups. Your tape will look like:
and will be positioned at the beginning of the tape because you used the the rewinding device to do the backup.
If you wish to restore files from the first backup, you can do the following:
$ mt -f /dev/ftape rewind<\ $ tar xzf /dev/ftape files_from_backup_1
This first line ensures that the tape has been rewound. The second line then restores files from the first backup. If you wish to get files from the second backup, do the following:
$ mt -f /dev/ftape rewind $ mt -f /dev/nftape fsf 1 $ tar xzf /dev/ftape file_from_backup_2
Similarly, to restore files from the third backup:
$ mt -f /dev/ftape rewind $ mt -f /dev/nftape fsf 2 $ tar xzf /dev/ftape files_from_backup_3
You can continue adding backups until your tape is full. Note that strictly speaking, you do not need to rewind the tape each time---if you are sure that the tape is rewound, you can skip the rewind command; however, I would recommend that you always rewind prior to use to avoid problems. If you are already at the beginning of the tape, the rewind command will not take any time at all. You are likely to get into all sorts of bother if you assume that the tape is rewound when it is not. You can find out where your tape actually is by:
$ mt -f /dev/nftape status
As you can imagine, maintaining backups this way is awkward and very time-consuming since if you do not know what is on the third backup, you have to rewind your tape, advance to the third backup and then read what is there.
Another problem with tar is that, when using the compression mode, tar compresses all the files and then writes them to the tape. This leads to a very serious problem. If your tape somehow becomes corrupted, you lose all the files after the corruption occurred.
Because of the above, and because of tar's command-line user-interface, the author was tempted to fill the gap with a user-friendly backup program---hence, taper was born...
Although tar is a very powerful and flexible program, it has no friendly user interface. taper provides many of the same features as tar, but provides a nice user-friendly interface.
See below for locations of the taper sources. At the time this article was written, the current version was 5.4. taper requires a recent version (1.9.6 or greater) of ncurses that supports ``forms'', which can be retrieved from any GNU mirror site or from the primary GNU ftp site, prep.ai.mit.edu. It is easy to configure, build, and install for Linux; installation instructions are included in the INSTALL file. As of this writing, the current version of ncurses is 1.9.7a and can be found in the file ncurses-1.9.7a.tar.gz.
To build and install the latest ncurses, I did the following as root:
$ cd /usr/local/sr $ tar xzf ncurses-1.9.7a.tar.gz $ cd ncurses-1.9.7a $ ./configure --with-normal --with-shared \ --with--debug --disable-termcap $ make $ make install
The steps you need to take may be slightly different; the INSTALL file gives plenty of details. The --with-shared option is especially important if you have ELF libraries and don't remove previous versions---otherwise, compiling taper may not work.
To make a binary of taper, issue the following commands:
$ tar xzf taper-xx.xx.tar.gz
where xx.xx is the the version that you have obtained. Next, you may need to edit the Makefile (e.g., if you are using a SCSI drive, remove the [cw]#[ecw] that is in front of the line -DHAVE_SCSI, or if you are using the new zftape driver, remove the # that is in front of the line -DUSING_ZFTAPE---see the Makefile for more details). Then type:
$ make clean $ make all $ make install
By default, the programs will have been installed in /sbin and the manual page in /usr/man/cat1. You can change these if you wish by appropriately editing the Makefile. If you do not have write permission to these directories, the files will remain in the current directory and will not be copied across.
Some terminology first:
When you create an archive, taper stores all the information about files on that archive (such as filename, file size, backup time, etc.) into a file called the archive information file. This file is stored on the hard disk in a directory (default is ~/.taper_info). In future, when accessing this archive, taper uses the archive information file to quickly gain access to all details about the archive---this speeds up performance since the tape doesn't need to be accessed.
The downside of this is that you need to make sure that this information file does not get deleted or corrupted (don't despair if it does, though, since you can recreate it). Also, if you wish to restore files on a different machine, you have to make sure that you either reconstruct the info file on the new machine, or take a copy of the info file with you on a floppy (or via an ftp transfer).
Each archive created is allocated a unique archive ID and you use that in future for accessing the archive if you don't have the tape handy.
You need to tell taper the name of the backup devices (both rewinding and non-rewinding). You can do that by giving the command line options -f (or --rewinding-device) for the rewinding device and -n (or --non-rewinding-device) for the non- rewinding device. Alternatively, you can start taper and then specify the names using the Global preferences option.
If you compiled with -DHAVE_SCSI option on, the default names are /dev/st0 (rewinding) and /dev/nst0 (non-rewinding). If you compiled with -DUSING_ZFTAPE, the default names are /dev/qftape(rewinding) and /dev/nqftape (non-rewinding). Otherwise, the default names are /dev/ftape (rewinding) and /dev/nftape (non-rewinding). It is also possible to set default names using environment variables---TAPE is the name of the rewinding device and NTAPE is the name of the non-rewinding device. Alternatively, you can use a preference file to set defaults (see below).
Start taper, by typing:
$ taper
You will then be presented with the main taper window. There are three main modules---backup, restore and mkinfo, as well as preference management options. Select the backup option.
If an archive exists, you will be asked whether you wish to append files to it, or whether you wish to overwrite it. As with all dialog boxes, the space-bar toggles between the options, and ENTER will select the currently highlighted option.
If the archive doesn't exist, you will be prompted for the archive title.
Next you will be prompted for the volume title.
You will then be presented with a screen with three panels. The top left shows the current directory on the hard disk, the top right shows what's currently on the archive and the bottom panel is used to show which files have been selected for backup. At the top of the screen is the archive ID and archive title.
To move between panels, press the TAB key. To get help on keys, press H.
You now need to select which files and directories you wish to backup. Use the cursor keys to move around the directory. Pressing ENTER when the highlight is on a directory will move into that directory.
When you find a file or directory you wish to back up, press S. The file/directory will then be sized and moved to the bottom window---if you selected a directory, taper will check with you that you really want to back it up. Press ENTER to confirm. To disable the confirmation, change this in global preferences (Prompt Directories).
In the bottom window, the file/directory will be printed as well as its size. Also, to the left of this, there will be an I or F. This indicates that the file/directory will be backed up in incremental mode (IM.) or full backup mode (F). To toggle between F and I modes, press S when the highlight is on the selected file or directory.
When you select directories, all directories under that directory are recursively included.
If you wish to deselect a file, move the cursor to the bottom window (using TAB) and then move the highlight to the file/directory you wish to deselect. Press D and the file/directory will be deselected.
If you select a file (e.g., /usr/home/john/xyz) and then select the directory in which the file resides (e.g., /usr/home/john), taper automatically recognizes that the file has been selected twice and will put brackets around the file (/usr/home/john/xyz) to tell you so. When doing the backup, the file will be backed up only once.
When you have finished selecting, press F and taper will commence the backup. Pressing Q at any time will abort the backup.
Select restore from the taper main menu.
You will then be presented with a list of all the archives taper knows about. They are sorted in archive ID order and the archive title is also printed. The highlight will be on the archive that is currently in the tape drive. Move the highlight onto the archive that you wish to restore from and press ENTER.
You will then be presented with three panels. The top left shows the files and directories currently on the archive, the top right shows a summary of the whole archive and the bottom panel is used to show the directories and files selected for restoring.
Use the cursor keys to move the highlight to select which files you wish to restore---pressing S selects the file/directory the highlight is currently on. Directories are recursively selected.
When you have selected a file/directory, it is transferred to the bottom window. In a similar way to backup, restore will put brackets around files selected twice.
In the select window, after the filename, the volume number is printed. This will show either a volume number or M. If M appears, taper is operating in most recent restore mode and will restore only the most recent copy of that file. If a volume number is displayed, the file will be restored from that volume, regardless of how recent the file in that volume is. You can toggle between the two modes by pressing S in the select window.
To deselect a file, position the highlight on the file you wish to deselect and press D.
When you have finished selecting files for restore, press F and taper will commence the restore. Pressing Q will abort the restore operation.
If you lose the archive information file (it gets deleted, corrupted or you try to restore on another machine and forget to take the archive information file with you), you can reconstruct the info file. Simply put the tape in the drive and select mkinfo from the taper main menu.
Below is a list of the more common preferences. For a complete list, see the man page.
You can save your preferences to customize your particular setup. There are two ways to s: one is to a preference file and the second is to a command line file which can then be used to start taper in the future. Simply select the appropriate option from the main menu.
taper looks for a preference file in the following order:
Some tape drives write zeros to the beginning of a tape and this can cause confusion with taper, which thinks it has reached the end of the tape when it detects zeros. To find out if your tape drive does this, put a tape in the drive and run the testzero program---note that the tape will be overwritten. taper will test your tape and print the result on the screen.
If taper says that your drive writes leading zeros, you will have to run mktape on every tape before you can use it with taper.
People using floppy tape drives have to both format and erase tapes. These users must either format tapes using DOS, OS/2 or WINDOWS or buy pre-formatted tapes. Erasing tapes is done automatically by taper.
People with SCSI tape drives generally don't need to format tapes. Some SCSI tape drives don't even need erasing (e.g., DAT). To tell taper not to erase tapes before using them, change the erase tape option in the backup preferences menu and save the preferences. Run mktape on all tapes before you use them so that taper doesn't think they are bad.
If you have a SCSI drive and are not sure whether you need to erase tapes, tell taper not to erase tapes and see what happens.
Linux has support for a /proc file system. This is a directory that looks like a normal directory, but actually contains information about the current machine state. It is useful for programs like ps which can read this directory and print the information contained in it.
However, we obviously don't want to back up this directory. You can tell taper to automatically exclude this directory. Run the which_device program:
$ ./which_device /proc
and the device on which the /proc directory is mounted is printed. Most probably this is 1, in which case you don't need to do anything because this is taper's default. If it is not 1, tell taper this via the backup options or via the command line (--proc-device num).
If you have many files to backup, taper can end up using quite a bit of memory. If you wish to minimize the amount of memory taper uses while running, edit your Makefile and un-comment the line (i.e., remove the #] in front of the line):
DEFINES=-DMEMORY_TIGHT
Now, when taper runs, it will be quite memory efficient. The downside (of course, there's a downside!) will be that performance will not be as good. However, on most machines, you will not notice a performance degredation until you reach 2,000-3,000 files.
There is a special preference that applies only to SCSI drives. This is the --block-size option. The SCSI kernel tape driver expects data to be presented to it in blocks of a maximum size---the default is 32K. For this reason, taper writes data in blocks of 28K by default. However, should you wish to change that, you can do so with the [cw]--block-size[ecw] optioni---for example, some tape drives may function more optimally if data is written in blocks of, say, 64K. Note that this must be less than the SCSI kernel tape driver's maximum size.
You can change this option for non-SCSI drives, but it won't really affect performance.
There is no distinction between file sets made in restore and file sets made in backup---they can be used interchangeably.
To make a file set, enter backup or restore and select the files and directories you wish to designate as your file set. Then press B and taper will prompt you for a name to give to the file set. After you enter the name, the file set will be saved.
Next time you wish to backup this particular file set, press L in backup or restore, and taper will show you a list of the file sets it knows about. Select one using the arrow keys and ENTER. This particular file set will then be loaded.
taper was designed to make backing up your Linux file system easy and painless. The traditional Unix utilities, tar and cpio, are very powerful, but they are not very user friendly. With Linux becoming more popular with non-hackers, another backup solution was badly needed. I hope taper fills this gap.
There are times, however, when you should use tar rather than taper. They are:
Unless you are in one of those situations, taper should be adequate for most of your needs. It is certainly easier to use than tar and cpio.
As this product is under development, suggestions, bug-fixes, comments, etc. are all welcome. Similarly, short messages saying that taper works for your system are greatly appreciated since it gives me an idea of how many people are using taper and what sort of hardware it works on. This can allow me to help other people who have similar hardware.
Yusuf Nagree is a part time doctor and a full time Linux hacker (aargh---sorry, full time doctor and part time Linux hacker). He has been a computer buff since his dad bought him a ZX-80 in 1980 and has had various computers over the years. Bored with DOS, OS/2 and Windows, the aspect of Linux he finds most enjoyable is the community spirit and general willingness to help and share knowledge and experience.
