Data Transfer in Unix
The article published below is part of a complete article from A.P.LawrenceYou need to transfer data from machine A to machine B. Maybe you are lucky enough to have networking on both machines, but maybe you aren't. Maybe you have tape drives, but they are incompatible. Or maybe they should be compatible, but they don't seem to want to work. Maybe you don't have tape drives or a network, but you still need to transfer that data. How?
There are a number of possibilities, depending on what you have and what your time frame is. In general, you are usually better off making archives on the machine you are transferring from rather than transferring individual files one by one. You might do that with tar, for example, using something like:
cd /somedir
tar cvf /tmp/files.tar .
Which will create an archive of the current directory into /tmp/files.tar which can be transferred to the new machine and extracted with:
cd /newdir
tar xvf /tmp/files.tar
However, there are times when it's simpler to just pick and choose individual files. If you are transporting binary files by any method that requires text files, the "uuencode" program is needed. You use uuencode like this:
uuencode myprog myprog > myprog.uu
(or)
cat myprog | uuencode myprog > myprog.uu
At the other end, you unpack it by:
uudecode myprog.uu
See "man uuencode" for more information. If you want to compress a file, compress it before you uuencode it.
- Data Conversion Firm
- Mail It
- Move the hard drive
- Tape Transfer
- Floppy Shuffle
- Other Media
- UUCP
- Other Serial Methods
- Network Transfers
- Visionfs, Samba
Data Conversion Firm
If this is a one time transfer of data (an upgrade, for example), a commercial data conversion company might be the simplest solution.
Conversion houses usually work only with tape or sometimes other media like the old Bernoulli cartridges, etc. If you don't have tape, and don't have any other way to move the data, a data recovery company could extract the data from your hard drive and put it on CDROM or tape. Admittedly, this is extreme, and it's apt to be expensive, but if there is no other way to get the job done in the time constraints you are working with, this can be the answer.
Mail It
This could be an option if you have a modem and can configure (or already have configured) mail on the old machine. You will need to uuencode files (see the top of this article).
Move the Hard Drive
Perhaps it is possible to mount the hard drive from one machine as a file system on another. This is very easy if the hardware is the same and you are both coming from and going to Unix. If, for example, you have SCSI drives, you'd add in the old drive exactly as you would add a new drive, except that you would NOT overwrite the file systems already there. You will need to name the divisions (they'll have no names when you get to divvy), and if they are old Xenix you'll also need to run "mkdev xenix" to add support for these old file systems.SCO has technical articles that covers this in detail:- OSR5
- Generic
- Unixware
If it's IDE, the procedure is basically the same, but if you need to mix SCSI and IDE you might have a problem because at this time, the IDE MUST be the boot drive- even if your computer BIOS allows otherwise, the SCO OS will not, at least not at the time this was written. You can always add a SCSI drive to an IDE configuration, but not IDE to a SCSI booted system. A possible way out is to temporarily install SCO on a spare IDE system, install the other drive, and finally transfer the data by tape or network to the system you really want it on.DOS drives can be mounted, but only (again, as of this writing) FAT-16, not FAT-32.
Tape Transfer
SCO Unix can support multiple tape drives. That means that you can have several different tape drives all physically installed and working at the same time. You can mix SCSI and non-SCSI tapes, DAT's, QIC's, whatever (though some older QIC drives with their own adaptor cards do have to be the default drive).
This might mean that you can take the old tape drive out of the old machine, install it in the new machine, and just restore your data.If that can't be done (physically inconvenient or no drivers for the older tape drive), it may be that newer drives can read the older tapes. For example, QIC tape drives have remained beautifully backward compatible- the newest 2.5 gigabyte models can still read the old DC600 60 megabyte tapes.
Sometimes you have the same physical tape and drives, but can't read the data. This can be caused by tape blocking being set differently on the two machines; you can change that with the "tape -a setblk" command. For example, if you were coming from a 3.2v4.2 system, that uses a fixed 512 byte tape block, while OSR5 uses variable tape blocking. The 3.2v4.2 can use variable, but it needs to be told to do so:
tape -a 0 setblk
(this functionality requires UOD376c from ftp://stage.caldera.com/SLS/) Unfortunately, compression is another issue entirely. Different manufacturers can use different hardware compression algorithms, so a DAT tape created on one manufacturer's drive may not be able to be read on another.
Another problem is drives that have drifted far out of alignment: the drive that wrote the tape may still be able to read it, but other drives can't. Data conversion and data recovery firms may be your only choice if you have these problems.
UUCP
UUCP is fairly easy to set up, and can work at high speed across a hardwired serial connection. You can get UUCP for DOS and Windows platforms, and of course all Unix platforms have it already. The major disadvantage is speed: the serial ports are not going to transfer gigabytes of data in any reasonable amount of time. Consider that under the best conditions, with no overhead taken into account, 115k baud would move less than 3/4 megabyte per minute. That's not great when you have hundreds of megabytes to move, and if it's an older machine, 115k may not even be possible.If it's just small text files, cu's "put" and "take" commands work quite well. You use these by cu'ing to the old system, and typing (for example)~%take /etc/group /tmp/groupwhich will transfer the /etc/group file on the old machine to /tmp/group on the machine you cu'd from. If you need binary files, you'll need to uuencode them; see the beginning of this article.Most of the UUCP books are out of print, so it's hard to find anything more than the documentation you have on your system. Fortunately the built in SCO documentation is pretty good. If you need UUCP for DOS or Windows, the following links may be useful:
- http://www.ramnet.net/
- http://www.cdrom.com/simtel.net/msdos/uucp-pre.html
Other Serial Methods
If you have small amounts of simple text to transfer, you can do it just by catting or even printing it to a serial terminal emulation program. This is a possibility when you have no real access to the computer that has the data on it. If you can print the data you want to a serial port, you can capture it using a terminal program.Some terminal software (for example ICE.TEN by James River) includes file transfer capability.
If you are already using such software, you may already have everything you need.For larger needs, where UUCP can't be used, many systems have or can use Kermit.Kermit runs on just about every OS ever made, and it can work over serial or TCP/Ip connections. This makes it an excellent choice for data transfer.Particularly see http://www.columbia.edu/kermit/gkermit.html#boot and Kermit-95Another possibility is Xt. If you are going to or coming from a Dos or Windows system with Zmodem, Xmodem or Ymodem, you can get Rz and Sz for Unix (See TLS025 at http://www.caldera.com/support/docs//skunkware/tls).
Another interestingsolution (assuming a Unix to Unix transfer) is to use Expect (see Book Review: Expect). This has the advantage of requiring nothing beyond the usual Unix utilities: (cat, compress, uuencode, etc.) on the "other" machine. You can download Expect from http://expect.nist.gov/. The file transfer program will be found in the "example" subdirectory after you unpack the distribution; it's called "ftp-inband". Note that you need TCL for Expect: if you install Expect from Skunkware it will automatically install TCL also.Kermit can sometimes be used in ways similar to Expect; see Kermit Scripts
Here's an example of the simple serial method: I had a Xenix 2.2 machine recently (Apr 2000) that was on its last legs. A Foxpro database had 9,000 records that the customer wanted to transfer to Act. Neither the Xenix floppy nor its tape drive worked at all. I wrote a small Foxplus program that put the data out to a file, and then used Hyperterminal to log in to the Xenix system, turned on "Capture" and typed "cat /tmp/datafile". At 19.2K, this took about a half an hour or so to transfer approximately 4 megabytes. I needed to do some further conversion work that would have been slow and painful on the old Xenix system and impossible on Windows, so I emailed the file to myself using the client's AOL account. After cleaning up the data back at my office and converting it to a CSV format, I emailed it back to the client.
Floppy Shuffle
Although not particularly fast, copying off to floppy on one machine and reading on another is certainly simple. Set up a two or three disk rotation so that you can be reading while copying. Beware of the disks wearing out: at 25 cents each, you really didn't expect them to last for a hundred uses, did you?
For transferring small files to and from Dos, the "doscp" (or mtools from Skunkware) can be used.If you have downloaded patches, etc. to a Windows machine and you have a network connection, of course you can just ftp the files over. If you do not, you need to do it by floppy, and you will immediately realize that most of the patches are disk images: they are 1.44 MB and will not fit on a DOS floppy.
There is a utility (RAWRITE) that will solve this problem for you. Download it from ftp://stage.caldera.com/TLS/tls096.zip. See http://stage.caldera.com/cgi-bin/ssl_reference?105004 for more information.
Other Media
Jaz drives, etc. can be installed on Unix systems if they are SCSI based.
Network Transfers
The ftp programis probably the easiest way to transfer files between machines. You can make it easier in several ways by learning a few basic ftp commands.
Here's an example ftp session that illustrates some of these things:
# ftp oldmachine
Connected to oldmachine.
220-
220 oldmachine FTP server (Version 2.1WU(1)) ready.
Name (oldmachine:root):
331 Password required for root.
Password:
230 User root logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> bin
200 Type set to I.
ftp> hash
Hash mark printing on (1024 bytes/hash mark).
ftp> prompt off
ftp> mget *.txt
The "bin" makes sure that binary files will be transferred correctly. That's the default on SCO machines, but it doesn't hurt to be certain. The ONLY time you want "text" (the opposite of "binary") is when you are transferring text files between different OS's; for example Windows to Unix. If you are transferring binary files, or transferring between the same OS, you want "binary".The "hash" turns on hash mark (#) printing for evey 1024 bytes transferred.
This gives you a visual indication of progress and lets you know if you are "hung" on Internet transfers.The "prompt off" allows us to use "mget" and "mput" without being prompted for every file the pattern matches (if you DO want to be prompted, use "prompt on".Ftp transfers can be automated. The Expect program can be used to recursively descend sub-directories, etc. Or, you can use ftp's .netrc file. The .netrc file must be in the home directory of the user invoking ftp, and must not be readable or writable by other users. Here's an example .netrc file:
The "macdef" keyword defines a macro. Macros are executed by typing the name preceded by a "$"; the "init" macro is executed automatically after login. Note that the .netrc must end with a blank line. See "man netrc" for more information.machine oldbox login root passwd rootpass
macdef init
hash
prompt off
mget *.cgi
macdef myget
cd somedir
mget *.data
cdup
Visionfs, Samba
If you are transferring data to or from Windows machines, and have Visionfs or Samba (see Visionfs) you can just drag and drop files over.
© May 1999 Tony Lawrence All Rights Reserved
This article is copyrighted material. You have permission to use it for any purpose, commercial or non-commercial, as long as it is kept intact or if excerpts are clearly marked as excerpts and credit is given as specified herein.You may publish it in paper or electronic form. That includes magazine, newsletters, and web pages, both internal and external, for profit or not. Banner ads and other graphics may be removed, but all other text, hyperlinks and copyright notices, including this, should remain (but see below also). You may not delete text, alter it, or add to it in any way that does not clearly delineate what is yours and what comes from this site. You may alter fonts, font sizes and the like and reformat text as is appropriate for your use.You may select specific paragraphs or sections, but if you do so, you must either include this entire notice also, noting that you have not published the entire article, or simply note that the paragraphs you have published are part of a larger article and give the http address of the actual article.My main concern is that no one would be confused that you wrote something I wrote or vice-versa. If your use meets that concern, and allows people to find the original article here, I have no objection.This general permission specifically does NOT apply to test questions and answers.Some articles here are copyrighted by other individuals or corporations; these paragraphs do not apply to those articles even if accidentally included. We do appreciate being advised of any such use: Email: tony@aplawrence.com.