NAME
buffer - very fast reblocking program
SYNTAX
buffer [-S size] [-b blocks] [-s size] [-m size] [-p percentage] [-u
microseconds] [-B] [-t] [-Z] [-i filename] [-o filename]
OPTIONS
-i filename
Use the given file as the input file. The default is stdin.
-o filename
Use the given file as the output file. The default is stdout.
-S size
After every chunk of this size has been written, print out how
much has been written so far. Also prints the total throughput.
By default this is not set.
-s size
Size in bytes of each block. The default blocksize is 10k to
match the normal output of the tar(1) program.
-z size
Combines the -S and -s flags.
-b blocks
Number of blocks to allocate to shared memory circular buffer.
Defaults to the number required to fill up the shared memory
requested.
-m size
Maximum size of the shared memory chunk to allocate for the
circular queue. Defaults to one megabyte.
-p percentage
Only start a write when the given percentage of the internal queue
is full. A percentage around 75 often proves best. Defaults to
zero.
-u microseconds
After every write pause for this many microseconds. Defaults to
zero. (Surprisingly a small sleep, 100 usecs, after each write
can greatly enhance throughput on some drives.)
-B Force each block written to be padded out to the blocksize. This
is needed by some tape and cartridge drives. Defaults to
unpadded. This only affects the last block written.
-t On exiting print to stderr a brief message showing the total
number of bytes written.
-Z If reading/writing directly to a character device (like a tape
drive) then after each gigabyte perform an lseek to the start of
the file. Use this flag with extreme care. It can only be used
on devices where an lseek does not rewind the tape but does reset
the kernels position flags. It is used to allow more than 2
gigabytes to be written.
Sizes are a number with an optional trailing character. A ’b’
multiplies the size by 512, a ’k’ by 1024 and an ’m’ by a meg.
DESCRIPTION
Buffer reads from standard input reblocking to the given blocksize and
writes each block to standard output.
Internally buffer is a pair of processes communicating via a large
circular queue held in shared memory. The reader process only has to
block when the queue is full and the writer process when the queue is
empty. Buffer is designed to try and keep the writer side continuously
busy so that it can stream when writing to tape drives. When used to
write tapes with an intervening network buffer can result in a
considerable increase in throughput.
The default settings for buffer are normally good enough. If you are a
heavy tape user then it is worth your while trying out various
different combinations of options. In particular running a buffer at
both ends of the pipe can provide a substantial increase (see last
example below).
EXAMPLES
$ buffer < /etc/termcap > /dev/rst8
$ tar cf - . | rsh somehost ’buffer > /dev/rst8’
$ dump fu - | rsh somehost ’buffer -s 16k > /dev/nrst8’
$ tar cf - . | buffer |
rsh somehost ’buffer -S 500K -p 75 > /dev/rst0’
BUGS
Internally, for printing purposes, buffer counts in terms of the number
of kilobytes output. If the blocksize you use is not a whole number of
kilobytes then the numbers printed will be inaccurate.
THANKS
Thanks to Kevin Twidle <kpt@doc.ic.ac.uk> for a lot of early
suggestions and patches to make it work with non-tar/dump tapes to
exabyte drives.
Thanks to Andi Karrer <karrer@bernina.ethz.ch>, Rumi Zahir
<rumi@iis.ethz.ch> and Christoph Wicki <wicki@iis.ethz.ch> for patches
to make buffer work when trying to write single tape files of greater
than 2 gigabytes.
COPYRIGHT
Copyright (C) 1990, 1991 by Lee McLoughlin.
SEE ALSO
dd(1), tar(1), rsh(1)
14 May 1990