LTO-5 磁带机使用

LTO-5之前的磁带机不支持LTFS,故本文内容只适用于LTO-5及之后的磁带机。

安装 ltfs

见:
https://github.com/LinearTapeFileSystem/ltfs

查看系统中的磁带机

1
ltfs -o device_list

在我的系统中,此命令输出为:

1
2
3
4
5
6
7
8
9
10
11
root@CSPM-Research-Team-Internal-Server:~/data/tmp/ITDT# ltfs -o device_list
32dd LTFS14000I LTFS starting, LTFS version 2.5.0.0 (Prelim), log level 2.
32dd LTFS14058I LTFS Format Specification version 2.4.0.
32dd LTFS14104I Launched by "ltfs -o device_list".
32dd LTFS14105I This binary is built for Linux (x86_64).
32dd LTFS14106I GCC version is 10.2.1 20210110.
32dd LTFS17087I Kernel version: Linux version 5.18.4 (root@chenlh-linux-workstation) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PREEMPT_DYNAMIC Wed Jun 15 19:41:39 CST 2022 i386.
32dd LTFS17089I Distribution: PRETTY_NAME="Debian GNU/Linux 11 (bullseye)".
32dd LTFS17085I Plugin: Loading "sg" tape backend.
Tape Device list:.
Device Name = /dev/sg4 (6.0.0.0), Vendor ID = HP , Product ID = Ultrium 5-SCSI , Serial Number = HU1215N2CE, Product Name =[Ultrium 5-SCSI].

格式化磁带为LTFS格式

1
mkltfs -d HU1215N2CE

HU1215N2CE即为磁带机的序列号,当然我们也可以写成:

1
mkltfs -d /dev/sg4

对于已经有文件系统的磁带,出于安全考虑,此命令不会对磁带进行格式化,我们需要使用-f参数进行强制格式化

复制大量文件

可能需要将脚本/usr/local/bin/ltfs_ordered_copy 第一行的python改为python3
并安装pyxattr

如果直接使用cprsync复制,可能会因为文件位置不连续导致多次倒带,
为了避免这种情况可以使用ltfs_ordered_copy来进行复制:

1
ltfs_ordered_copy -ar 源目录 目标目录

挂载/卸载LTFS磁带

挂载:

1
ltfs -o devname=HU1215N2CE 挂载点

卸载:

1
umount 挂载点

弹出磁带

1
ltfs -o devname=HU1215N2CE -o release_device

查看磁带和磁带机

可以通过ITDT来查看磁带的 使用次数、历史读取和写入错误 等,也可以对磁带机进行性能测试。

具体的ITDT的下载见:
https://www.ibm.com/docs/en/spectrum-archive-ee/1.2.5.0?topic=tools-downloading-tape-diagnostic-tool

磁带检查保存方法

见:
https://www.dell.com/support/kbdoc/zh-cn/000122049/%e5%a6%82%e4%bd%95%e5%a4%84%e7%90%86%e5%a4%87%e4%bb%bd%e7%a3%81%e5%b8%a6

附录

mkltfs 的帮助手册

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
Usage: mkltfs <options>

Available options are:
-d, --device=<name> Tape device (required)
-f, --force Force to format medium
-s, --tape-serial=<id> Tape serial number (6 alphanumeric ASCII characters)
-n, --volume-name=<name> Tape volume name (empty by default)
-r, --rules=<rules> Rules for choosing files to write to the index partition.
The syntax of the rule argument is:
size=1M
size=1M/name=pattern
size=1M/name=pattern1:pattern2:pattern3
A file is written to the index partition if it is no larger
than the given size AND matches at least one of the name
patterns (if specified). The size argument accepts K, M, and G
suffixes. Name patterns might contain the special characters
'?' (match any single character) and '*' (match zero or more
characters)
--no-override Disallow mount-time data placement policy changes
-w, --wipe Restore the LTFS medium to an unpartitioned medium (format to a legacy scratch medium)
-q, --quiet Suppress progress information and general messages
-t, --trace Enable function call tracing
--syslogtrace Enable diagnostic output to stderr and syslog
-V, --version Version information
-h, --help This help
-p, --advanced-help Full help, including advanced options
-i, --config=<file> Use the specified configuration file (default: /usr/local/etc/ltfs.conf)
-e, --backend=<name> Use the specified tape device backend (default: sg)
--kmi-backend=<name> Use the specified key manager interface backend (default: none)
-b, --blocksize=<num> Set the LTFS record size (default: 524288)
-c, --no-compression Disable compression on the volume
-k, --keep-capacity Keep the tape medium's total capacity proportion
-x, --fulltrace Enable full function call tracing (slow)
--long-wipe Unformat the medium and erase any data on the tape by overwriting special data pattern.
This operation takes over 3 hours. Once you start, you cannot interrupt it
--destructive Use destructive format/unformat. This operation takes longer time
in the LTO9 drive or later because of the media optimization procedure


LTFS17085I Plugin: Loading "flatfile" kmi backend.
Key manager interface flatfile plug-in options:
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=FILE
Data key and data key identifier pairs' list file

.
LTFS17085I Plugin: Loading "simple" kmi backend.
Key manager interface simple plug-in options:
-o kmi_dk=<DK>
Data key
-o kmi_dki=<DKi>
Data key identifier
-o kmi_dk_for_format=<DK>
Data key to format a cartridge
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=<DK>:<DKi>/<DK>:<DKi>/.../<DK>:<DKi>
Data key and data key identifier pairs' list

.

Usage example:
mkltfs --device=0 --rules="size=100K"
mkltfs --device=0 --rules="size=1M/name=*.jpg"
mkltfs --device=0 --rules="size=1M/name=*.jpg:*.png"

ltfs 帮助手册

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
usage: ltfs mountpoint [options]

general options:
-o opt,[opt...] mount options
-h --help print help
-V --version print version

FUSE options:
-d -o debug enable debug output (implies -f)
-f foreground operation
-s disable multi-threaded operation

-o allow_other allow access to other users
-o allow_root allow access to root
-o auto_unmount auto unmount on process termination
-o nonempty allow mounts over non-empty file/dir
-o default_permissions enable permission checking by kernel
-o fsname=NAME set filesystem name
-o subtype=NAME set filesystem type
-o large_read issue large read requests (2.4 only)
-o max_read=N set maximum size of read requests

-o hard_remove immediate removal (don't hide files)
-o use_ino let filesystem set inode numbers
-o readdir_ino try to fill in d_ino in readdir
-o direct_io use direct I/O
-o kernel_cache cache files in kernel
-o [no]auto_cache enable caching based on modification times (off)
-o umask=M set file permissions (octal)
-o uid=N set file owner
-o gid=N set file group
-o entry_timeout=T cache timeout for names (1.0s)
-o negative_timeout=T cache timeout for deleted names (0.0s)
-o attr_timeout=T cache timeout for attributes (1.0s)
-o ac_attr_timeout=T auto cache timeout for attributes (attr_timeout)
-o noforget never forget cached inodes
-o remember=T remember cached inodes for T seconds (0s)
-o nopath don't supply path if not necessary
-o intr allow requests to be interrupted
-o intr_signal=NUM signal to send on interrupt (10)
-o modules=M1[:M2...] names of modules to push onto filesystem stack

-o max_write=N set maximum size of write requests
-o max_readahead=N set maximum readahead
-o max_background=N set number of maximum background requests
-o congestion_threshold=N set kernel's congestion threshold
-o async_read perform reads asynchronously (default)
-o sync_read perform reads synchronously
-o atomic_o_trunc enable atomic open+truncate support
-o big_writes enable larger than 4kB writes
-o no_remote_lock disable remote file locking
-o no_remote_flock disable remote file locking (BSD)
-o no_remote_posix_lock disable remove file locking (POSIX)
-o [no_]splice_write use splice to write to the fuse device
-o [no_]splice_move move data while splicing to the fuse device
-o [no_]splice_read use splice to read from the fuse device

Module options:

[iconv]
-o from_code=CHARSET original encoding of file names (default: UTF-8)
-o to_code=CHARSET new encoding of the file names (default: UTF-8)

[subdir]
-o subdir=DIR prepend this directory to all paths (mandatory)
-o [no]rellinks transform absolute symlinks to relative

LTFS options:
-o devname=<dev> Tape device
-o config_file=<file> Configuration file (default: /usr/local/etc/ltfs.conf)
-o work_directory=<dir> LTFS work directory (default: /tmp/ltfs)
-o atime Update index if only access times have changed
-o noatime Do not update index if only access times have changed (default)
-o tape_backend=<name> tape backend to use (default: sg)
-o iosched_backend=<name> I/O scheduler implementation to use (default: unified, use "none" to disable)
-o kmi_backend=<name> Key manager interface implementation to use (default: none, use "none" to disable)
-o umask=<mode> Override default permission mask (3 octal digits, default: 000)
-o fmask=<mode> Override file permission mask (3 octal digits, default: 000)
-o dmask=<mode> Override directory permission mask (3 octal digits, default: 000)
-o min_pool_size=<num> Minimum write cache pool size. Cache objects are 1 MB each (default: 25)
-o max_pool_size=<num> Maximum write cache pool size. Cache objects are 1 MB each (default: 50)
-o rules=<rules> Rules for choosing files to write to the index partition.
The syntax of the rule argument is:
size=1M
size=1M/name=pattern
size=1M/name=pattern1:pattern2:pattern3
A file is written to the index partition if it is no larger
than the given size AND matches at least one of the name
patterns (if specified). The size argument accepts K, M, and G
suffixes. Name patterns might contain the special characters
'?' (match any single character) and '*' (match zero or more
characters)
-o quiet Disable informational messages (same as verbose=1)
-o trace Enable diagnostic output (same as verbose=3)
-o syslogtrace Enable diagnostic output to stderr and syslog(same as verbose=303)
-o fulltrace Enable full call tracing (same as verbose=4)
-o verbose=<num> Override output verbosity directly (default: 2)
-o eject Eject the cartridge after unmount
-o noeject Do not eject the cartridge after unmount (default)
-o sync_type=<type> Specify sync type (default: time@5)
<type> should be specified as follows:
time@min: LTFS attempts to write an index each 'min' minutes.
min should be a decimal number from 1 to 153722867280912930
It is equivalent to "-o sync_type=unmount" when 0 is specified
(default: min=5)
close: LTFS attempts to write an index when a file is closed
unmount: LTFS attempts to write an index when the medium is unmounted
-o force_mount_no_eod Skip EOD existence check when mounting (read-only mount)
Only use for a CM corrupted medium
-o device_list Show available tape devices
-o rollback_mount=<gen|index_file> Attempt to mount on previous index generation on tape or specified index file (read-only mount)
-o release_device Clear device reservation (should be specified with -o devname
-o symlink_type=<type> Specify symbolic link type (default: posix)
<type> should be specified with one of the following values:
posix: LTFS behavior is same as standard symbolic link
live: LTFS replaces mount point path by current mount point
-o capture_index=<dir> Capture index to the specified directory by dir when index is updated
-o scsi_append_only_mode=<on|off> Set the tape device append-only mode (default=on)
-a Advanced help, including standard FUSE options
-V, --version Output version information and exit
-h, --help Display this help and exit

33de LTFS17085I Plugin: Loading "flatfile" kmi backend.
Key manager interface flatfile plug-in options:
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=FILE
Data key and data key identifier pairs' list file

.
33de LTFS17085I Plugin: Loading "simple" kmi backend.
Key manager interface simple plug-in options:
-o kmi_dk=<DK>
Data key
-o kmi_dki=<DKi>
Data key identifier
-o kmi_dk_for_format=<DK>
Data key to format a cartridge
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=<DK>:<DKi>/<DK>:<DKi>/.../<DK>:<DKi>
Data key and data key identifier pairs' list

ltfsck 帮助手册

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
Usage: ltfsck [options] filesys

filesys Device file for the tape drive

Available options are:
-g, --generation=<generation> Specify the generation to roll back
-r, --rollback Roll back to the point specified by -g
-n, --no-rollback Do not roll back. Verify the point specified by -g (default)
-f, --full-recovery Recover extra data blocks into directory _ltfs_lostandfound
-z, --deep-recovery Recover EOD missing cartridge.
Some blocks might be erased, but recover to final unmount point
with an index version of at least 2.0.0 or earlier.
(Must be used for a cartridge that cannot be recovered by a normal option.)
-l, --list-rollback-points List rollback points
-m, --full-index-info Display full index information (Effective only for -l option)
-v, --traverse=<strategy> Set traverse mode for listing roll back points. Strategy should be forward or backward. (default: backward)
-j, --erase-history Erase history at rollback
-k, --keep-history Keep history at rollback (default)
-q, --quiet Suppress informational messages
-t, --trace Enable diagnostic output
--syslogtrace Enable diagnostic output to stderr and syslog
-V, --version Version information
-h, --help This help
-p, --advanced-help Full help, including advanced options
-i, --config=<file> Use the specified configuration file (default: /usr/local/etc/ltfs.conf)
-e, --backend=<name> Override the default tape device backend
--kmi-backend=<name> Override the default key manager interface backend
-x, --fulltrace Enable full function call tracing (slow)
--capture-index=<dir> Capture indexes to the specified directory (-g is effective for this option)
--salvage-rollback-points List the rollback points of the cartridge that has no EOD


LTFS17085I Plugin: Loading "flatfile" kmi backend.
Key manager interface flatfile plug-in options:
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=FILE
Data key and data key identifier pairs' list file

.
LTFS17085I Plugin: Loading "simple" kmi backend.
Key manager interface simple plug-in options:
-o kmi_dk=<DK>
Data key
-o kmi_dki=<DKi>
Data key identifier
-o kmi_dk_for_format=<DK>
Data key to format a cartridge
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=<DK>:<DKi>/<DK>:<DKi>/.../<DK>:<DKi>
Data key and data key identifier pairs' list

.

ltfsindextool 帮助手册

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Usage: ltfsindextool [options] [filename]

Available options are:
-d, --device=<name> Tape device (Capture index to specified file when this option is specified. Otherwise run check process with specified file)
-p, --partition=<part_num> Partition number 0 or 1, capture both partitions if not specified
-s, --start-pos=<block> Block number to start capturing (default: 5)
--output-dir=<dir> Directory to store captured indexes (default: '.')
-b, --blocksize=<num> Specify the LTFS record size (default: 524288)
-i, --config=<file> Use the specified configuration file (default: /usr/local/etc/ltfs.conf)
-e, --backend=<name> Use the specified tape device backend (default: sg)
--kmi-backend=<name> Use the specified key manager interface backend (default: none)
-q, --quiet Suppress progress information and general messages
-t, --trace Enable function call tracing
-V, --version Version information
-h, --help This help


LTFS17085I Plugin: Loading "flatfile" kmi backend.
Key manager interface flatfile plug-in options:
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=FILE
Data key and data key identifier pairs' list file

.
LTFS17085I Plugin: Loading "simple" kmi backend.
Key manager interface simple plug-in options:
-o kmi_dk=<DK>
Data key
-o kmi_dki=<DKi>
Data key identifier
-o kmi_dk_for_format=<DK>
Data key to format a cartridge
-o kmi_dki_for_format=<DKi>
Data key identifier to format a cartridge
-o kmi_dk_list=<DK>:<DKi>/<DK>:<DKi>/.../<DK>:<DKi>
Data key and data key identifier pairs' list

.

Usage example:
ltfsindextool --device=0 -p=0

ltfs_ordered_copy 帮助手册

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
LTFS_ORDERED_COPY(1)                                                                              LTFS Command Reference                                                                              LTFS_ORDERED_COPY(1)

NAME
ltfs_ordered_copy - Copy files from source to destination with LTFS order optimization

SYNOPSIS
ltfs_ordered_copy [ -p ] [ -r ] [ -t TARGET_DIRECTORY ] [ --keep-tree CUTOFF_PREFIX ] [ -a ] [ -v ] [ --verbose LOG_LEVEL ] [ -q ] [ -h ] [ SOURCE ... ] [ DESTINATION ]

DESCRIPTION
ltfs_ordered_copy is a program to copy files from source to destination with LTFS order optimization. This command tries to acquire file list to copy to DESTINATION when no SOURCE is specified.

OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes ('-'). A summary of options is included below. For a complete description, see the Info files.

-p preserve attributes with shutil.copy2() in Python interpriter

-r, --recursive
copy directories recursively

-t, --target-directory=TARGET_DIRECTORY
copy all SOURCE arguments into TARGET_DIRECTORY

--keep_tree=CUTOFF_PREFIX
CUTOFF_PREFIX, effective only when file list is provided from stdin and keep-tree option is enabled. CUTOFF_PREFIX is removed from the file paths copied to the destination.

-a, --all
achieve files recursively and preserve attributes

-v Verbose output. Set VERBOSE level 5

--verbose=LOG_LEVEL
Configure verbosity of logger. VERBOSE shall be 0-6. (Default: 4)

-q, --quiet
No message outout

COMMAND EXAMPLES
This section shows various command examples.

SIMPLE USAGE
Copy file /foo/aaa to file /foo/bbb

$ ltfs_ordered_copy /foo/aaa /foo/bbb

Copy file /foo/aaa and /foo/bbb to directory /bar/

% ltfs_ordered_copy /foo/aaa /foo/bbb /bar

Copy directory /foo/ddd and /foo/DDD to directory /bar/

% ltfs_ordered_copy /foo/ddd /foo/DDD /bar

HOW TO USE WITH FIND COMMAND
Copy all files under /foo/ddd to directory /bar

$ find /foo/ddd -type f | ltfs_ordered_copy -t /bar

Copy all directories just under /foo/ddd to directory /bar

$ find /foo/ddd -type d -maxdepth 1 | ltfs_ordered_copy -t /bar

Copy all files just under /foo/ddd to directory /bar with keeping tree (Chop /foo/ddd from source list)

$ find /foo/ddd -type f | ltfs_ordered_copy -t /bar --keep-tree=/foo/ddd