#《快乐的 Linux 命令行》

#cat

Ctrl-d可以告诉cat到达文件末尾。
-A选项可以显示不可见字符,比如制表符等:

1
2
3
4
5
6
$ cat > onns.txt
The quick brown fox jumped over the lazy dog.

$ cat -A onns.txt
^IThe quick brown fox jumped over the lazy dog.$
^I^I$

-n选项给文本行添加行号。
-s选项禁止输出多个空白行。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ cat > onns.txt
The quick brown fox


jumped over the lazy dog.


$ cat onns.txt
The quick brown fox


jumped over the lazy dog.


$ cat -ns onns.txt
1 The quick brown fox
2
3 jumped over the lazy dog.
4

不知道为什么这个cat之后会在最前面增加五个空格,之前我在MacOS上的wc之类的命令也会,有机会查一下!下面是我在Mac上的运行结果,也是有空格:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[14:30:49] onns@Onns ~/Desktop $ cat > onns.txt
The quick brown fox


jumped over the lazy dog.


[14:31:04] onns@Onns ~/Desktop $ cat onns.txt
The quick brown fox


jumped over the lazy dog.


[14:31:08] onns@Onns ~/Desktop $ cat -ns onns.txt
1 The quick brown fox
2
3 jumped over the lazy dog.
4

#TODO

  • 查一下为什么cat会有多余的空格。

#sort

sort程序对标准输入的内容,或命令行中指定的一个或多个文件进行排序,然后把排序结果发送到标准输出。

之前已经列举过参数了,不重复了,具体可以参考:ip 访问记录查询 top3

显示10个最大的空间消费者:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ du -s /usr/share/* | head
112 /usr/share/GConf
6824 /usr/share/GeoIP
108 /usr/share/ImageMagick-6
28 /usr/share/PackageKit
5848 /usr/share/X11
676 /usr/share/aclocal
160 /usr/share/aclocal-1.15
8 /usr/share/acpi-support
8 /usr/share/adduser
612 /usr/share/adium
$ du -s /usr/share/* | sort -nr | head
166684 /usr/share/fonts
122520 /usr/share/sogouimebs
81468 /usr/share/doc
68532 /usr/share/icons
44384 /usr/share/man
40920 /usr/share/ibus
38372 /usr/share/locale
37360 /usr/share/backgrounds
32064 /usr/share/vim
29984 /usr/share/help

-n可以按照数值排序,但是这是因为这个数字在最开头,ls的就不在最开始,所以需要-k参数:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ ls -l /usr/bin | head
total 174060
-rwxr-xr-x 1 root root 96 2月 27 23:10 2to3-2.7
-rwxr-xr-x 1 root root 10104 4月 23 2016 411toppm
lrwxrwxrwx 1 root root 11 9月 25 2020 GET -> lwp-request
lrwxrwxrwx 1 root root 11 9月 25 2020 HEAD -> lwp-request
lrwxrwxrwx 1 root root 11 9月 25 2020 POST -> lwp-request
lrwxrwxrwx 1 root root 4 12月 1 01:57 X -> Xorg
lrwxrwxrwx 1 root root 1 9月 25 2020 X11 -> .
-rwxr-xr-x 1 root root 2438904 12月 1 01:57 Xephyr
-rwxr-xr-x 1 root root 274 12月 1 01:57 Xorg
$ ls -l /usr/bin | sort -nr -k 5 | head
-rwxr-xr-x 1 root root 19428040 2月 2 16:21 snap
-rwxr-xr-x 1 root root 7619056 10月 30 19:45 gdb
-rwxr-xr-x 1 root root 5893648 6月 24 2019 ubuntu-report
-rwxr-xr-x 1 root root 4551912 1月 13 2020 gnome-control-center
-rwxr-xr-x 2 root root 4526456 1月 26 23:33 python3.6m
-rwxr-xr-x 2 root root 4526456 1月 26 23:33 python3.6
-rwxr-xr-x 1 root root 3633000 2月 27 23:10 python2.7
-rwxr-xr-x 1 root root 3111952 2月 12 16:45 x86_64-linux-gnu-ld.gold
-rwxr-xr-x 1 root root 2886928 2月 12 16:45 x86_64-linux-gnu-dwp
-rwxr-xr-x 1 root root 2675336 10月 13 23:49 vim.basic
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ vi distros.txt
SUSE 10.2 12/07/2006
Fedora 10 11/25/2008
SUSE 11.0 06/18/2008
Ubuntu 8.04 04/24/2008
Fedora 8 11/08/2007
SUSE 10.3 10/04/2007
Ubuntu 6.10 10/26/2006
Fedora 7 05/31/2007
Ubuntu 7.10 10/18/2007
Ubuntu 7.04 04/19/2007
SUSE 10.1 05/11/2006
Fedora 6 10/24/2006
Fedora 9 05/13/2008
Ubuntu 6.06 06/01/2006
Ubuntu 8.10 10/30/2008
Fedora 5 03/20/2006
$ sort distros.txt
Fedora 10 11/25/2008
Fedora 5 03/20/2006
Fedora 6 10/24/2006
Fedora 7 05/31/2007
Fedora 8 11/08/2007
Fedora 9 05/13/2008
SUSE 10.1 05/11/2006
SUSE 10.2 12/07/2006
SUSE 10.3 10/04/2007
SUSE 11.0 06/18/2008
Ubuntu 6.06 06/01/2006
Ubuntu 6.10 10/26/2006
Ubuntu 7.04 04/19/2007
Ubuntu 7.10 10/18/2007
Ubuntu 8.04 04/24/2008
Ubuntu 8.10 10/30/2008

sort中一个关键值可能包括一个字段区域,如果没有指定区域,sort程序会使用一个键值,其始于指定的字段,一直扩展到行尾。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ sort -k 1,1 -k 2n distros.txt
Fedora 5 03/20/2006
Fedora 6 10/24/2006
Fedora 7 05/31/2007
Fedora 8 11/08/2007
Fedora 9 05/13/2008
Fedora 10 11/25/2008
SUSE 10.1 05/11/2006
SUSE 10.2 12/07/2006
SUSE 10.3 10/04/2007
SUSE 11.0 06/18/2008
Ubuntu 6.06 06/01/2006
Ubuntu 6.10 10/26/2006
Ubuntu 7.04 04/19/2007
Ubuntu 7.10 10/18/2007
Ubuntu 8.04 04/24/2008
Ubuntu 8.10 10/30/2008

1,1的意思是排序始于并且结束于第一个字段
2n2是排序的键值,并且按照数值排序(n)。一个选项字母可能被包含在一个键值说明符的末尾,其用来指定排序的种类,这些选项字母和sort程序的全局选项一样。
key选项允许在字段中指定偏移量:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt
Fedora 10 11/25/2008
Ubuntu 8.10 10/30/2008
SUSE 11.0 06/18/2008
Fedora 9 05/13/2008
Ubuntu 8.04 04/24/2008
Fedora 8 11/08/2007
Ubuntu 7.10 10/18/2007
SUSE 10.3 10/04/2007
Fedora 7 05/31/2007
Ubuntu 7.04 04/19/2007
SUSE 10.2 12/07/2006
Ubuntu 6.10 10/26/2006
Fedora 6 10/24/2006
Ubuntu 6.06 06/01/2006
SUSE 10.1 05/11/2006
Fedora 5 03/20/2006

3.7nbr代表第三个字段中的第七个字符开始。

-t指定分隔符:

1
2
3
4
5
6
7
8
9
10
11
$ sort -t ':' -k 7 /etc/passwd | head
hs:x:1000:1000:hs,,,:/home/hs:/bin/bash
root:x:0:0:root:/root:/bin/bash
gdm:x:121:125:Gnome Display Manager:/var/lib/gdm3:/bin/false
gnome-initial-setup:x:120:65534::/run/gnome-initial-setup/:/bin/false
hplip:x:117:7:HPLIP system user,,,:/var/run/hplip:/bin/false
speech-dispatcher:x:111:29:Speech Dispatcher,,,:/var/run/speech-dispatcher:/bin/false
whoopsie:x:112:117::/nonexistent:/bin/false
sync:x:4:65534:sync:/bin:/bin/sync
_apt:x:104:65534::/nonexistent:/usr/sbin/nologin
avahi-autoipd:x:106:112:Avahi autoip daemon,,,:/var/lib/avahi-autoipd:/usr/sbin/nologin

#uniq

uniq程序能完成任务,其输入必须是排好序的数据。

可以参考:ip 访问记录查询 top3

#cut

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
$ cut --help
Usage: cut OPTION... [FILE]...
Print selected parts of lines from each FILE to standard output.

With no FILE, or when FILE is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
-b, --bytes=LIST select only these bytes
-c, --characters=LIST select only these characters
-d, --delimiter=DELIM use DELIM instead of TAB for field delimiter
-f, --fields=LIST select only these fields; also print any line
that contains no delimiter character, unless
the -s option is specified
-n (ignored)
--complement complement the set of selected bytes, characters
or fields
-s, --only-delimited do not print lines not containing delimiters
--output-delimiter=STRING use STRING as the output delimiter
the default is to use the input delimiter
-z, --zero-terminated line delimiter is NUL, not newline
--help display this help and exit
--version output version information and exit

Use one, and only one of -b, -c or -f. Each LIST is made up of one
range, or many ranges separated by commas. Selected input is written
in the same order that it is read, and is written exactly once.
Each range is one of:

N N'th byte, character or field, counted from 1
N- from N'th byte, character or field, to end of line
N-M from N'th to M'th (included) byte, character or field
-M from first to M'th (included) byte, character or field

GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report cut translation bugs to <http://translationproject.org/team/>
Full documentation at: <http://www.gnu.org/software/coreutils/cut>
or available locally via: info '(coreutils) cut invocation'

一个由tab分离的文件,每行不太可能包含相同的字符数,这就使计算每行中字符的位置变得困难或者是不可能。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ cut -f 3 distros.txt
12/07/2006
11/25/2008
06/18/2008
04/24/2008
11/08/2007
10/04/2007
10/26/2006
05/31/2007
10/18/2007
04/19/2007
05/11/2006
10/24/2006
05/13/2008
06/01/2006
10/30/2008
03/20/2006
$ cut -f 3 distros.txt | cut -c 7-10
2006
2008
2008
2008
2007
2007
2006
2007
2007
2007
2006
2006
2008
2006
2008
2006

expand命令可以把tab展开成空格:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ expand distros.txt | cut -c 23-
2006
2008
2008
2008
2007
2007
2006
2007
2007
2007
2006
2006
2008
2006
2008
2006

-d指定分隔符:

1
2
3
4
5
6
7
8
9
10
11
$ cut -d ':' -f 1 /etc/passwd | head
root
daemon
bin
sys
sync
games
man
lp
mail
news

#paste

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ paste --help
Usage: paste [OPTION]... [FILE]...
Write lines consisting of the sequentially corresponding lines from
each FILE, separated by TABs, to standard output.

With no FILE, or when FILE is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
-d, --delimiters=LIST reuse characters from LIST instead of TABs
-s, --serial paste one file at a time instead of in parallel
-z, --zero-terminated line delimiter is NUL, not newline
--help display this help and exit
--version output version information and exit

GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report paste translation bugs to <http://translationproject.org/team/>
Full documentation at: <http://www.gnu.org/software/coreutils/paste>
or available locally via: info '(coreutils) paste invocation'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
$ sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt > distros-by-date.txt
$ cat distros-by-date.txt
Fedora 10 11/25/2008
Ubuntu 8.10 10/30/2008
SUSE 11.0 06/18/2008
Fedora 9 05/13/2008
Ubuntu 8.04 04/24/2008
Fedora 8 11/08/2007
Ubuntu 7.10 10/18/2007
SUSE 10.3 10/04/2007
Fedora 7 05/31/2007
Ubuntu 7.04 04/19/2007
SUSE 10.2 12/07/2006
Ubuntu 6.10 10/26/2006
Fedora 6 10/24/2006
Ubuntu 6.06 06/01/2006
SUSE 10.1 05/11/2006
Fedora 5 03/20/2006
$ cut -f 1,2 distros-by-date.txt > distros-versions.txt
$ cat distros-versions.txt
Fedora 10
Ubuntu 8.10
SUSE 11.0
Fedora 9
Ubuntu 8.04
Fedora 8
Ubuntu 7.10
SUSE 10.3
Fedora 7
Ubuntu 7.04
SUSE 10.2
Ubuntu 6.10
Fedora 6
Ubuntu 6.06
SUSE 10.1
Fedora 5
$ cut -f 3 distros-by-date.txt > distros-dates.txt
$ cat distros-dates.txt
11/25/2008
10/30/2008
06/18/2008
05/13/2008
04/24/2008
11/08/2007
10/18/2007
10/04/2007
05/31/2007
04/19/2007
12/07/2006
10/26/2006
10/24/2006
06/01/2006
05/11/2006
03/20/2006
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ paste distros-dates.txt distros-versions.txt
11/25/2008 Fedora 10
10/30/2008 Ubuntu 8.10
06/18/2008 SUSE 11.0
05/13/2008 Fedora 9
04/24/2008 Ubuntu 8.04
11/08/2007 Fedora 8
10/18/2007 Ubuntu 7.10
10/04/2007 SUSE 10.3
05/31/2007 Fedora 7
04/19/2007 Ubuntu 7.04
12/07/2006 SUSE 10.2
10/26/2006 Ubuntu 6.10
10/24/2006 Fedora 6
06/01/2006 Ubuntu 6.06
05/11/2006 SUSE 10.1
03/20/2006 Fedora 5

#join

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
$ join --help
Usage: join [OPTION]... FILE1 FILE2
For each pair of input lines with identical join fields, write a line to
standard output. The default join field is the first, delimited by blanks.

When FILE1 or FILE2 (not both) is -, read standard input.

-a FILENUM also print unpairable lines from file FILENUM, where
FILENUM is 1 or 2, corresponding to FILE1 or FILE2
-e EMPTY replace missing input fields with EMPTY
-i, --ignore-case ignore differences in case when comparing fields
-j FIELD equivalent to '-1 FIELD -2 FIELD'
-o FORMAT obey FORMAT while constructing output line
-t CHAR use CHAR as input and output field separator
-v FILENUM like -a FILENUM, but suppress joined output lines
-1 FIELD join on this FIELD of file 1
-2 FIELD join on this FIELD of file 2
--check-order check that the input is correctly sorted, even
if all input lines are pairable
--nocheck-order do not check that the input is correctly sorted
--header treat the first line in each file as field headers,
print them without trying to pair them
-z, --zero-terminated line delimiter is NUL, not newline
--help display this help and exit
--version output version information and exit

Unless -t CHAR is given, leading blanks separate fields and are ignored,
else fields are separated by CHAR. Any FIELD is a field number counted
from 1. FORMAT is one or more comma or blank separated specifications,
each being 'FILENUM.FIELD' or '0'. Default FORMAT outputs the join field,
the remaining fields from FILE1, the remaining fields from FILE2, all
separated by CHAR. If FORMAT is the keyword 'auto', then the first
line of each file determines the number of fields output for each line.

Important: FILE1 and FILE2 must be sorted on the join fields.
E.g., use "sort -k 1b,1" if 'join' has no options,
or use "join -t ''" if 'sort' has no options.
Note, comparisons honor the rules specified by 'LC_COLLATE'.
If the input is not sorted and some lines cannot be joined, a
warning message will be given.

GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report join translation bugs to <http://translationproject.org/team/>
Full documentation at: <http://www.gnu.org/software/coreutils/join>
or available locally via: info '(coreutils) join invocation'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
$ cut -f 1,1 distros-by-date.txt > distros-names.txt
$ paste distros-dates.txt distros-names.txt > distros-key-names.txt
$ cat distros-key-names.txt
11/25/2008 Fedora
10/30/2008 Ubuntu
06/18/2008 SUSE
05/13/2008 Fedora
04/24/2008 Ubuntu
11/08/2007 Fedora
10/18/2007 Ubuntu
10/04/2007 SUSE
05/31/2007 Fedora
04/19/2007 Ubuntu
12/07/2006 SUSE
10/26/2006 Ubuntu
10/24/2006 Fedora
06/01/2006 Ubuntu
05/11/2006 SUSE
03/20/2006 Fedora
$ cut -f 2,2 distros-by-date.txt > distros-vernums.txt
$ paste distros-dates.txt distros-vernums.txt > distros-key-vernums.txt
$ cat distros-key-vernums.txt
11/25/2008 10
10/30/2008 8.10
06/18/2008 11.0
05/13/2008 9
04/24/2008 8.04
11/08/2007 8
10/18/2007 7.10
10/04/2007 10.3
05/31/2007 7
04/19/2007 7.04
12/07/2006 10.2
10/26/2006 6.10
10/24/2006 6
06/01/2006 6.06
05/11/2006 10.1
03/20/2006 5

为了使join命令能正常工作,所有文件必须按照关键数据域排序:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ join distros-key-names.txt distros-key-vernums.txt
11/25/2008 Fedora 10
10/30/2008 Ubuntu 8.10
06/18/2008 SUSE 11.0
05/13/2008 Fedora 9
04/24/2008 Ubuntu 8.04
11/08/2007 Fedora 8
10/18/2007 Ubuntu 7.10
10/04/2007 SUSE 10.3
05/31/2007 Fedora 7
04/19/2007 Ubuntu 7.04
12/07/2006 SUSE 10.2
10/26/2006 Ubuntu 6.10
10/24/2006 Fedora 6
06/01/2006 Ubuntu 6.06
05/11/2006 SUSE 10.1
03/20/2006 Fedora 5

默认情况下,join命令使用空白字符做为输入字段的界定符,一个空格作为输出字段的界定符。[1]

#comm

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
$ comm --help
Usage: comm [OPTION]... FILE1 FILE2
Compare sorted files FILE1 and FILE2 line by line.

When FILE1 or FILE2 (not both) is -, read standard input.

With no options, produce three-column output. Column one contains
lines unique to FILE1, column two contains lines unique to FILE2,
and column three contains lines common to both files.

-1 suppress column 1 (lines unique to FILE1)
-2 suppress column 2 (lines unique to FILE2)
-3 suppress column 3 (lines that appear in both files)

--check-order check that the input is correctly sorted, even
if all input lines are pairable
--nocheck-order do not check that the input is correctly sorted
--output-delimiter=STR separate columns with STR
--total output a summary
-z, --zero-terminated line delimiter is NUL, not newline
--help display this help and exit
--version output version information and exit

Note, comparisons honor the rules specified by 'LC_COLLATE'.

Examples:
comm -12 file1 file2 Print only lines present in both file1 and file2.
comm -3 file1 file2 Print lines in file1 not in file2, and vice versa.

GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report comm translation bugs to <http://translationproject.org/team/>
Full documentation at: <http://www.gnu.org/software/coreutils/comm>
or available locally via: info '(coreutils) comm invocation'
1
2
3
4
5
6
7
8
9
10
$ vi file1.txt
a
b
c
d
$ vi file2.txt
b
c
d
e
1
2
3
4
5
6
$ comm file1.txt file2.txt
a
b
c
d
e

第一列包含第一个文件独有的文本行;第二列文本行是第二列独有的;第三列包含两个文件共有的文本行。

comm支持-n形式的选项,这里n代表123。这些选项使用的时候,指定了要隐藏的列:

1
2
3
4
$ comm -12 file1.txt file2.txt
b
c
d

今天先到这里。

292/505


  1. 感觉和MySQLjoin一样的,也不知道是谁学谁😂。 ↩︎