Real programmers use cat
I've started a new project; I'm exploring the history of the Unix command line. This is a lot more daunting than writing a history of junk mail. I don't understand all the technical details (I'm not a kernel developer and don't know any low level programming languages) and I wasn't around when all the exciting stuff happened — I wasn't even born when Unix was first created and I was in nappies when Version 6 came out. Plus, there are lots of Unix greybeards ready to, quite rightly, turn their pitchforks on me. Or at least, that's my perception.
Over the last few weeks I've done lots of reading. And, as with the history of junk mail, there are lots of titbits that are too nice to just discard. For instance, while researching the history of text editors I came across the statement real programmers use
in an interview with Bill Joy, the creator of ex and vi, from 1984. The problem with ed (the standard editor, written by Ken Thompson in 1969 and already available in the first AT&T Unix version) was that it was slow. When you entered a command, such as cat as their editora (to append text after the current line) it might take a few seconds for the character to appear on the screen. That's also why Ed isn't very talkative; all that rendering of text is just a waste of system resources.
Editing with cat
The recommendation to use cat rather than the mighty ed is of course a little tongue in cheek. However, it can be done, and for simple tasks it can be the right tool. For instance, creating a simple file is as easy as using cat with output redirection:
$ cat > poem.txt Great fleas have little fleas upon their backs to bite 'em, And little fleas have lesser fleas, and so ad infinitum. ^D
Like all good Unix utilities, cat reads from STDIN when you don't provide an operand. It will read any text you type until you send the End-of-File character (Ctrl+D). To add another stanza to the poem you simply append STDIN (cat >> poem.txt).
ed, the standard editor
Composing the same poem with ed involves a few more key strokes. If you have ever dipped your toes in ed you probably understand what's happening here:
$ ed a Great fleas have little fleas upon their backs to bite 'em, And little fleas have lesser fleas, and so ad infinitum. . w poem.txt 121 q
Just like vi and Vim do nowadays, the editor starts in command mode. That means you can't just start writing code or prose straight away — for that you need to change to insert mode. In the above example I used the a command to append text, which automatically switches to input mode. Once I finished the first stanza I returned to command mode by entering a dot on a single line, so that I could next save the file using the command w poem.txt. The editor confirmed I had typed 121 characters and I then used the q command to exit ed.
ed's command mode is surprisingly capable. You can run sed commands to substitute text (i.e. %s/fleas/bugs/g replaces all instances of the string "fleas" with "bugs") and use regular expressions (g/^And/p prints any lines starting with the string "And"). This were not new inventions — Thompson had previously added regular expressions to the QED editor, which was the inspiration for ed, and the QED manual shows the editor already had a substitute command as well. So, the grep and sed syntax already existed before ed. The reason why ed is seen as the ancestor of sed and grep is probably because QED was written for the Berkeley Timesharing System, which unlike Unix didn't stand the test of time.
The editor's insert mode is a lot less fancy. Like cat, ed is line-oriented and uses the terminal's cooked mode. That means that all the input is processed by the terminal driver rather than the program. You can do basic editing on a single line (i.e. move the cursor to the right and use backspace) but you can't use the up arrow key to navigate to the previous line. As far as terminal is concerned the up arrow has no special meaning, and it will therefore print the escape sequence for the up arrow (^[[A), rather than move the cursor to the previous line.
em, the editor for mortals
Editors such as vi use raw mode rather then cooked mode. In raw mode, characters you type are processed by the program rather than the terminal. That is why Ctrl+D moves the cursor down half a page in vi, and not terminate the program, like it does in ed (and utilities such as cat).
Unix already supported raw mode in the early 1970s and George Coulouris created a version of ed that took advantage of it in 1975: em. Coulouris described em as "a sort of single-line screen editor". Among others, it featured an o command (also known as open mode) to visually edit a line and an x command to interactively substitute text. This are all the options shown when running h (the help command) in em on NetBSD, which has an em package in pkgsrc:
b line breaks on r read file
b- line breaks off s substitute /old/new/
d delete line(s) t copy line(s)
e edit a new file v globally if not
f set file name w write on file
g globally if x substitute interactively
h display this summary (`.` to confirm)
k mark line / delimit search string
m move line(s) ? delimit backward
l list line(s) search string
o open line(s) for mods ! unix command
(^H for help in `o`) = line number
o\ replace line(s) > prompts off
o- insert new lines % display context
o+ append new lines & .16, .p
p print line(s) " .+1, +17p
q quit editor +,- .+p, .-1p
The x command works nicely. When you run 1,$x/fleas/bugs/, the editor underlines the first instance of the string "fleas" with caret symbols. Entering a dot confirms the change, while hitting Return skips to the next instance.
> 1,$x/fleas/bugs/
Great fleas have little fleas
^^^^^
The o commands also work but it doesn't show the line you are editing and you don't get any feedback as you type. For instance, when you insert a new line 2 with the command 2o- you get to see a prompt (a backslash), and you can then start typing. However, nothing is printed to the screen, which I assume is a bug. Interestingly, Joy noted in the above-mentioned interview that em's "strange version of open modes […] didn't work very well" on his ADM-3A.
em has a few other noteworthy features. In command mode you get a prompt (a greater than symbol), which is a visual reminder of when you are in command mode. That's quite handy — in ed you just have to know if you are in command or insert mode. And, you can exit open mode with the Esc key, which will sound familiar if you know vi and/or Vim.
The reason em didn't make it into the hall of fame is that using raw mode for line editing meant that the current line needed to be redrawn after every key press. That required lots of CPU cycles, which were scarce in the early Unix days — remember that even running the a command might take a few seconds. According to Coulouris, Ken Thompson wasn't impressed with em either. He had commented that he didn't feel a need for screen editors, as he didn't need to see the state of a file he was editing. This prompted Coulouris to dub em "the editor for mortals".
Brian Kernighan and Rob Pike shared Thompson's disdain for screen editors. In The UNIX Programming Environment (1984) they recommended good 'ol ed:
Almost every UNIX system has a screen editor, an editor that takes advantage of modern terminals to display the effects of your editing changes in context as you make them. Two of the most popular ones are vi and emacs. We won't describe any specific screen editor here, however, partly because of typographic limitations, and partly because there is no standard one.
There is, however, an older editor called ed that is certain to be available on your system. It takes no advantage of special terminal features, so it will work on any terminal.
ex, and vi
Of course, screen editors were the future, and it's probably fair to say that em was ahead of its time. Whether or not em would work depended on the terminal display (if the computer had one) and computers just weren't powerful enough yet for features such as editing text in context. em wasn't ahead of its time by much though. As the above quote shows, by 1984 both vi and Emacs had appeared on the scene.
The reason we remember em is that Bill Joy used the source code to first create ex and then vi (if it wasn't for Joy we might all be part of the Church of Emacs). The first version of ex was written in 1977, just two years after Coulouris had created em. ex still exists today — if you have vi or Vim installed then you should have ex as well.
The ex editor is a lot fancier than em. When you start the editor it clears the screen, so that your whole terminal is dedicated to the editor. If you run ex poem.txt you should see something like this at the bottom of the screen:
"poem.txt" 4L, 119B Entering Ex mode. Type "visual" to go to Normal mode. :
This is the command mode, and the colon is the prompt, where you can enter all the usual ed commands. You can also tweak lots of settings using the set command — the manual from 1977 already mentions settings such as autoindent, shiftwidth and, of course, errorbells. To illustrate, below I enable line numbers with the command set number and then print the first line in the file (1p).
:set number
:1p
1 Great fleas have little fleas
:
The real magic, though, is ex's visual mode. This is the not the visual selection mode in modern versions of vi and Vim — that didn't exist yet. Instead, the visual (or vi) command essentially entered the vi editor. After entering the command the entire file is printed (without having to run a print command) and you can navigate through the file:
1 Great fleas have little fleas
2 upon their backs to bite 'em,
3 And little fleas have less fleas,
4 and so ad infinitum.
~
~
~
:set number
This was quite a departure from ed and em. The visual editor still has a command and insert mode but in command mode (the default) you can now navigate through the file and make edits as you go along. That meant lots of new commands had to be introduced, both for navigating through files and operating on text. To navigate through a file you can use either the arrow keys or the h, j, k and m keys, as well as keys such as w and b (to move the cursor to the next and previous word, respectively). To edit text you can use commands such as c (change), d (delete), y ("yank", which is American for "copy") and p (put/paste yanked text). You can also combine navigation and operating commands. For instance, dw deletes the word under the cursor (from the cursor position to the end of the word) and ch lets you change the character to the left of the cursor.
Ex mode is also available in the visual editor; any commands starting with a colon appear at the bottom of the screen and are run as ex commands. And, you can of course also navigate through a file and make edits in insert mode. So, you now have three different modes in which you can edit text. If, say, you want to ruin the poem by changing the word "backs" to "bottoms" then you can take your pick:
- In command mode you can navigate to the first letter of the word and use
cwto change the word. The command selects the words and enters insert mode — once you have replaced the word you use the Esc key to return to command mode. - In insert mode you can navigate to the word and use the Delete or Backspace key to delete the string "backs", one character at a time. You can then enter "bottoms".
- In ex mode you can run
2s/backs/bottoms/.
I imagine seasoned ed users were impressed with ex but horrified by its visual mode — why would you navigate to a string you want to change when you can run a simple substitute command? Alas, it wasn't long before the visual editor became vi (which was really just a version of ex that opened in visual mode).
Vim (Vi iMpossible to Master)
Vim stands for Vi iMproved, though I suspect it was named after the cleaning product. The developer, Bram Moolenaar, was Dutch and Vim was a household name in the Netherlands (or, at least it was in our household). In any case, Vim was first released in 1991 and added lots of bells and whistles to vi. Yet, the editor is arguably easier to use than vi, if only because it can undo (and redo) more than one change and because the status line at the bottom of the editor is a little more verbose.
Vim is still actively developed and is probably the most popular command line text editor. And although it has lots of features that Thompson, Coulouris and Joy could never have imagined, all the ed and ex stuff is still there. In fact, if you want to master Vim, which some say is impossible, you can learn a lot by opening the editor in ex mode and resisting the temptation to enter visual mode.
All this history makes Vim a rather large and odd beasts. By "odd", I mean that much of the editor's design is accidental. Vim inherited the h, j, k and m keys from vi, and using those keys instead of the arrow keys is seen as efficient (because your hands can stay on the middle row of the keyboard). However, Joy didn't deliberately choose those keys as an efficient alternative for the arrow keys. He did so because his ADM-3A didn't have arrow keys — instead, the h, j, k and m keys doubled-up as arrow keys.
The same goes for text editor modes. ed used a command and insert mode because it otherwise wouldn't know if a string like "s/fleas/bugs/" was a command or text to be inserted. The concept wasn't invented to allow power users to edit more efficiently. It is an example of serendipitous design.
I guess Vim is a bit like a greybeard. They are a bit odd too, but in a good way. And with that I better find a place to hide from all those pitchforks!