Basic shell navigation, commands, features

pwd = print working directory (your current location)cddirectory = move into that directoryls = list contents of current directoryecho = print something onscreen- a string, variable, filename etc etccatmyfile = display contents of a text fileopen . = open current directory in FinderCtrl-A = go to start of lineCtrl-E = go to end of lineCompletion: Press [TAB] while typing a command/filename to auto-complete it; it beeps if ambiguous – press [TAB] again and the alternatives are shown.Drag n drop: For commands using/requiring files/folders with their full path, drag and drop the file/folder onto the command line. e.g. type "cd " then drag a folder from Finder to make that the current directory.Permissions: If you aren’t permitted to execute a command, put sudo (super-user do) at the start of the command, which will ask for the admin password before continuing.Inspector: [CMD]-I = launch the Terminal Inspector to change settings for shell fonts, colours, cursor, etc.

history = display shell history – all commands you’ve run↑ and ↓ = scroll through previously used commandsCtrl-R = search history. Ctrl-R, type one or more letters, then Ctrl-R again repeatedly to step back through your history.historyn = display last n commands

Help!

Type help, or to see all available commands, hold ESC until prompted, then answer y.mancommand = display manual pages on a command.Multi-page results: Navigate the manual pages, the output of less etc a page at a time with : SPACE = see next page, b = back one page, q = quit back to terminal prompt, /string = search – highlights occurrences of string. Cycle forwards through the highlighted strings with n, or backwards with N.info bash = many pages of info about bash. [SPACE] = next page, B = back, U = Up a level, N = Next, P = Prev. Move cursor to any section title and press ENTER to go to that section.man bash = exhaustive bash documentationman man = display more info on the man commandHelp: helpcommand. Some commands also have short help messages, use command-hhelp * = help on all builtin commands – very useful to read!help = nice 1-page summary of builtin commands + optionsaproposblang or man -kblang = search manual pages for blang.typeblang = searches the environment for matching commands & displays them. use option -a to display all matches, not just the first.locateblang = find a file or command

Controlling input/output

Pipes: A concept central to Unix. Input can be taken from, and output to, files or other commands, with pipelines |, which make the output of one command the input of the next. E.g. cat * | wc -l counts the total number of lines in all files in current directory – this takes the output of cat and feeds it to wc. The idea is to use a range of simple tools together, rather than try to do everything with one program.Less: Use | less at the end of a command line to display results one page at a time. Or lessmyfile to display a text file. q=quit, SPACE=next page, ENTER=next line, b=prev page, /=search mode: Type /string to find string in the less output, and n to cycle forward, N to cycle back, through occurrences of string.

Quotes: These are very important in bash, and often used to protect strings from the various expansions the shell performs on every input line.'single quotes' = treat all contents literally, i.e. ignore the special character meanings in bash."double quotes" = treat everything literally except $ (variable/arithmetic substitution) and command substitution.

* = Match any string of zero or more characters.? = Match any single character.[abc…] = Match any one of the enclosed characters; a hyphen can specify a range (e.g., a-z, A-Z, 0–9).[!abc…] = Match any character not enclosed as above.~ = Home directory of the current user.~name Home directory of user name.~+ = Current working directory ($PWD).~- = Previous working directory ($OLDPWD).

Internet commands

ifconfig = display network information

curl -Ohttp://www.somewebsite.com/files/file.mp4 = download a file to current directorycurl -L -omyfile.dmg 'http://www.somewebsite.com/files/getdmg?id=24' = download file to current directory and rename. -L allows curl to follow any redirects. Putting the URL in single quotes allows non-alphanumeric characters to be written as is.

Ctrl-rr = use previous history search stringCtrl-k = delete from cursor to end of lineCtrl-u = delete from cursor to start of lineCtrl-w = delete word (from cursor to the previous space)Ctrl-– = undo last editing commandCtrl-l = clear screen, put current line at top of screenCtrl-t = transpose (swap) 2 charactersCtrl-y = paste deleted text at cursor (yank)Ctrl-] = reads next keypress and moves cursor forwards to that characterCtrl-xx = toggle between the start of line and current cursor positionCtrl-v – literal next. Used for entering special characters into arguments
e.g. to change a space to a TAB character with sed: (for the TAB, press Ctrl-v then TAB)

bash$ echo t u v w x | sed 's/ / /g'
t u v w x

META commands: These may work instead with the ALT/META key on your system. Otherwise Esc = METAfy the next key pressed.Esc f = move forward 1 wordEsc b = move back 1 wordEsc t = transpose (swap) 2 wordsEsc d = delete from cursor to end of wordEsc BACKSPACE = delete from cursor to start of word (Esc DEL on some systems)Esc y = yank-pop (paste a previously deleted item with each press)Esc Ctrl-] = reads next keypress and moves cursor backwards to that characterEsc Ctrl-e = expand the line as the shell does: alias, history, word expansions etcEsc . = paste last word of previous commandEsc u = make every character upper case, from the cursor to the end of the current word.Esc l = make every character lower case, from the cursor to the end of the current word.Esc c = Capitalize the character under the cursor and move to the end of the word.Esc ? = List all possible completions (e.g. on empty line, will display all commands)Esc * = Insert all possible completionsEsc { = Put all filename completions, comma-separated, between { }.

There are many more, and they’re customizable. See GNU bash manual
See a list of your current shell key bindings with:

bind -p | grep -Ev 'not bound|self-|do-' | sed 's/"//g' | less

More shell history commands

!! = Repeat previous command (can be used as a part of a line or script)!-2 = 2nd last command!-2$ = last argument from 2nd last command!n = Repeat command number n in the history!$ = the last parameter from the previous command!* = all parameters from the previous command!blang = run the most recent command that started with blang!blang:p = print out the command that !blang would run and add it to the command history!$ = last word of the prev command!$:p = print out that word!string = Execute the last command that begins with string

^str – removes ‘str‘ from the previous command and executes^str1^str2 = substitutes str1 for str2 in the previous command and executes:

diff -yfile1 file2 = displays all differences between 2 similar files, -y means ‘format in 2 columns’. Also can be used to display the differences between 2 similar directories.vimdifffile1 file2 [file3] = colourful visual display of the differences between 2 or 3 text files.

grep (Global Regular Expression Print) finds a particular pattern of characters in files, with many options for different searches. See regex/grep/sed page.grep -ic but article.txt = displays the number of lines containing ‘but’ in article.txt.++-i = ignore (upper/lower) case++-c = display only the number of lines, not the lines themselves.++-n = show line number where match was found.++-o = show only matching part. This can be used to find the number of matches, e.g.:

textutil -catrtf-outputcombined.rtf *.doc = combine all Word documents in the current directory into a single rich-text document called combined.rtf textutil -converthtml foo.rtf = converts foo.rtf into foo.html. textutil -convertrtf-fontTimes-fontsize10 foo.txt = converts foo.txt into foo.rtf, using Times 10 for the font.textutil -cat html -title "Several Files" -output index.html *.rtf = loads all RTF files in the current directory, concatenates their contents, and writes the result out as index.html with the HTML title set to “Several Files”.textutil -convert html myfile.rtf; open myfile.html = convert specified rich text file to HTML, save as myfile.html and display page in default browser
(format can be one of: txt, html, rtf, rtfd, doc, wordml, or webarchive)

cut – cuts columns of text from files. Specify the character or field positions. -d = Specify the field delimiter character. -f = which fields to select. Reads from a file or read from std input with – e.g.cut -d ' ' -f 2myfile = print the 2nd word from each line of myfile.

paste – pastes columns of text together, side-by-side

xargs – to format input to be used as command arguments.
Can also be used to format text, like this:

uniq – report or omit repeated lines++-c = precede each line with N, the number of times it was repeated++-d = don’t output lines that aren’t repeated++-u = don’t output repeated lines++-f N = ignore the first N fields++-s N = ignore the first N characters

Note on quoting

bash$ echo 'Don\'t quote me' # This doesn't work.

Bruce Barnett says “I bet many of you programmers are confused by this. All of us are very familiar with strings in programming languages like C. This is why we get confused. [In bash] the quotes turn substitution on and off. They are not used to indicate the starting and ending of a string.”

bash$ echo 'Don'\''t do that' # This works.

e.g. shell script printcol:

#!/bin/bash
# printcol
# e.g. printcol 3
awk '{print $'$1'}'

Notice how the 1st command-line argument $1 isn’t single-quoted, but is expanded by the shell, to become in the awk program $n, meaning in awkese the nth word/field on the input line.

More on redirecting input/output

<<label = Here-document. Accepts multi-line input from a file or keyboard, which starts and ends with an arbitrary string label.
● Use brackets to redirect the output from a group of commands. This actually runs the commands in a subshell. e.g. (pwd; ls; cd ../; pwd; ls) >outputfile< = process substitution e.g. for reading lines from output of commands
e.g. To make the shell scroll down, with command line at top:

Environment variables

$* = all the command line parameters as a single text value$@ = all the command line parameters as separate text values$# = the number of command line parameters$? = the exit status of the most recently used foreground process$- = the current command line option flags$$ = The process ID of the current shell$! = the process ID of the most recently executed background process$0 = the name of the command from the command line$_ = the absolute pathname of the shell

bash$ set one two three four # set some fake positional parameters
bash$ echo ${!#} # get the last ('nth', not 'previous') argument
four

(# stores the number of arguments, so !# will reference the last argument)

String variable operators

bash$ string=abcdefghijklmnopqrstuvwxyz
bash$ echo ${#string} = print the length of the variable/string
26
bash$ echo ${string:4} = substring from character 4 to end of string
efghijklmnopqrstuvwxyz
bash$ echo ${string:4:8} = substring from char 4 with length 8
efghijkl
bash$ echo ${string:(-1)} = substring from last character to the end
z
bash$ echo ${string:(-2):2} = substring from 2nd last char, length 2.
yz
bash$ var="This is a line of text"
bash$ echo ${var/line/REPLACED} #Replacing some text in a variable
This is a REPLACED of text

● These string operations using the ${name …} notation can also be applied to all string elements in an array, with the ${name[@] …} or ${name[*] …} notation.Indirection${!param} = the referenced parameter is not param itself, but the parameter whose name is stored as the value of param, e.g.

myfiles=(*.txt) = Create an array myfiles that contains the names of all .txt files in the current directory.echo ${myfiles[@]} = print the members of that arrayfor f in ${myfiles[@]};do echo $f;done = print the members of that array, 1 per lineread -ramyarray = Chop a line into fields and store the fields in myarray. $IFS=delimiter

● Single quotes inhibit all expansions—the characters enclosed by the quotes pass through the expansions unscathed.
● Double quotes permit some expansions and inhibit others. Word expansion and command, arithmetic, and process substitution take place—the double quotes only affect how the result is handled—but brace and tilde expansion do not.

Compound statements/flow control

Test/If-then statements

Use [[ to compare strings, files, (( for integers. (The only benefit of [ is compatibility with some other and ancient shells.)

String tests: =++!=++<++>[[ -nstr1]] – checks if str1 has length > 0[[ -zstr1]] – checks if str1 has length 0.NB The and symbols must be escaped (e.g. \>), or the shell uses them as redirection symbols, with the string values as filenames. Also, what is considered greater or less than is different from the order used by sort.

File tests:[[ -efile]] checks if file exists. Check if the file: -d = is a directory, -f = is a file, -r = is readable by you, -s = is not empty, -w = is writable by you, -x = is executable by you.file1-nt/-otfile2 – checks if file1 is newer/older than file2[[condition1]] && [[condition2]] = AND[[condition1]] || [[condition2]] = OR

if

The form is : if list; then list; [ elif list; then list; ] …[ else list; ] fi
Unlike the usual programming if-then, these are lists of commands, and the list in iflistthen takes the return value of the last command in list, e.g.

Aliases & functions

● Functions, unlike aliases, can take arguments.
● Putting alias definitions in single quotes means variables won’t be evaluated until the command is run. Inside double quotes, they’re evaluated immediately, and never change after that – which is probably not what you want.type blang = print info on an alias/function/builtinbuiltinblang = ignore shell functions and aliases to run the actual built-in commandcommandblang = ignore shell functions and aliases to run the actual external command\blang = prevent alias expansiondeclare -f = display all function definitionsdeclare -F = display all functions, name onlydeclare or set = display all shell parameters, aliases and functions

d – used when downloading a lot of files from online links. Copy the first file link to clipboard, then run function. Return to browser, copy more links one at a time, and the files will be automatically downloaded each time the clipboard changes.

Put this in scripts to return altered colours/fonts to normal in the event of a Ctrl-C quit:

trap control_c SIGINT
control_c() {
tput sgr0
exit 3 #for c=3...
}

Timing commands

time for ((i=0;i<5000;i++));do result=$(($i%2)); done = get times for 5000 loops, on my machine:
expression++++++++++++timeresult=$(($i%2))++++++0.249sresult=`expr "$i%2"`+24.389slet result=$i%2+++++++0.216sresult=$((i%2))+++++++0.141sresult=$[i%2]++++++++0.156s

Bash scripts

Use: #!/usr/bin/env bash as the shebang in all bash scripts. This finds the bash in the $PATH.
(#!/bin/sh or #!/bin/bash have been more common. #!/bin/sh/ is fine if bash isn’t needed, if bash features aren’t used.)bashfilename = load a shell script/program from a text file in the current directory, and run it in a new subshell.
Run a user-written script like a shell command (just by typing its name) by adding its directory to the command search path, and giving it execute permission with chmod +xscriptname. OR add to the .bash_profile file as a function – e.g. myfunc() {blahbla; blah; }.bash_profile is read when Bash is invoked as a login shell..bashrc is executed every time a new shell starts.
aliases and functions should be in .bashrc, otherwise they’re not available to subshells. But DONT put things that will grow in .bashrc, e.g. PATH=$PATH:blah/blah/blah would be expanded with each new shell.Running/executing vs sourcing a scriptsource script.sh or . script doesn’t start a new process, so variables can be shared e.g.

Hmm this seems to give weird line numbers sometimes…Dealing with errors, interrupts, exiting
Use || for error messages:command|| echo “commanddidnt work!”trap -l = list trappable signals
To do stuff in a script after a Ctrl-c, before quitting:

trap 'echo INT signal received;exit' INT

${:?} – use for parameter errors, e.g.

DIR=${1:?"Error. Must give a directory."}
SRC=${2:?"Error. Must give a source file."}
GV=${3:?"Error. Must give a giblet vendor."}

Some sample scripts

Download something, if interrupted keep resuming until download is completed
I added this to my .bash_profile:

Strip html from a filesed 's|]*>||g'myfile = remove the pattern “, then >” anywhere that pattern’s found. Can’t just use because it will always just find the last > in the line, not the next.
Reverse the numbering on sequentially numbered files

Convert programs/scripts to HTML-safe and QuickLatex-safe for pasting online
It takes the contents of the clipboard and transforms < into &lt;, > into &gt; and $ into \$.pbpaste | sed -e 's/</\&lt;/g' -e 's/>/\&gt;/g' -e 's/\$/\\$/g' | pbcopy
(This script doesn’t work applied to itself. I was going to paste here the HTML needed to produce that line on the page, but the HTML needed to display that properly would be much longer again… etc. Very like something out of Hofstadter’s Gödel, Escher, Bach, which features such things as a record that can’t be played by a certain record player, in illustration of Gödel’s ideas.)
Scrolling ad banner

\a = The ASCII bell character (007)\A = The current time in 24-hour HH:MM format\d = The date in “Weekday Month Day” format\D {format} = The format is passed to strftime(3) and the result is inserted into the prompt string; an empty format results in a locale-specific time representation; the braces are required\e = The ASCII escape character (033)\H = The hostname\h = The hostname up to the first “.“\j = The number of jobs currently managed by the shell\l = The basename of the shell’s terminal device name\n = A carriage return and line feed\r = A carriage return\s = The name of the shell\T = The current time in 12-hour HH:MM:SS format\t = The current time in HH:MM:SS format\@ = The current time in 12-hour a.m./p.m. format\u = The username of the current user\v = The version of bash (e.g. 2.00)\V = The release of bash; the version and patchlevel (e.g. 3.00.0)\w = The current working directory\W = The basename of the current working directory\# = The command number of the current command\! = The history number of the current command\$ = If the effective UID is 0, print a #, otherwise print a $\nnn = Character code in octal\\ = Print a backslash.\[ = Begin a sequence of nonprinting characters, such as terminal control sequences\] = End a sequence of nonprinting characters

Miscellaneous

stty size = returns height & width of current shell window

scriptmyfile = captures everything from the shell window to the file.++-a = append (put at end of existing file)++Ctrl-D to stop recording.