ARSC HPC Users' Newsletter 419, March 29, 2011

Contents

Advanced vim part II

Part one of this series appeared in
issue #370
, September 2007, offering these VIM tips:

Cursor movement perfection

Combine cursor movement with change/delete/pipe operations

Use regular expressions

Let VIM handle repetitive tasks

Here’s one more skill for improving your VIM efficiency:

Use VIM’s built-in line editing (command-line mode) operations

For many of us, vi and VIM are a huge improvement over editors which make you grab the mouse. In VIM, you move the cursor with efficient keystrokes, but the need for any cursor movement at all can be reduced by using the command line.

Rather than moving the cursor and performing some action, you can often simply type a command to produce the same result, saving yourself keystrokes and improving your odds of doing exactly what you want without mistakes.

My hope with this article is to get you started with this new approach to editing (which is actually an old approach, if you were ever "privileged" to use TECO).

Background

When you’re in VIM "normal-mode" (AKA "command-mode") and hit the colon key, you enter the under-used "command-line mode" and can enter "ex" commands. Every VIM user already uses these:

You can examine the command before committing it!
In normal-mode, much of the command is invisible as you enter it: the movement commands, the delete buffer, etc. In command-line mode, it’s right in front of you.

How to do line editing in VIM

The first thing to learn is that all command line operations work on either one line or a range of lines. There are several ways to specify a line, and they’re all useful in different situations.

A LINE specification is

one of the predefined, special lines:
"." is the current cursor line
"$" is the last line in the file
(you can also specify an offset relative to "." or "$" as a line specification. E.g., "$-3" or ".+12" or ".-12".)

an explicit line number, e.g., "120". (Enter ":se[t] nu[mber]" to show line numbers.)

a mark. (In normal-mode, hit m followed by a letter to set a mark at the cursor position.

E.g, "ma" sets mark "a." On the command line use ":ma[rk]". E.g., ":120ma a" places mark "a" on line 120--without moving the cursor.)

the line on which a search operation finds a match. (E.g., :/pattern/)

A RANGE specification is

a comma delimited line specification. E.g.:

:120,130 (lines 120-130 inclusive)
:.,$ (current cursor line to end of file)
:’a,’b (mark "a" to mark "b")
:/Apple/,/Zebra/ (Next occurance of "Apple" to next occurance of "Zebra")

the predefined, special range:

"%" is the range 1,$ (i.e., the entire file)

ARGUMENTS to command-line commands

The commands I’m recommending take either a line address, a register name, a mark name or nothing as their arguments. What are these things?

Line specification:
One line specified as described above.

Register name:
When you yank or delete a range, the contents of that range are stored in a buffer, called a register. In addition to the unnamed (default) register you have 26 named registers, a-z. I suggest always yanking/deleting into a named register so you’re less likely to overwrite it. In MacVIM, buffers are shared between all tabs in a window, so you can easily yank a couple things from one file (one tab) and put them in another.

Mark name:
You can set up to 52 unique marks: a-z and A-Z.

Command-Line Commands

Syntax notes:

Items in square brackets are optional.

The default for [range] is the current cursor line.

The commands have the given abbreviations, e.g., "ya" works for "yank"

The default for [register name] is, well, the default register

Items in curly braces are required.

COMMANDS: Explanations and Examples

Yank
:[range]ya[nk] [register name]

In normal-mode, whenever you delete anything, with "x" or "d", the thing deleted overwrites the unnamed register. Many times, I’ve yanked a block of text, but before putting it in its new location, I’ve managed to overwrite it by accident. Thus, I’ve learned to always yank to a named buffer. E.g.,:

:.,122ya k

Best of all, it’s not necessary to move the cursor first. Just specify both ends of the range, wherever they are:

:12,13 ya x
:$-4,$ ya y
:’a,’b ya z

Delete
:[range]d[elete] {address}

With delete, it’s even more important to use a named register because (obviously) you’ve deleted the thing, making it harder to recover if you overwrite the default register.

:98,789d k

Deleting the current line:

:d k
:.d k

Put
:[line]pu[t] [register name]

You can put the contents of a register at the current cursor position:

:pu k
:.pu k

Or anywhere:

:333pu k
:’a pu k

You can put the default register at the current location:

:pu

Or at the next occurance of "Zebra"

:/Zebra/pu

Copy
:[range]co[py] {address} or :[range]t {address}

Copy a single line to another location:

:$-5co81

Note, ":t" is an alias for copy, and saves one character of typing:

:11,22t33

You can copy a range of lines inside itself, if needed:

:11,33t22

Move
:[range]m[ove] {address}

Example:

:11,22m33

Fold
:[range]fo[ld]

"Folding" hides a range of lines. Say you want to see and edit a file’s header and footer simultaneously, hiding everything in between:

:20,$-20 fold

How about hiding everything between the opening and closing BODY tag in an HTML file:

:/<body>/,/<.body>/ fold

Foldopen
:[range]foldo[pen]

Fold definitions persist, even after you "open" them to see what’s inside. You can open all folds in a range with foldopen:

:%foldo
:100,200 foldo

Foldclose
:[range]foldc[lose]

Similarly, close all folds in a range:

:%foldc

(Note, that there are other commands for erasing fold definitions.)

Mark
:[line]ma[rk] {register}

Set mark "a" at the current cursor position:

:ma a

A nice thing about "mark" is that it doesn’t move the cursor:

:111ma a
:/apple/ma a
:/Apple/ma A
:/zebra/ma z
:/Zebra/ma Z

Join
:[range]j[oin]

Rather than moving the cursor and hitting JJJJJJJJJJJJJJ or guessing "15J", use command-line joing:

:234,256j

Global
:[range]g[lobal]/pattern/ command

I don’t get fancy with global, maps, or search and replace, strongly preferring to record and replay complicated combinations (see issue 370). That said, "global," has its place. "Global" selects every line in a range which matches a pattern, and then executes an ex command on those lines. Here are some examples:

Change "T3E" to "HPC" but only on lines, from the current cursor line to the end of the file, containing the text, "Users’ Newsletter."

:.,$g/Users\’ Newsletter/s/T3E/HPC/g

Copy every line containing "Users’ Newsletter" to the end of the file.

:g/Users\’ Newsletter/t$

Delete all empty lines:

:g/^$/d

Shift matching lines, with "global" with "mark" and "copy," using multiple ex commands. This shifts every line containing BOBBY up four lines:

:g/BOBBY/mark k
move ’k-4

Help
:h[elp] [command or topic]

Help on "mark." Command-line and then normal-mode versions:

:h :ma
:h m

Help on ranges, :foldopen, and :global:

:h cmdline-ranges
:h :foldopen
:h :global

Help main page:

:h

Other Commands...

I’ve listed most of the commands I find useful, but there are many more:

:h ex-cmd-index

Miscellany

Executing multiple commands

The "|" serves as a separator between ex commands, so you can type several at once if you know what you want. E.g.:

:33,40t$
47,50t$
33,50d

:set noignorecase
%s/Apple/apple/gc
%s/Zebra/zebra/gc
set ignorecase

Repeating a command-line ex command

As described in issue 370, VIM stores the last command in a special register aptly named ":", and you can execute the contents of any register with "@". Thus, you can easily re-execute your last command line ex command:

In normal-mode, repeat the previous ex command:

@:

In command-mode, repeat the previous ex command:

:@:

Stop escaping forward slashes!!!

The pattern delimiter in "global" and "substitution" is NOT restricted to the forward slash ("/"). I always use a comma (",").

So, instead of this mess (to change /usr/local/bin to /usr/bin):

:g/\/usr\/local\/bin/s/\/local//

I type this:

:g,/usr/local/bin,s,/local,,

The delimiter can be any non-alphanumeric character except ’\’, ’"’ or ’|’, so use whatever you like, e.g.:

:g!/usr/local/bin!s!/local!!
:g#/usr/local/bin#s#/local##

If your "personal" delimiter character appears in the pattern, escape it:

:s,The time is come\, the walrus said,Let’s,

Next Steps...

The way I suggest you master VIM is to print a detailed reference card, like this, and keep it handy:

Most of the command-line commands I describe in this article aren’t on the card, so write them in the margins.

Then, whenever you’re ready for a stretch, look at the card, pick any command or feature you find intriguing, learn it and use it till it’s second nature. Then pick another... and another... ad infinitum...

Document Freedom Day

For anyone who has ever had trouble reading files in some closed-source format, there is a Document Freedom Day.

Still more git

[by Kate Hedstrom]

If you were foolish enough to be learning git only from my rantings, you would not yet have heard about tags and git bisection. It is time to remedy that situation.

Tags

Tags are a way to provide a more user-friendly name for a particular revision. You can always refer to revision numbers by their SHA1 number, but you might want to have a better way to label the code used for a particular simulation. To tag the currently checked out code:

% git tag version_2.3

You can also tag a particular revision:

git tag stable-1 f3a7081fb9

You would then be able to see your tags like:

% git tag
stable-1
version_2.3

You can later access code using:

% git checkout version_2.3

You can add a message to a tag with the "-a" argument:

% git tag -a testing-1 -m ’my first tag string’

and then see that string later with:

% git show testing-1

One thing about tags is that they only apply to your current repository. Pushing changes, then downloading them on another system will make the code corresponding to the tagged version visible elsewhere, but not the tag itself. The same code base on one machine gives:

mg57 116% git tag -l
NEP5_45
testing-1

and this on another:

332% git tag
nep5_run38

I’ve clearly missed tagging several runs between 38 and 45!

Bisection

Say you know you had a working version of the code a couple months ago. Then you bring in two months worth of updates from somewhere, including conflict resolution. You find that at the end of this, something doesn’t work. What to do? Assuming that there are a number of discrete revisions, you can ask git to help you narrow down which change it was that created difficulties for you. Here’s how:

% git bisect start
% git bisect good nep5_run38
% git bisect bad

This is saying the current version is bad, while that at tag nep5_run38 is good. If you don’t have tags, the SHA1 numbers work as well. Say things look like:

HEAD
a123
b3452
73f8g
nep5_run38

Git will check out the b3452 code and ask you to check it. Once it’s been checked, tell it either "git bisect good" or "git bisect bad". It will then check out a123 or 73f8g, accordingly. You will then need to check that one and report on it, until you are down to just one step between a "good" and a "bad" case.

When I was forced to try out the bisection, I ran into one pitfall. Not all codes compiled off the bat so I made some local changes. Because git wants to check out new codes after a "git bisect good/bad", it won’t do it when there are local modifications. Make sure you do your test, then "git reset --hard", then "git bisect good/bad". Make sure "git status" is clean before your report!

Once you get your answer about which change to focus on, clean up the bisect business with "git bisect reset".

Quick-Tip Q & A

A.[[ After I added an "echo" statement to the .bashrc of a certain machine,
[[ I was no longer able to scp to it. When I tried to scp to that
[[ machine, I would see the echo and nothing else, and the prompt returned
[[ without any indication of failure. Why did an echo statement in my
[[ .bashrc prevent scp from working?
#
# We received this tip from reader Samy Bara:
#
Old known issue, see
https://bugzilla.redhat.com/show_bug.cgi?id=20527
.
#
# Elaborating just a bit, especially for readers who don’t use web browsers:
#
The scp command reads from ~/.bashrc. It is a known issue that if the
~/.bashrc generates output, it breaks scp. The referenced RedHat ticket
was closed with status "WONTFIX" and a note that this bad behavior is
the fault of OpenSSH.

Q: I’m setting up a new ocean simulation and we have an idea for how to
handle land-fast ice. It requires reading in the land-fast ice from some
observations. My problem is that the model grid is in lat/lon and the
model reads netCDF files while the observations are in various GIS
formats. Available formats are GeoTiff, ESRI shapefiles, and Microsoft
Access Database. Any advice on dealing with these?

The University of Alaska Fairbanks is an affirmative action/equal
opportunity employer and educational institution and is a part of the University
of Alaska system.
Arctic Region Supercomputing Center (ARSC) |PO Box 756020, Fairbanks, AK 99775 | voice: 907-450-8602 | fax: 907-450-8601 | Supporting high performance computational research in science and engineering with emphasis on high latitudes and the arctic.
For questions or comments regarding this website, contact info@arsc.edu