5. The IDLWAVE Shell

The IDLWAVE shell is an Emacs major mode which permits running the IDL
program as an inferior process of Emacs, and works closely with the
IDLWAVE major mode in buffers. It can be used to work with IDL
interactively, to compile and run IDL programs in Emacs buffers and to
debug these programs. The IDLWAVE shell is built on `comint', an
Emacs packages which handles the communication with the IDL program.
Unfortunately IDL for Windows does not have command-prompt versions
and thus do not allow the interaction with Emacs(5) -- so the IDLWAVE shell currently only
works under Unix and MacOSX.

5.1 Starting the Shell

The IDLWAVE shell can be started with the command M-x
idlwave-shell. In idlwave-mode the function is bound to
C-c C-s. It creates a buffer `*idl*' which is used to
interact with the shell. If the shell is already running, C-c
C-s will simply switch to the shell buffer. The command C-c
C-l (idlwave-shell-recenter-shell-window) displays the shell
window without selecting it. The shell can also be started
automatically when another command tries to send a command to it. To
enable auto start, set the variable
idlwave-shell-automatic-start to t.

In order to create a separate frame for the IDLWAVE shell buffer, call
idlwave-shell with a prefix argument: C-u C-c C-s or
C-u C-c C-l. If you always want a dedicated frame for the shell
window, configure the variable
idlwave-shell-use-dedicated-frame.

To launch a quick IDLWAVE shell directly from a shell prompt without
an IDLWAVE buffer (e.g., as a replacement for running inside an
xterm), define a system alias with the following content:

emacs -geometry 80x32 -eval "(idlwave-shell 'quick)"

Replace the `-geometry 80x32' option with `-nw' if you prefer
the Emacs process to run directly inside the terminal window.

To use IDLWAVE with ENVI or other custom packages which change the
`IDL> ' prompt, you must change the
idlwave-shell-prompt-pattern, which defaults to `"^ ?IDL>
"'. Normally, you can just replace the `IDL' in this expression
with the prompt you see. A suitable pattern which matches the prompt
for both ENVI and IDL simultaneously is `"^ ?\\(ENVI\\|IDL\\)> "'.

5.2 Using the Shell

The IDLWAVE shell works in the same fashion as other shell modes in
Emacs. It provides command history, command line editing and job
control. The UP and DOWN arrows cycle through the input
history just like in an X terminal(6). The history is preserved
between emacs and IDL sessions. Here is a list of commonly used
commands:

UP, M-p

Cycle backwards in input history

DOWN, M-n

Cycle forwards in input history

M-r

Previous input matching a regexp

M-s

Next input matching a regexp

return

Send input or copy line to current prompt

C-c C-a

Beginning of line; skip prompt

C-c C-u

Kill input to beginning of line

C-c C-w

Kill word before cursor

C-c C-c

Send ^C

C-c C-z

Send ^Z

C-c C-\

Send ^\

C-c C-o

Delete last batch of process output

C-c C-r

Show last batch of process output

C-c C-l

List input history

In addition to these standard `comint' commands,
idlwave-shell-mode provides many of the same commands which
simplify writing IDL code available in IDLWAVE buffers. This includes
abbreviations, online help, and completion. See 4.2 Routine Info and
4.3 Online Help and 4.4 Completion for more information on these
commands.

Non-nil means UP and DOWN arrows move through command
history like xterm.

User Option:idlwave-shell-comint-settings

Alist of special settings for the comint variables in the IDLWAVE Shell.

User Option:idlwave-shell-file-name-chars

The characters allowed in file names, as a string. Used for file name
completion.

User Option:idlwave-shell-graphics-window-size

Size of IDL graphics windows popped up by special IDLWAVE command.

IDLWAVE works in line input mode: You compose a full command line, using
all the power Emacs gives you to do this. When you press RET, the
whole line is sent to IDL. Sometimes it is necessary to send single
characters (without a newline), for example when an IDL program is
waiting for single character input with the GET_KBRD function.
You can send a single character to IDL with the command C-c C-x
(idlwave-shell-send-char). When you press C-c C-y
(idlwave-shell-char-mode-loop), IDLWAVE runs a blocking loop
which accepts characters and immediately sends them to IDL. The loop
can be exited with C-g. It terminates also automatically when the
current IDL command is finished. Check the documentation of the two
variables described below for a way to make IDL programs trigger
automatic switches of the input mode.

User Option:idlwave-shell-use-input-mode-magic(nil)

Non-nil means IDLWAVE should check for input mode spells in
output.

User Option:idlwave-shell-input-mode-spells

The three regular expressions which match the magic spells for input
modes.

5.3 Commands Sent to the Shell

The IDLWAVE buffers and shell interact very closely. In addition to the
normal commands you enter at the IDL> prompt, many other special
commands are sent to the shell, sometimes as a direct result of invoking
a key command, menu item, or toolbar button, but also automatically, as
part of the normal flow of information updates between the buffer and
shell.

By default, much of this background shell input and output is hidden
from the user, but this is configurable. The custom variable
idlwave-abbrev-show-commands allows you to configure which
commands sent to the shell are shown there. For a related customization
for separating the output of examine commands, see 5.5 Examining Variables.

User Option:idlwave-shell-show-commands('(run misc breakpoint))

A list of command types to echo in the shell when sent. Possible values
are run for .run, .compile and other run commands,
misc for lesser used commands like window,
retall,close, etc., breakpoint for breakpoint
setting and clearing commands, and debug for other debug,
stepping, and continue commands. In addition, if the variable is set to
the single symbol 'everything, all the copious shell input is
displayed (which is probably only useful for debugging purposes).
N.B. For hidden commands which produce output by side-effect, that
output remains hidden (e.g., stepping through a print command).
As a special case, any error message in the output will be displayed
(e.g., stepping to an error).

5.4 Debugging IDL Programs

Programs can be compiled, run, and debugged directly from the source
buffer in Emacs, walking through arbitrarily deeply nested code,
printing expressions and skipping up and down the calling stack along
the way. IDLWAVE makes compiling and debugging IDL programs far less
cumbersome by providing a full-featured, key/menu/toolbar-driven
interface to commands like breakpoint, .step,
.run, etc. It can even perform complex debug operations not
natively supported by IDL (like continuing to the line at the cursor).

The IDLWAVE shell installs key bindings both in the shell buffer and
in all IDL code buffers of the current Emacs session, so debug
commands work in both places (in the shell, commands operate on the
last file compiled). On Emacs versions which support it, a debugging
toolbar is also installed. The toolbar display can be toggled with
C-c C-d C-t (idlwave-shell-toggle-toolbar).

5.4.1 A Tale of Two Modes

The many debugging, compiling, and examination commands provided in
IDLWAVE are available simultaneously through two different interfaces:
the original, multi-key command interface, and the new Electric Debug
Mode. The functionality they offer is similar, but the way you
interact with them is quite different. The main difference is that,
in Electric Debug Mode, the source buffers are made read-only, and
single key-strokes are used to step through, examine expressions, set
and remove breakpoints, etc. The same variables, prefix arguments,
and settings apply to both versions, and both can be used
interchangeably. By default, when breakpoints are hit, Electric Debug
Mode is enabled. The traditional interface is described first.
See section 5.4.6 Electric Debug Mode, for more on that mode.

Note that electric debug mode can be prevented from
activating automatically by customizing the variable
idlwave-shell-automatic-electric-debug.

5.4.2 Debug Key Bindings

The standard debugging key bindings are always available by default on
the prefix key C-c C-d, so, for example, setting a breakpoint is
done with C-c C-d C-b, and compiling a source file with C-c
C-d C-c. You can also easily configure IDLWAVE to use one or more
modifier keys not in use by other commands, in lieu of the prefix
C-c C-d (though these bindings will typically also be available
--- see idlwave-shell-activate-prefix-keybindings). For
example, if you include in `.emacs':

(setq idlwave-shell-debug-modifiers '(control shift))

a breakpoint can then be set by pressing b while holding down
shift and control keys, i.e. C-S-b. Compiling a
source file will be on C-S-c, deleting a breakpoint C-S-d,
etc. In the remainder of this chapter we will assume that the
C-c C-d bindings are active, but each of these bindings will
have an equivalent shortcut if modifiers are given in the
idlwave-shell-debug-modifiers variable (see section 3.2 Lesson II: Customization). A much simpler and faster form of debugging for
running code is also available by default -- see 5.4.6 Electric Debug Mode.

User Option:idlwave-shell-prefix-key(C-c C-d)

The prefix key for the debugging map
idlwave-shell-mode-prefix-map.

User Option:idlwave-shell-activate-prefix-keybindings(t)

Non-nil means debug commands will be bound to the prefix
key, like C-c C-d C-b.

User Option:idlwave-shell-debug-modifiers(nil)

List of modifier keys to use for additional binding of debugging
commands in the shell and source buffers. Can be one or more of
control, meta, super, hyper, alt, and
shift.

5.4.3 Breakpoints and Stepping

IDLWAVE helps you set breakpoints and step through code. Setting a
breakpoint in the current line of the source buffer is accomplished
with C-c C-d C-b (idlwave-shell-break-here). With a
prefix arg of 1 (i.e. C-1 C-c C-d C-b), the breakpoint gets a
/ONCE keyword, meaning that it will be deleted after first use.
With a numeric prefix greater than one (e.g. C-4 C-c C-d C-b),
the breakpoint will only be active the nth time it is hit.
With a single non-numeric prefix (i.e. C-u C-c C-d C-b), prompt
for a condition -- an IDL expression to be evaulated and trigger the
breakpoint only if true. To clear the breakpoint in the current line,
use C-c C-d C-d (idlwave-clear-current-bp). When
executed from the shell window, the breakpoint where IDL is currently
stopped will be deleted. To clear all breakpoints, use C-c C-d
C-a (idlwave-clear-all-bp). Breakpoint lines are highlighted
or indicated with an icon in the source code. Note that IDL places
breakpoints as close as possible on or after the line you specify.
IDLWAVE queries the shell for the actual breakpoint location which was
set, so the exact line you specify may not be marked. You can re-sync
the breakpoint list and display at any time (e.g., if you add or
remove some on the command line) using C-c C-d C-l

Once the program has stopped somewhere, you can step through it. The
most important stepping commands are C-c C-d C-s to execute one
line of IDL code ("step into"); C-c C-d C-n to step a single line,
treating procedure and function calls as a single step ("step over");
C-c C-d C-h to continue execution to the line at the cursor and
C-c C-d C-r to continue execution. See section 5.3 Commands Sent to the Shell, for information on displaying or hiding the breakpoint and
stepping commands the shell receives. Here is a summary of the
breakpoint and stepping commands:

C-c C-d C-b

Set breakpoint (idlwave-shell-break-here)

C-c C-d C-i

Set breakpoint in function named here (idlwave-shell-break-in)

C-c C-d C-d

Clear current breakpoint (idlwave-shell-clear-current-bp)

C-c C-d C-a

Clear all breakpoints (idlwave-shell-clear-all-bp)

C-c C-d [

Go to the previous breakpoint.

C-c C-d ]

Go to the next breakpoint.

C-c C-d C-j

Set a breakpoint at the beginning of the enclosing routine.

C-c C-d C-s

Step, into function calls (idlwave-shell-step)

C-c C-d C-n

Step, over function calls (idlwave-shell-stepover)

C-c C-d C-k

Skip one statement (idlwave-shell-skip)

C-c C-d C-u

Continue to end of block (idlwave-shell-up)

C-c C-d C-m

Continue to end of function (idlwave-shell-return)

C-c C-d C-o

Continue past end of function (idlwave-shell-out)

C-c C-d C-h

Continue to line at cursor position (idlwave-shell-to-here)

C-c C-d C-r

Continue execution to next breakpoint, if any (idlwave-shell-cont)

C-c C-d C-up

Show higher level in calling stack (idlwave-shell-stack-up)

C-c C-d C-down

Show lower level in calling stack (idlwave-shell-stack-down)

All of these commands have equivalents in Electric Debug Mode, which
provides faster access (see section 5.4.6 Electric Debug Mode).

5.4.4 Compiling Programs

In order to compile the current buffer under the IDLWAVE shell, press
C-c C-d C-c (idlwave-save-and-run). This first saves the
current buffer and then sends the command `.run path/to/file' to the
shell. You can also execute C-c C-d C-c from the shell buffer, in
which case the most recently compiled buffer will be saved and
re-compiled.

When developing or debugging a program, it is often necessary to execute
the same command line many times. A convenient way to do this is
C-c C-d C-y (idlwave-shell-execute-default-command-line).
This command first resets IDL from a state of interrupted execution by
closing all files and returning to the main interpreter level. Then a
default command line is send to the shell. To edit the default command
line, call idlwave-shell-execute-default-command-line with a
prefix argument: C-u C-c C-d C-y. If no default command line has
been set (or you give two prefix arguments), the last command on the
comint input history is sent.

User Option:idlwave-shell-mark-stop-line(t)

Non-nil means mark the source code line where IDL is currently
stopped. The value specifies the preferred method. Legal values are
nil, t, arrow, and face.

User Option:idlwave-shell-overlay-arrow(">")

The overlay arrow to display at source lines where execution halts, if
configured in idlwave-shell-mark-stop-line.

User Option:idlwave-shell-stop-line-face

The face which highlights the source line where IDL is stopped, if
configured in idlwave-shell-mark-stop-line.

5.4.5 Walking the Calling Stack

While debugging a program, it can be very useful to check the context in
which the current routine was called, for instance to help understand
the value of the arguments passed. To do so conveniently you need to
examine the calling stack. If execution is stopped somewhere deep in a
program, you can use the commands C-c C-d C-UP
(idlwave-shell-stack-up) and C-c C-d C-DOWN
(idlwave-shell-stack-down), or the corresponding toolbar buttons,
to move up or down through the calling stack. The mode line of the
shell window will indicate the position within the stack with a label
like `[-3:MYPRO]'. The line of IDL code at that stack position
will be highlighted. If you continue execution, IDLWAVE will
automatically return to the current level. See section 5.5 Examining Variables,
for information how to examine the value of variables and expressions on
higher calling stack levels.

5.4.6 Electric Debug Mode

Even with a convenient debug key prefix enabled, repetitive stepping,
variable examination (see section 5.5 Examining Variables), and other
debugging activities can be awkward and slow using commands which
require multiple keystrokes. Luckily, there's a better way, inspired
by the lisp e-debug mode, and available through the Electric
Debug Mode. By default, as soon as a breakpoint is hit, this minor
mode is enabled. The buffer showing the line where execution has
halted is switched to Electric Debug Mode. This mode is visible as
`*Debugging*' in the mode line, and a different face (violet by
default, where color is available) for the line stopped at point. The
buffer is made read-only and single-character bindings for the most
commonly used debugging commands are enabled:

Go to the previous breakpoint in the file (idlwave-shell-goto-previous-bp)

]

Go to the next breakpoint in the file (idlwave-shell-goto-next-bp)

j

Set breakpoint at beginning of enclosing routine (idlwave-shell-break-this-module)

k

Skip one statement (idlwave-shell-skip)

m

Continue to end of function (idlwave-shell-return)

n

Step, over function calls (idlwave-shell-stepover)

o

Continue past end of function (idlwave-shell-out)

p

Print expression near point or in region with C-u p (idlwave-shell-print)

q

End the debugging session and return to the Shell's main level
(idlwave-shell-retall)

r

Continue execution to next breakpoint, if any (idlwave-shell-cont)

s or SPACE

Step, into function calls (idlwave-shell-step)

t

Print a calling-level traceback in the shell

u

Continue to end of block (idlwave-shell-up)

v

Turn Electric Debug Mode off
(idlwave-shell-electric-debug-mode)

x

Examine expression near point (or in region with C-u x)
with shortcut of examine type.

z

Reset IDL (idlwave-shell-reset)

+ or =

Show higher level in calling stack (idlwave-shell-stack-up)

- or _

Show lower level in calling stack (idlwave-shell-stack-down)

?

Help on expression near point or in region with C-u ?
(idlwave-shell-help-expression)

C-?

Show help on the commands available.

Most single-character electric debug bindings use the final keystroke
of the equivalent multiple key commands (which are of course also
still available), but some differ (e.g. t,q,x).
Some have additional convenience bindings (like SPACE for
stepping). All prefix and other argument options described in this
section for the commands invoked by electric debug bindings are still
valid. For example, C-u b sets a conditional breakpoint, just
as it did with C-u C-c C-d C-b.

You can toggle the electric debug mode at any time in a buffer using
C-c C-d C-v (v to turn it off while in the mode), or from
the Debug menu. Normally the mode will be enabled and disabled at the
appropriate times, but occassionally you might want to edit a file
while still debugging it, or switch to the mode for conveniently
setting lots of breakpoints.

To quickly abandon a debugging session and return to normal editing at
the Shell's main level, use q (idlwave-shell-retall).
This disables electric debug mode in all IDLWAVE buffers(7). Help is
available for the command shortcuts with C-?. If you find this
mode gets in your way, you can keep it from automatically activating
by setting the variable idlwave-shell-automatic-electric-debug
to nil, or 'breakpoint. If you'd like the convenient
electric debug shortcuts available also when run-time errors are
encountered, set to t.

User Option:idlwave-shell-automatic-electric-debug('breakpoint)

Whether to enter electric debug mode automatically when a breakpoint
or run-time error is encountered, and then disable it in all buffers
when the $MAIN$ level is reached (either through normal program
execution, or retall). In addition to nil for never, and
t for both breakpoints and errors, this can be
'breakpoint (the default) to enable it only at breakpoint
halts.

User Option:idlwave-shell-electric-zap-to-file(t)

If set, when entering electric debug mode, select the window displaying
the file where point is stopped. This takes point away from the shell
window, but is useful for immediate stepping, etc.

5.5 Examining Variables

Do you find yourself repeatedly typing,
e.g. print,n_elements(x), and similar statements to remind
yourself of the type/size/structure/value/etc. of variables and
expressions in your code or at the command line? IDLWAVE has a suite
of special commands to automate these types of variable or expression
examinations. They work by sending statements to the shell formatted
to include the indicated expression.

These examination commands can be used in the shell or buffer at any
time (as long as the shell is running), and are very useful when
execution is stopped in a buffer due to a triggered breakpoint or error,
or while composing a long command in the IDLWAVE shell. In the latter
case, the command is sent to the shell and its output is visible, but
point remains unmoved in the command being composed -- you can inspect
the contituents of a command you're building without interrupting the
process of building it! You can even print arbitrary expressions from
older input or output further up in the shell window -- any expression,
variable, number, or function you see can be examined.

If the variable idlwave-shell-separate-examine-output is
non-nil (the default), all examine output will be sent to a
special `*Examine*' buffer, rather than the shell. The output of
prior examine commands is saved. In this buffer c clears the
contents, and q hides the buffer.

The two most basic examine commands are bound to C-c C-d C-p, to
print the expression at point, and C-c C-d ?, to invoke help on
this expression(8). The expression at point is
either an array expression or a function call, or the contents of a
pair of parentheses. The selected expression is highlighted, and
simultaneously the resulting output is highlighted in the shell.
Calling the above commands with a prefix argument will use the current
region as expression instead of using the one at point. Two prefix
arguments (C-u C-u C-c C-d C-p) will prompt for an expression.

For added speed and convenience, there are mouse bindings which allow
you to click on expressions and examine their values. Use
S-Mouse-2 to print an expression and C-M-Mouse-2 to invoke
help (i.e. you need to hold down META and CONTROL while
clicking with the middle mouse button). If you simply click, the
nearest expression will be selected in the same manner as described
above. You can also drag the mouse in order to highlight
exactly the specific expression or sub-expression you want to examine.
For custom expression examination, and the powerful customizable
pop-up examine selection, See section 5.6 Custom Expression Examination.

The same variable inspection commands work both in the IDL Shell and
IDLWAVE buffers, and even for variables at higher levels of the calling
stack. For instance, if you're stopped at a breakpoint in a routine,
you can examine the values of variables and expressions inside its
calling routine, and so on, all the way up through the calling stack.
Simply step up the stack, and print variables as you see them
(see section 5.4.5 Walking the Calling Stack, for information on stepping back
through the calling stack). The following restrictions apply for all
levels except the current:

Array expressions must use the `[ ]' index delimiters. Identifiers
with a `( )' will be interpreted as function calls.

N.B.: printing values of expressions on higher levels of the calling
stack uses the unsupported IDL routine ROUTINE_NAMES,
which may or may not be available in future versions of IDL. Caveat
Examinor.

User Option:idlwave-shell-expression-face

The face for idlwave-shell-expression-overlay.
Allows you to choose the font, color and other properties for
the expression printed by IDL.

User Option:idlwave-shell-output-face

The face for idlwave-shell-output-overlay.
Allows to choose the font, color and other properties for the most
recent output of IDL when examining an expression."

User Option:idlwave-shell-separate-examine-output(t)

If non-nil, re-direct the output of examine commands to a special
`*Examine*' buffer, instead of in the shell itself.

5.6 Custom Expression Examination

The variety of possible variable and expression examination commands is
endless (just look, for instance, at the keyword list to
widget_info()). Rather than attempt to include them all, IDLWAVE
provides two easy methods to customize your own commands, with a special
mouse examine command, and two macros for generating your own examine
key and mouse bindings.

The most powerful and flexible mouse examine command of all is
available on C-S-Mouse-2. Just as for all the other mouse
examine commands, it permits click or drag expression selection, but
instead of sending hard-coded commands to the shell, it pops-up a
customizable selection list of examine functions to choose among,
configured with the idlwave-shell-examine-alist
variable(9). This variable is a list of key-value pairs (an
alist in Emacs parlance), where the key gives a name to be
shown for the examine command, and the value is the command strings
itself, in which the text ___ (three underscores) will be
replaced by the selected expression before being sent to the shell.
An example might be key Structure Help with value
help,___,/STRUCTURE. In that case, you'd be prompted with
Structure Help, which might send something like
help,var,/STRUCTURE to the shell for output.
idlwave-shell-examine-alist comes configured by default with a
large list of examine commands, but you can easily customize it to add
your own.

In addition to configuring the functions available to the pop-up mouse
command, you can easily create your own customized bindings to inspect
expressions using the two convenience macros
idlwave-shell-examine and idlwave-shell-mouse-examine.
These create keyboard or mouse-based custom inspections of variables,
sharing all the same properties of the built-in examine commands.
Both functions take a single string argument sharing the syntax of the
idlwave-shell-examine-alist values, e.g.:

Now pressing f9, or middle-mouse dragging with the
SUPER key depressed, will print the dimensions of the nearby or
highlighted expression. Pressing f10 will give the type string,
and f11 will show the contents of a nearby structure. As you can
see, the possibilities are only marginally finite.

User Option:idlwave-shell-examine-alist

An alist of examine commands in which the keys name the command and are
displayed in the selection pop-up, and the values are custom IDL examine
command strings to send, after all instances of ___ are replaced
by the indicated expression.