pty

(note: you're probably looking for pexpect, below. This is the lower-level part of that)

When a program wants to be interactive, you want something like the pty module, to interact with the program via this a pseudoterminal. (When running non-interactive commands, subprocess does everything you want, and is probably much easier to deal with)

The pty module mostly just wraps existing C functions,
adds some convenience,
though is not always friendly or well documented.

Before you write a custom pty-based class around your specific command -- which can make sense in itself -- you may want to check what pexpect can do for you.

(pexpect wraps pty with some pure-python convenience functions, geared to easier pattern-based interactions. Notes on pexpect are below those on pty)

communication

pty.fork() gives you a file descriptor object that is tied to the child's stdin that you can os.read() on, and stdout which you can os.write() on.

You may want to poll the file descriptor, to avoid blocking reads.
See e.g. [1]}} -- and e.g. read pty.spawn()'s implementation, as it select()s on the fd and only calls the callback when there is something to do.

the functions

pty.fork() - creates a child which sees the calling process as its pseudoterminal master

returns a pair (child_pid, fd)

in the master process, child_pid is the subprocess's PID, and fd is connected to the child's controlling terminal

you can use os.read() and os.write() on the fd. That's stdout and stdin; stderr is not handled(verify)

in the child process, child_pid is 0, and fd isn't valid

You'll probably use one of the os.exec* functions in the child

basically consists of pty.openpty() plus os.fork(), and fiddling with the standard streams

pty.spawn(argv_string[, master_read[, stdin_read]])

returns nothing

spawns child process, sets its controlling terminal to be us

Not useful for interaction, but nice when you're looking just to fool a program into thinking it's in a pty

master_read and stdin_read arguments are functions that take a file descriptor and return (by default) 1KB of data from it. These arguments seem to exist to let you intercept the data and do something more with the data (e.g. this example logs the output to a file)

pty.openpty() - creates new pty (like os.openpty, but is slightly more portable than it ()(verify))

returns a 2-tuple: the file descriptors for master and slave end(verify)

Examples

#!/usr/bin/python# I have this in a script named logbatch"Need a command to run"'%s.log'# executable name + '.log'"\nRefusing to overwrite existing log %r\n\nRename or remove it and try again\n"'wb'# probably'Writing log to %r\n''Log file for command: %r\n---------------------\n'%' ''Done with logging %r\n'

If I run

logbatch ls -l

, this script itself runs

ls -l

and write its stdin+stdout to ls.log(It was made for automatically generated batch scripts, and to avoid typing something like scriptname_1 >& scriptname_1.log &. And no,

See also

pexpect

pexpect lets you deal cleverly with things spawned via pty.spawn.

(It also has a module that eases remote interaction via SSH)

Quick intro

# The simplest intraction is a "wait until you see this, then send this" thing:'ftp ftp.openbsd.org')
p.expect('[Nn]ame''anonymous')
p.expect('[Pp]assword''noah@example.com')# etc.# But when responses are conditional or otherwise flexible, this stops working.# If you hand in a list, the return value is the index to which of the patterns matched.# This makes it easier to deal with errors, timeouts # and varying code paths (you could build a rule system or state machine)
p = pexpect.spawn('ftp ftp.openbsd.org''ame', 'assword','[#\$]''refused''sending username''foo''sending password''pw12''looks like a prompt''put myfile''program finished''program did not respond fast enough''connection refused'# note: not a working example, but you get the idea

Timeout:

defaults to 30 seconds.

You can change that default

you can specify a timeout on each expect() call

By default, timeout is raised as an exception.

If you added pexpect.TIMEOUT to the list of patterns, you get the according index instead of an exception.

spawn()

When you have arguments, you can put it in the single command string, or hand them in as a list of strings, so e.g.

You could use full paths, or (in the first style) use /bin/env, but pexpect tries to imitate the behaviour of

which

anyway, so ideally will Do What You Want.

Spawn mostly just forks and execs, so if you want to use shell features in your command, you explicitly have to hand your command to a shell. That means handing in something like 'bash -c "ls > test "' (and more escaping fun)

On the environment:

The process inherits from the spawning python process - i.e. os.environ. The simplest way to make a specific environment is to alter that

If you use the env parameter on spawn, it sends that instead of os.environ. The cleanest way to hand in a tailored environment is probably to make a copy of os.environ and alter it for specific spawn(env=thatcopy) calls.

expect()

# The arguments to expect:

pattern can be:

a string type - will be compiled as regular expressions

an already-compiled regexp object, used as-is

pexpect.EOF - return this, rather than raise an exception on EOF

pexpect.TIMEOUT - return this, rather than raise an exception on timeout

a list of any of these

return value will be the index of the matching pattern. If nothing matches, you will get a timeout as an exception - or as an index, if it's explicitly in your list (what happens if nothing matches and timeout==None?(verify))

A timeout value of -1 means fallback to the class default, which seems to be 30 (seconds).
You could raise or lower it, or block indefinitely by using timeout=None.

searchwindowsize: how far back to search within the incoming data buffer.
Default is to search the whole thing (since the last match), which can be unnecessarily slow if, say, you want to match the 'done' message after a large bulk of debug output)

Returns:

If you didn't hand in a list: 0

If you hand in a list: the index into the first applicable match (so if one of your patterns is a longer version of another, you probably want it first)

After expect() returns, you can inspect the contents of the latest bit of input buffer, probably one or more of:

p.before - the text before the match

p.match - the matched text

p.after - the text after the match

Instead of using this general-purpose expect(), you can use some of the underlying functions:

expect_exact() - skips regexp stuff, so can be faster when just doing simple substring tests

expect_list() - when your list contains only compiled regexps (or EOF or TIMEOUT), you can skip some of the overhead and go straight to the matching

Reading data without expect

This article/section is a stub — probably a pile of half-sorted notes, is not well-checked so may have incorrect bits. (Feel free to ignore, fix, or tell me)

Instead of using expect(), you can do the work yourself, after one of:

Not waiting indefinitely

This often isn't enough for a "okay moving on now" thing, yet is preferable when you want to report how long the process has been at it,
terminate() it when you know it's pointless or when a program would run forver but you want to stop it, etc.

Details you may care about

isalive()

- tests whether the child process is still there

wait()

- blocking wait for child to exit. Note that a child is considered alive until all its output is read, so wait() is only useful if you know it won't.

In most cases, expect(pexpect.EOF, timeout=None) is what you'ld want instead of wait() (verify)

close()

- closes connection with child. If the child process had not closed its streams, this will look like a broken pipe to it (if and when it tries to use them), which often means it doesn't exit cleanly.

if you don't, the handle will stay open. If you run a lot of subprocesses via pexpect, you will run out.