8 Answers
8

Note that if you want to send data to
the process’s stdin, you need to
create the Popen object with
stdin=PIPE. Similarly, to get anything
other than None in the result tuple,
you need to give stdout=PIPE and/or
stderr=PIPE too.

I missed that warning. I'm glad I asked (even though I thought I had the answer).
–
Daryl SpitzerOct 3 '08 at 16:02

5

This is NOT a good solution. In particular, you cannot asynchronously process p.stdout.readline output if you do this since you'd have to wait for the entire stdout to arrive. It's is also memory-inefficient.
–
OTZAug 20 '10 at 21:59

Per the subprocess documentation: Warning - Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.
–
Jason MockAug 25 '10 at 20:49

fyi, tempfile.SpooledTemporaryFile.__doc__ says: Temporary file wrapper, specialized to switch from StringIO to a real file when it exceeds a certain size or when a fileno is needed.
–
Doug FAug 9 '13 at 20:18

Because shell=True is so commonly used for no good reason, and this is a popular question, let me point out that there are a lot of situations where Popen(['cmd', 'with', 'args']) is decidedly better than Popen('cmd with args', shell=True) and having the shell break the command and arguments into tokens, but not otherwise providing anything useful, while adding a significant amount of complexity and thus also attack surface.
–
tripleeeOct 29 '14 at 13:43

Beware that Popen.communicate(input=s)may give you trouble ifsis too big, because apparently the parent process will buffer it before forking the child subprocess, meaning it needs "twice as much" used memory at that point (at least according to the "under the hood" explanation and linked documentation found here). In my particular case,swas a generator that was first fully expanded and only then written tostdin so the parent process was huge right before the child was spawned,
and no memory was left to fork it: