Python Scripts as a Replacement for Bash Utility Scripts

Often in Python scripts that are used on the command line, arguments
are used to give users options when they run a certain command. For
instance, the head command takes a
-n argument that takes the number
following it and prints only that number of lines. Each argument that
is provided to a Python script is exposed through the
sys.argv array,
which can be accessed by first importing sys. The
code below shows how to take
single words as arguments. This program is a simple adder, which takes two
number arguments and adds them, and prints that out to the user. However,
this format of taking in command-line arguments is rather basic. It is
easy to make mistakes—for instance, pass two strings, such as
hello
and world, to this command, and you will start to get
errors:

Thankfully, Python has a number of modules to
deal with command-line arguments. My personal favorite is
OptionParser. OptionParser
is part of the optparse module that is provided by the standard
library. OptionParser allows you to do a range of very useful things with
command-line arguments:

Specify a default if a certain argument is not provided.

It supports both argument flags (either present or not) and arguments
with values (-n 10000).

It supports different formats of passing arguments—for example, the
difference between -n=100000 and -n 100000.

Let's use the OptionParser to enhance the sending-mail script. The
original script had a lot of variables hard-coded into place, such as the
SMTP details and the users' login credentials. In the code provided below,
command-line arguments are used to pass in these variables:

This script shows the usefulness of OptionParser. It provides a simple,
easy-to-use interface for command-line arguments, allowing you to define
certain properties for each command-line option. It also allows you to
specify default values. If certain arguments are not provided, it allows
you to throw specific errors.

So what have you learned? Instead of replacing a series of bash commands
with one Python script, it often is better to have Python do only
the heavy lifting in the middle. This allows for more modular and
reusable scripts, while also tapping into the power of all that Python
offers. Using stdin as a file object allows Python to read input, which
is piped to it from other commands, and writing to stdout allows it to
continue passing the information through the piping system. Combining
information like this can make for some very powerful programs. The
examples I have given here are all for a fictional service that logs
to a file.

As a real-world example, recently I have been working with gigabytes of
CSV files that I have been converting using a Python script to a file
that contains SQL commands to insert the information. To understand the
sort of data I'm concerned with here, I ran the data for a single
table, and the script took 23 hours to execute and generated an SQL file that
was 20GB in size. The advantage of using a Python script in the fashion
described in this article is that the whole file does not need to be read
into memory. This means that an entire 20GB+ file can be processed one
line at a time. Also it is easier to think about a problem when each
step (reading, sorting, manipulation and writing) is separated into these
logical steps. The guarantee that each of these commands, which are part
of the core utilities of UNIX-like environment, is efficient and stable
helps the entire experience to be more stable and secure.

The other benefit is that there is no hard-coded file that is read
in. Often having the flexibility to pass it strings rather than the
concept of files is very powerful. For instance, if 20,000
lines through a certain file, the script breaks, instead of re-running
the script from the start, tail can be used to read only from the line
on which the script failed.

There are a lot of aspects to Python in the shell that go beyond the
scope of this article, such as the os module and the
subprocess
module. The os module is a standard library function that holds a lot
of key operating system-level operations, such as listing directories and
stating files, along with an excellent submodule os.path that deals
with normalizing directories paths. The subprocess module allows Python
programs to run system commands and other advanced operations, such as
handling piping as described above within Python code between spawned
processes. Both of these libraries are worth checking out if you intend
to do any Python shell scripting.

Richard Delaney is a software engineer with Demonware Ireland. Richard works on back-end Web
services using Python and the Django Web framework. He has been an avid Linux user and evangelist for the past five years.

Great post. I must say thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.hot girls

WONDERFUL Post.thanks for share..more wait .. …There are certainly a lot of details like that to take into consideration. That is a great point to bring up. I offer the thoughts above as general inspiration but clearly there are questions like the one you bring up where the most important thing will be working in honest good faith. I don?t know if best practices have emerged around things like that, but I am sure that your job is clearly identified as a fair game. Both boys and girls feel the impact of just a moment’s pleasure, for the rest of their lives.

Between that system, the taillights – which according to Audi, transforms the car’s “rear end into a large, continuous light surface, with innumerable small points of light flickering 12W LED work light like a swarm of fish” — there’s also headlamp development for which to account. On the volition of the gorgeous, serpentine headlamps on its current models, the brand has gracefully become one of the most coveted imports in suburbs and city alike.

It will do all the logging and graphing the inexpensive software does but with the Ford specific bundle you can command engine actuators (I think) and run diagnostic routines. Very powerful diagnostic tool. That leaves scanners. There are many many inexpensive ones. Given the other options available, I would skip the cheap ones. Dual Core Tablet PC

Incorporating is one of the best ways a business owner can protect his or her personal assets. Most people choose to incorporate solely for this reason, but there are other advantages as well. For example, the corporate business structure saves you money in taxes, provides greater business flexibility, avoids audit chances, better itemization and lets you more easily raise capital.

Wow! The article topic is about how difficult it is to interpret shell scripts and then renders, yet another convoluted syntactical language. The problem with most script languages are that, they lack strict programming semantics. Granted most newbie programmers are educated using C++, which is main hype around Python, but the real problem of simplicity, isn't being resolved. If a language offers a multitude of ways to code, it forces the programmer to open a book again. How many programmers can honestly say, they are fluent in all high/low level languages? I suggest to you that "C" is the basic language we all know and that, TCL is the most compatible syntactically, is shell universal, not to mention strict and simple. It has all the good qualities of a scripting language, an interpreter (type tclsh), can be used on a shell command line to pipe with other shell commands, has a good GUI package/s, has a great debugger, not to mention adheres to strict syntactical rules. The problem with programmers these days are that they forget the KISS philosophy.

It sound that this is a perfect combination between mobile phones and automotive, no doubt, android industries have begun to enter our life, and I have been thinking recently, if Obd2 auto diagnostic software can be integrated into mobile applications, it will greatly change our lives, and even bring about a revolution again. Now many shop began selling android Obd2 auto diagnostic software product, I was no exception, welcome to my shop to see what the latest android Obd2 auto diagnostic software products. VOLVO VCADS Pro 2.4

That is very interesting Smile I love reading and I am always searching for informative information like this. This is exactly what I was looking for. Thanks for sharing this great articlefree cell phone spy softwarecell spyware

Aftermarket products can cause for Check Engine lights to light. This is only caused by products improperly installed or manufactured. We have heard stories about the Check Engine Light illuminating when swapping out exhausts or intakes, but these codes being set will not affect your power output. Truck Diagnostic Software

Hi there! I know this iѕ somеwhаt off topic but
I was wonԁering which blog plаtfοrm aге you using for this ωebѕite?

I'm getting sick and tired of Wordpress because I've hаԁ prοblems with hackеrѕ аnԁ I'm looking at alternatives for another platform. I would be awesome if you could point me in the direction of a good platform.

Great article! This is the type of information that are meant to be shared around the net.
Disgrace on Google for now not positioning this submit upper!
Come on over and seek advice from my site . Thank you studio lighting and mind machine

UNIX/Linux is all about choices. I pick between Python and Bash all the time (with a sprinking of awk). The more serious the task or the more likely the script will have a long life, the more likely it is that I will use Python.

The one thing I will say to support the shell and pipes approach is how easy it is to incrementally debug your work. At any step, replacing the remainder of the pipeline with more gives you a quick way to see how you are doing. While tossing in print statements in Python is not hard, it just is more typing.

Depending on your programming knowledge, the right approach for you will probably be different. But, that said, if you are building something that is not write-only (that is, write the code, run it and throw it away) picking a scripting language such as Python or Ruby will generally pay off in the long run.

I recently made the leap into python programming and I have to say that it is a easy language to use. Forcing code structure in a language was pure genious if you ask me.

I like the fact that core "modules" are included out of the box helps when you need to write a program that must function across hundreds of linux servers. You don't have this luxury with Perl. Using yum or aptget on servers with different OS patch levels, or lacking internet connections, firewall issues, etc is too much of a headache.
Shell scripting can be a pain too depending on what shell you are using (Bash, Korn, etc) and the personal preferences of the admin responsible for that box. The minute differences in syntax can cause hours of troubleshooting due to spaces, braces, brackets, character case...

Python is defintely a great tool to use if you need to write scipts/programs that must be used widely and interpreted by many.

An excellent post. The post affects a lot of urgent issues in our minds. We can not be indifferent to these problems. Your article gives the light in which we are able to watch our real life. Keep it up. Find me a lover

Not to be snarky, but regarding "You will find in a lot of the bsd operating systems, the unix commands have less command line arguments.":

This site is LinuxJournal.com, is it not?

One would think the articles are geared towards the Linux, and not any *nix, community. It doesn't hurt to revise an article after it's been published, especially when it'll remain published indefinitely.

If using shortened switches clouds readability, one may always resort to the longer switches, e.g., --unique rather than -u for the sort command.