Pages

Friday, December 19, 2014

Shell script "libraries"

If you are a security profesional (or an IT profesional) probably you -like us- are constantly writing shell scripts, so that you can automate certain tasks in your linux (or unix) environment.

We don't usually use shell scripting to write complex applications (although some shell scripts become quite big), but we do use it extensively to create some "utilities" or little tools to quickly fulfill certain needs that arise along the way.

This happens to us all the time when doing pentesting. Very often, we have to write a shell script very quickly just to solve a particular problem, so we write it as fast as possible, without regard to any software design aspect. When you do this, you know that that is not right way to write programs, but you accept it because you think the extra work that would entail doing it well is not worth it, and you prefer to have a quick working result over a well designed code.

An obvious consequence is that you end up writing the same piece of code again and again. One of the most infamous examples that applies to our case is the argument parsing function: we cannot count the number of times we have written a function to handle script options and arguments and display usage help in a way that is reasonably comfortable for us.

During the latest few months, we have been working on a job that has required us to write (and use) many shell scripts, and this time, since we suspected in advance that that would be the case, we decided to take a -let's say- cleaner approach: we decided to write what we call "shell script libraries", which turned out to be a big help for us with the aforementioned situation.

These "shell script libraries" are sets of shell functions that you can import and use from within your shell scripting code, and some of the functions can be useful even if invoked directly from the shell command line.

In this article we present the following shell libraries:

lk_option_parser.sh

lk_math.sh

lk_net.sh

lk_option_parser.sh

We started out by writing an option parser library. If your shell script needs to be able to behave in different ways depending on its invocation or if you need to pass information to it, you usually achieve this through the use of options and/or arguments. We liked the way this is handled in libraries that you can find in C or python languages, so we tried to write something similar. The library that we have written is intented to be generic and easy to use.

Note: Perhaps there is something similar out there, but none of the code we found and tested happened to match exactly what we were looking for

To use the library you have to download it and put it in a directory that is in your PATH environmental variable (or in the same directory as the invoking shell script).

Then, source it from within your code, for example as follows:

. lk_option_parser.sh || exit 1

Then, call add_program_option as many times as options you have to handle, in this way:

Note: In this context we use option and argument as synonyms; see considerations below

add_program_option "-h" "--help" "Shows this help." "NO" "NO"

where:

-h is the short flag of the option
--help is the long flag of the option
"Shows this help" is the explanation that will appear when the usage is shown
"NO" means that this is not a mandatory option
"NO" means that this option doesn't have an associated value

After you have all your options added, you just call:

parse_program_options $@

And then you may call:

show_program_usage "-h" && exit 0

Which will test if "-h" (or "--help") is present and, in that case, will show program usage and then exit. You can also specify no arguments to show_program_usage in which case no test will be performed.

If latter in your code you want to know if an option is present, you can do it like this:

if is_option_present "-h"
then
...
fi

And if you want to get the value for a specific option, you can do it in this way:

_myvar=`get_option_value "-h"`

_myvar will take the value associated to the option. A value is everything between the option and the next short or long option, or the end of the command line. Obviously in this example _myvar will simply be assigned an empty string.

That's _almost_ everything you need to know to use the library! In the code comments you have deeper explanation of the functions, although you probably won't need it.

Let us add just a couple of considerations we think you should be aware of if you are considering using the library:

The library is written for bash, because that is the shell interpreter that we use, and we haven't tested it on other interpreters. Perhaps it could be re-written in a more universal way, but we have no plans to move in that direction because, at least for now, bash is enough for us.

We know there is much discussion about the right terminology regarding arguments, options and parameters. Please note that, arbitrarily, we decided to use the terms "option", "argument" and "parameter" as synonyms in the context of our shell scripting libraries, and we, also arbitrarily, decided that all options would always include an explicit switch (e.g: "-h", "--help"), some of them with an associated value (e.g: "-i INTERFACE") and some without (e.g: "-h" for help or "-v" for verbose), and finally, we also decided that each option will be either mandatory (its presence will be required) or optional. Please note that therefore, in this context, "option" does not mean "optional" :-)

The lk_option_parser.sh library worked so well for us that we decided to take the same approach to tackle other problems, and so we started two more libraries that are described in the following sections. They are far from being complete, but our idea is to continue expanding them, and any new libraries we may find interesting to create, with ever growing functionality.

lk_math.sh

lk_math.sh is a library that will contain mathematical utilities. At the present moment, it just includes the following functions:

Conclusion and future work

We found these small shell libraries to be really useful for us, and so we thought we would share them. We hope you find it useful. You are free to use them in almost any way you see fit, since we are publishing them under the GPLv3 license.

Obviously, the code can be improved and expanded, and while we will certainly do so, we would also be more than happy to get your comments and contributions, which we would study and eventually include in the code, giving you the appropriate credit, of course.

67 comments:

Hi friends, This is Christy from Chennai. I did Unix certification course in Chennai at Fita academy. This is really useful for me to make a bright career. Suppose if anyone interested to learn Unix Training in Chennai please visit Fita academy located at Chennai.

Hi, I have read your blog and I got a useful information from this blog. Thanks for sharing, keep posting..Salesforce is a cloud based CRM software. Today's most of the IT industry use this software for customer relationship management.RegardsSalesforce Course in Chennai

SAP is one of the customer relationship management software and it support all end to end customer related process. To know more details about SAP modules please refer the following site.Regards..SAP Training Chennai

Really awesome blog. Software testing is a method of executing the application or program with the intent of searching the software errors. Testing Training in Chennai offering this course at reasonable cost.

There are lots of information about latest technology and how to get trained in them, like Hadoop Training Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies(Hadoop Training in Chennai). By the way you are running a great blog. Thanks for sharing this. FITA chennai reviews

This information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic.Regards,Regards

Thanks for your great article post.You shared more informative and interesting article.We are providing best Salesforce training course in our institute at reasonble price with placement support.The people who are interested to learn salesforce,can join in our institute.For more details reach us @Salesforce developer training in Chennai

well defined expalanation about linux commands of its shell scripts. helpful to our candidates to understand about the shell scriopts information. keep updating more about shell scripts. thanks.veritas volume manager training in chennai

too good piece of information, I had come to know about your site from my friend sajid, bangalore,i have read atleast 7 posts of yours by now, and let me tell you, your web-page gives the best and the most interesting information. This is just the kind of information that i had been looking for, i'm already your rss reader now and i would regularly watch out for the new post, once again hats off to you! Thanks a lot once again, Regards, informatica mdm training in hyderabad,informatica training in hyderabad

These provided information was really so nice,thanks for giving that post and the more skills to develop after refer that post. Your articles really impressed for me,because of all information so nice.

I am not sure the place you are getting your information, however good topic. I needs to spend some time studying more or understanding more. Thank you for wonderful information I was in search of this info for my mission.

nice posts...SAS Institute introduced the SAS Certified Professional Program,training proper understanding of how the SAS software works. Among the five certification programs that SAS Institute has come up with, SAS training can be considered as the entry point into the big data and the data analytics industry.SAS online training in hyderabad

It's like you read my mind! You seem to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a little bit, but instead of that, this is fantastic blog. A great read. I will definitely be back.

Thank you for having taken your time to provide us with your valuable information relating to your stay with us.we are sincerely concerned.., Most importantly, you Keepit the major... seo company in chennai

Superb. I really enjoyed very much with this article here. Really it is an amazing article I had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article.thank you for sharing such a great blog with us. expecting for your.Digital Marketing Company in India

Very Nice Blog I like the way you explained these things. I’ve been looking for ways to improve my website and overall rankings.I hope your future article will help me further.Take SEO Training in Chennai to mould yourself.