Thursday, February 28, 2013

If you've ever wondered how Unix systems identify files, you might be surprised to learn that file names really aren't an important factor. Unix systems reach into files looking for special codes called "magic numbers" to figure out what your files really are. They also examine other file content to pick out additional details. - See more at: http://www.itworld.com/operating-systems/344491/unix-under-spell-magic-numbers#sthash.QJPMtyqh.dpuf

Magic numbers play a very important role on Unix systems. They help the OS recognize details about files that aren't obvious even to Unix wizards.
What are magic numbers?

Magic numbers are values, not generally visible unless you dump file contents using a command like od that displays file contents in hex, that serve as signatures for file type identification. Some are fairly obvious once you see them -- like if you spot "\x89PNG" in a file dump. Others won't offer much of a clue.

In general, they are special values set up at or near the beginning or nearby in files that allow command like file to distinguish files by their type so even if you call a jpg file my.image, the OS can still figure out what it is.
How are they used?

The tool for file identification is a command named "file". Use the file command to examine your oddly named my.image file and might just tell you something like this:

$ file my.image
my.image: JPEG image data, JFIF standard 1.01

The "JFIF" tag in this description stands for JPEG File Interchange Format. This output tells you that this complies. This tells you that the magic number that is associated with JPEG files is both stored in this file and stored in the right location to identify the file. The same value at some other location would have no effect and would likely be coincidental.

Use file to examine a file that isn't a JPEG file, but has been assigned a .jpg file extension and the file command comes through and identifies this file as an executable.

Most magic numbers aren't readable unless you use a tool such as od to examine the file's contents. The exception is the shebang line that we use in scripts to identify the shell to be used when they are run. This serves as a kind of magic number on its own and is used by the file command as such.

For most files, lines in the /usr/share/file/magic file describe what the file command needs to look for. The line for a PNG file, for example, looks like this:

0 string \x89PNG PNG image data,

This line tells us that there's no offset (the magic number appears at the beginning of the file),

PNG files begin with this 8-byte signature which is very obvious in the output shown above.

\211 P N G \r \n \032 \n

Magic numbers help ensure that Unix systems have more to go on than file names when trying to identify file content. Examine the /usr/share/file/magic on your Linux system and you'll notice that there's a lot of information there about different types of files.
- See more at: http://www.itworld.com/operating-systems/344491/unix-under-spell-magic-numbers#sthash.QJPMtyqh.dpuf

School budgets never seem to get any larger, but one way
educational institutions may be able to cut costs is by deploying open
source software. The open source community has developed applications
that educators can use directly in the classroom, apps that are great
for use at home and tools that administrators can use for school
management.
This month, we've updated our list of top open source applications.
And for reference sake, we've included comparable closed-source software
that these applications can replace. However, note that when we say
"replace," we don't mean that these open source applications have
exactly the same features as the closed-source versions — only that the
programs serve a similar function.
As always, if you know of additional open source applications that
you think should be on our list, please let us know in the comments
section below.

Astronomy

2. Celestia
Replaces Starry Night, Seeker
A fabulous tool for home or the classroom, Celestia contains an
interactive map of the known universe built with actual imagery of
objects in space whenever possible. Not only can you view the sky from
any point on earth, this app lets you "fly" throughout the galaxy and
see how the stars would look from Mars, Jupiter, Pluto or beyond.
Operating System: Windows, Linux, OS X.
3. Stellarium
Replaces The SkyX, Perseus, SkyMap Lite
While Stellarium doesn't include Celestia's "flying" capabilities, it
does present an incredibly accurate and detailed picture of the night
sky from any point on earth at any time. In fact, it's so well-done that
many planetariums use it to power their shows. Operating System:
Windows, Linux, OS X.
4. KStars
Replaces The SkyX, Perseus, SkyMap Lite
Similar to Stellarium, KStars lets users view "up to 100 million
stars, 13,000 deep-sky objects, all 8 planets, the sun and moon, and
thousands of comets and asteroids." It also includes a number of tools
helpful for amateur astronomers, such as an observation list, an FOV
editor, a sky calendar, supernova alerts and a glossary of technical
terms. (Note that in order to use KStars on Windows, you'll have to
download KDE for Windows.) Operating System: Windows, Linux.
5. PP3
Replaces Starry Night, SkyMap Software
Science teachers often need to reproduce star charts for use during
class lectures, as well as for handouts and tests. This app works with LaTeX,
to simplify the process of creating star charts for use in presentation
slides or for printing. Operating System: Windows, Linux.
6. StarChart
Replaces Starry Night, SkyMap Software
StarChart describes itself simply as a program that "draws maps of
things in the sky." Like PP3, it does a good job of producing
high-quality star charts for study purposes. Operating System: Linux.

Art

7. Tux Paint
Replaces Crayola Art Studio
This basic drawing program features a kid-friendly interface,
numerous drawing tools, special effects and Tux the Linux penguin.
Although the website says it's for children ages 3 to 12, it's probably
best for those 7 and under. Operating System: Windows, Linux, OS X.
8. GPaint
Replaces Microsoft Paint
Students who are beyond Tux Paint but not quite ready for
professional drawing and photo manipulation tools might enjoy this Gnu
app. It offers an easy-to-use interface and a wide variety of drawing
tools. Operating System: Linux.
9. Inkscape
Replaces Illustrator, CorelDraw
This professional-caliber vector drawing program is suitable for
advanced high school and college students or adults who want to
experiment with digital art. The site also features a library of open
source clip art to help you get started with your own creations.
Operating System: Windows, Linux, OS X
10. Alchemy
Replaces Microsoft Paint
Designed to help artists get their creative juices flowing, Alchemy
has a deliberately limited feature set--no undo, no selecting, no
editing. Art teachers can use it as a starting point to help students
"sketch" new ideas before going on to create finished pieces of art
using real-world media or more advanced applications. Operating System:
Windows, Linux, OS X

Chemistry

11. Kalzium
Replaces Periodic Table of Chemical Elements
Need help with introductory chemistry? This KDE app allows students
to explore the periodic table, and it comes complete with a molecular
weight calculator, an isotope table, a 3D molecule editor and an
equation solver for stoichiometric problems. (Note that in order to use
Kalzium on Windows, you'll have to download KDE for Windows.) Operating System: Windows, Linux.
12. Avogadro
Replaces ChemDraw
For more advanced students and professional chemists, Avogrado offers
an intuitive interface for creating visualizations of molecules. The
website also includes some tips for educators on integrating Avogadro
into the classroom. Operating System: Windows, Linux.

Educational Games

13. GCompris
Replaces Various JumpStart Packages, Various Reader Rabbit Packages
Children 10 and under will enjoy this collection of more than 100
educational games. In addition to activities designed to teach lessons
about reading, history, math, science and other subjects, it also
includes "fun stuff" like chess, memory, sudoku and similar activities.
Operating System: Windows, Linux.
14. ChildsPlay
Replaces Various JumpStart Packages, Various Reader Rabbit Packages
ChildsPlay is also a collection of apps, although it doesn't include
nearly as many activities as GCompris. It offers 11 games, including
memory games, alphabet and number recognition activities and puzzles.
Operating System: Windows, Linux, OS X.

Foreign Language

15. ZWDisplay
Replaces Declan's ReadWrite Chinese
A combination dictionary/flashcard program, ZWDisplay helps Mandarin
students learn to read and pronounce Chinese characters. It uses a
unique coloring scheme and a wide variety of options to simplify the
language learning process. Operating System: Linux.
16. Step Into Chinese
Replaces Declan's ReadWrite Chinese
Step Into Chinese describes itself as a "language mining tool" that
helps native English speaker understand Mandarin in context. It includes
8,300 Chinese characters that correspond to 26,000 modern Chinese words
and concepts. Operating System: Windows, Linux, OS X.
17. Zkanji
Replaces JISHOP
This app for students learning Japanese includes a dictionary with
more than 190,000 words, as well as a flashcard program. It also
includes helpful information about how to write kanji characters.
Operating System: Windows.

Flashcards

18. FlashQard
Replaces Flash Card Manager, studyPerfect, WinFlash
Using the Leitner method of presenting flashcards, this app optimizes
your study time for maximum efficiency by focusing on the information
you don't yet know. Create your own cards for any subject or use one of
the pre-made card sets for learning languages or geography facts.
Operating System: Windows, Linux.
19. jVLT
Replaces Flash Card Manager, studyPerfect, WinFlash
Very similar to FlashQard, Java-based jVLT also uses the Leitner
method to help students master concepts. However, this app focuses
primarily on vocabulary, making it ideal for students who are studying a
foreign language. Operating System: Windows, Linux, OS X.

20. The Mnemosyne Project
Replaces Flash Card Manager, studyPerfect, WinFlash
One of the better flashcard apps available, Mnemosyne was born from a
research project that is studying the nature of long-term memory. It
supports multiple card formats and uses an algorithm to determine which
card should appear next. Users who choose to do so can upload their
progress anonymously in order to provide data for the research team.
Operating System: Windows, Linux, OS X, Android/
21. Pauker
Replaces Flash Card Manager, studyPerfect, WinFlash
Java-based Pauker lets you take your flashcards with you on almost
any device. Like many of the other apps in this category, it uses the
Leitner method of flashcard presentation to strengthen your
ultra-short-term, short-term, and long-term memory. Operating System: OS
Independent.
22. Parley
Replaces Flash Card Manager, studyPerfect, WinFlash
In addition to traditional flashcards, Parley also utilizes anagrams,
multiple choice, fill-in-the-blank, synonym/antonym and other types of
tests to improve your vocabulary. Many user-created card sets are
available at KDE for Windows. Operating System: Windows, Linux.
23. Anki
Replaces Flash Card Manager, studyPerfect, WinFlash
This app boasts that it can help students learn just about any type
of information, whether they are learning a new language, studying for
medical exams or trying to remember guitar chords. It comes in iOS and
Android versions, making it ideal for learning on the go. Operating
System: Windows, Linux, OS X, iOS, Android.

Geography

24. WorldWind
Replaces Google Earth
Very similar to Google Earth, this NASA-developed tool lets you view
satellite imagery and maps for any location on earth. It also includes
an API for developers who want to include WorldWind functionality within
their own apps. Operating System: OS Independent.
25. Marble
Replaces Google Earth
This "virtual globe" goes far beyond basic mapping, integrating
weather, photos, traffic and Wikipedia information. With the mobile
versions (currently only for select Nokia phones, an Android version is
in the works), it also becomes a turn-by-turn navigation system.
Operating System: Windows, Linux, OS X.

Library Automation

26. VuFind
Replaces Alexandria, ResourceMate, Surpass, Mandarin Library Automation
This library management system boasts Web 2.0 capabilities and design
done "by libraries for libraries." The modular architecture means it's
easy to install just the features you need--or to add or modify modules
as necessary. Operating System: OS Independent.
27. LibLime Koha
Replaces Alexandria, ResourceMate, Surpass, Mandarin Library Automation
Used by thousands of libraries around the world, Koha describes
itself as "the most advanced and cost effective open source automation
solution on the market." Paid hosting, support, consulting,
implementation and development services are also available on the site.
Operating System: OS Independent.
28. OpenBiblio
Replaces Alexandria, ResourceMate, Surpass, Mandarin Library Automation
OpenBiblio features include an online public access catalog (OPAC),
circulation, cataloging, and staff administration. See the site for
extensive documentation and templates. Operating System: OS Independent.
29. Evergreen
Replaces Alexandria, ResourceMate, Surpass, Mandarin Library Automation
This library automation solution prides itself on its scalability.
The Evergreen community is unusually active, with regular newsletters
and updates, and you can find demos of the software and an interactive
of libraries using Evergreen at the website. Operating System: OS
Independent.
30. OPALS
Replaces Alexandria, ResourceMate, Surpass, Mandarin Library Automation
Short for "OPen-source Automated Library System," OPALS is one of the
most highly rated library automation solutions available. It provides
Web-based access to information databases, library collections and
digital archives. Paid hosting and other services are available.
Operating System: OS Independent.
31. NewGenLib
Replaces Alexandria, ResourceMate, Surpass, Mandarin Library Automation
NewGenLib proudly proclaims itself the "best open source library
system." Notable features include support for Android smartphones and
tablets, standards compliance, an intuitive interface and much more.
Operating System: OS Independent.

Logic/Debate

32. Argumentative
Replaces Rationale
Ideal for students taking logic, debate, law, writing, speech and
similar classes, Argumentative makes it easy to create a visual
representation of an argument. It tracks premises, reasons, objections,
helpers, reasoning, analysis and more to help users create well-reasoned
papers and speeches. Operating System: Windows.
33. Argunet
Replaces Rationale
Argunet offers both an argument editor (similar to Argumentative) and
a library of arguments that students can browse and analyze. You can
use it on a standalone system or in a client-server setup to encourage
collaboration among students or debate teams. Operating System: Windows,
Linux, OS X.
34. iLogos
Replaces Rationale
Java-based iLogos isn't quite as fancy as some of the other argument
mappers, but it does make it fairly easy to diagram arguments with a
combination of words and pictures. The site also includes a helpful list
of similar tools. Operating System: Windows, Linux, OS X.

Math

35. Sage
Replaces Visual Math, Mathematica
Sage brings together more than 100 other open source math projects
under a single unifying interface. It can handle everything from basic
algebra to calculus to cryptography to group theory and everything in
between. However, the interface is very basic and will feel most
comfortable to users with some background in programming. Operating
System: Windows, Linux, OS X.
36. Genius
Replaces Visual Math, Mathematica
This app functions both as a calculator and as a research/educational
tool. To use it, you'll have to enter a mathematical expression using
the Genius Extension Language (GEL), but the language is designed to
look as much like regular math syntax as possible. Operating System:
Linux, OS X.
37. Scilab
Replaces Visual Math, Mathematica
Designed for engineers and scientists, Scilab can handle maths and
simulation, 2D and 3D visualizations, optimization, statistics, control
system design and analysis, signal processing and more. Paid support,
training and services are available through Scilab Enterprises. Operating System: Windows, Linux, OS X.
38. Maxima
Replaces Visual Math, Mathematica
A descendant of MIT's Macsyma, Maxima describes itself as "a system
for the manipulation of symbolic and numerical expressions, including
differentiation, integration, Taylor series, Laplace transforms,
ordinary differential equations, systems of linear equations,
polynomials, and sets, lists, vectors, matrices, and tensors." It can
also create both 2D and 3D graphs. Operating System: Windows, Linux, OS
X.
39. GeoGebra
Replaces Visual Math, Mathematica
With its wide range of capabilities, GeoGebra is suitable for
elementary students learning arithmetic, high school students learning
algebra and geometry, and even for college students studying calculus
and advanced mathematics. Specifically designed as an educational tool,
it is much more user-friendly than many of the other math apps on our
list, and the website also includes helpful resources for educators.
Operating System: OS Independent.
40. GraphCalc
Replaces Visual Math
GraphCalc isn't so much as replacement for other software as a
replacement for handheld graphing calculators. The site boasts,
"GraphCalc can be your first, last, and only line of offense against the
mathematics that threaten to push you over the brink of insanity. It
slices, dices, shreds and purees functions that leave other calculators
wondering what hit them." The project hasn't been updated in a while,
but it remains a helpful alternative to graphing calculators. Operating
System: Windows, Linux.

41. gnuplot
Replaces Visual Math, Mathematica
Unlike many of the other math apps on our list, Gnuplot does just one
thing--create graphs. It works from the command line and supports both
2D and 3D plots. Operating System: Windows, Linux, Unix, OS X, and
others.
42. Dr. Geo
Replaces Cabri
This award-winning app invites people age 7 to 77 to "be a geometer."
It allows students to create and interact with sketches of geometric
objects so that they can better understand geometry lessons. Operating
System: Windows, Linux, OS X.
43. Kig
Replaces Cabri
Similar to Dr. Geo, KDE's geometry app also provides an interface for
working with and learning more about geometric figures. (Note that in
order to use Kig on Windows, you'll have to download KDE for Windows.) Operating System: Windows, Linux.

Physics

47. Step
Replaces Yenka Science
Step is an interactive physics simulator that allows students to
explore classical mechanics, particles, springs, gravity, coulomb
forces, collisions, sound waves and much more. It also includes an
equation solver and a unit conversion tool. (Note that in order to use
Step on Windows, you'll have to download KDE for Windows.) Operating System: Windows, Linux.

Online Course Management Systems/E-Learning

48. Moodle
Replaces Pearson Learning Studio, Articulate Storyline, TrainCaster LMS
Incredibly popular, Moodle is a highly scalable course management
system that currently powers nearly 7 million online courses for more
than 65 million users. The website offers helpful demonstrations so that
you can see this CMS in action from both an instructor's and a
student's point of view. Operating System: Windows, Linux, OS X.
49. Sakai
Replaces Pearson Learning Studio, Articulate Storyline, TrainCaster LMS
Sakai offers two separate open source products: Sakai CLE, which is
similar to other online learning systems, and Sakai OAE which is a
collaboration and social sharing platform for academics, researchers and
students. Several commercial affiliates offer hosting and related
services. Operating System: OS Independent.
50. Claroline
Replaces Pearson Learning Studio, Articulate Storyline, TrainCaster LMS
Available in many different languages, Claroline is used by
organizations in more than 100 different countries. It boasts a highly
intuitive interface, and it's flexible enough to meet the needs of
businesses and associations, as well as colleges and universities.
Operating System: Windows, Linux, OS X.
51. eFront
Replaces Pearson Learning Studio, Articulate Storyline, TrainCaster LMS
Unlike most of the other learning management systems on our list,
eFront places particular emphasis on meeting the training needs of
enterprises. It's available in paid hosted and enterprise versions, as
well as the open source edition. Operating System: Windows, Linux.
52. ILIAS
Replaces Pearson Learning Studio, Articulate Storyline, TrainCaster LMS
ILIAS claims to be the first open source learning management system
in the world to achieve SCORM 2004 (3rd Edition) compliance. It also
enjoys a very active community with conferences and user events
regularly scheduled in many different countries. Operating System:
Windows, Linux.
53. ATutor
Replaces Pearson Learning Studio, Articulate Storyline, TrainCaster LMS
ATutor was designed to comply with industry standards, such as SCORM
Content Packaging, SCORM 1.2 LMS RTE3, OAuth Authentication Protocol and
others, and accessibility standards, such as W3C WCAG 2.0, W3C ATAG
2.0, US Section 508 and many others. Try the online demo to see it in
action. Operating System: OS Independent.

Operating System

54. Edubuntu
Replaces Microsoft Windows
This Ubuntu-based Linux distribution aims to collect all the best in
open source educational software and make it available along with an
easy-to-use, free operating system. It's developed by a community
committed to the idea that "learning and knowledge should be available
to everyone who wants to improve themselves and the world around them."
55. Qimo for Kids
Replaces Microsoft Windows
As you might guess from the name, this Ubuntu-based operating system
was designed to be used by kids, specifically by those as young as three
years of age. It features an extensive library of educational games,
and its large icons make it easy for young children to find what they're
looking for.
56. Debian Edu/Skolelinux
Replaces Microsoft Windows
This Debian variation incorporates many educational games and apps,
and it's available in multiple languages. The name "Skolelinux" is
Norwegian for "school Linux."
57. Fedora Education Spin
Replaces Microsoft Windows
Fedora, the community version of Red Hat's Linux distribution, comes
in a special education "spin." It includes the Sugar desktop
environment, the KDE education apps and a version of the Eclipse
integrated development environment so that students can learn to write
their own code.

Religion

58. BibleTime
Replaces Logos
Top-of-the-line Bible study tools like Logos can cost hundreds or
even thousands of dollars, but BibleTime offers many of the same
capabilities for free. It provides searchable access to more than 200
Bibles, commentaries and related resources. Operating System: Windows,
Linux, OS X.
59. Xiphos
Replaces Logos
Xiphos provides access to the same Biblical texts and resources as
BibleTime—it just has a different interface. It also provides the
capabilities for users to create their own modules such as journals,
prayer lists and personal commentaries that they can share with friends.
Operating System: Windows, Linux.
60. Zekr
Replaces Al-Qari Plus
This app provides access to the Quran and commentaries in a number of
different translations. It's completely free and open source because
its developers follow a very simple rule: "Never Profit Off The
Prophet." Operating System: Windows, Linux, OS X.

Research/Bibliography

61. WIKINDX
Replaces EndNote, Biblioscape
WIKINDX combines tools for tracking bibliographic information with
tools for collaboratively authoring papers. It can be deployed on a
single system or on a Web server, and it knows how to format entries for
the most popular style guides, including the Chicago Manual, APA and others. Operating System: OS Independent.
62. Zotero
Replaces EndNote, Biblioscape
Zotero stores research information where you found it—in your
browser. It's a Firefox plug-in that automatically collects and
organizes your research so that you can easily search it later.
Operating System: OS Independent.

Student Information System

63. openSIS
Replaces PowerSchool, Schoolmaster, RenWeb
Downloaded more than 50,000 times and used by more than 12,000
schools, openSIS bills itself as "the only open source Global SIS in the
world." In addition to the free community edition it comes in
commercially supported school and district versions. Operating System:
Windows, Linux.
64. ClaSS
Replaces PowerSchool, Schoolmaster, RenWeb
Used by some of the largest international schools in Europe, ClaSS
describes itself as "an information management system which puts
students first." It includes a student database, messaging, reporting
and school management features.
65. OpenAdmin
Replaces PowerSchool, Schoolmaster, RenWeb
Downloaded nearly 25,000 times, this SIS has been deployed by schools
all around the world—in fact, you can see a count of how many schools
in each country use it right on the project home page. It includes
modules for demographics, attendance, discipline, report cards, fees,
online gradebook, transcripts, special needs and more. Operating System:
OS Independent.
66. SchoolTool
Replaces PowerSchool, Schoolmaster, RenWeb
Designed for schools in the developing world, SchoolTool runs on
Ubuntu and has been translated into numerous languages. The latest
update can track skills, outcomes and standards, which is helpful for
complying with regulations in many areas. Operating System: Linux.
67. Open-School
Replaces PowerSchool, Schoolmaster, RenWeb
This very polished SIS describes boasts user-friendliness, security
and scalability. A free community version is available, but the website
primarily promotes the supported commercial version. Operating System:
Windows, Linux.
68. Akura
Replaces PowerSchool, Schoolmaster, RenWeb
Akura was designed for a "college" (which equates to a K-12 school in
the U.S.) in Sri Lanka and serves many Asian institutions. It's won
many awards, and it's particularly popular with religious schools.
Operating System: Windows, Linux.

Testing

69. TCExam
Replaces Exam Pro, QuestionMark
This computer-based assessment software aims to simplify the
creation, delivery and scoring of tests. It's available under an open
source license for non-commercial use or with a paid commercial license.
Operating System: OS Independent.
70. iTest
Replaces Exam Pro, QuestionMark
This client-server based software makes it easy to give each student a
different version of a test because it can pull questions from a
pre-established database. It's available in nine different languages.
Operating System: Windows, Linux, OS X.
71. Safe Exam Browser
Replaces Exam Pro, QuestionMark
If you're giving a browser-based test (for example, through a
learning management system), this tool locks down the student's system
so they can't browse the Internet or access other applications while
taking the exam. It can be used with student-owned PCs, as well as with
school-owned, networked systems. Operating System: Windows.

Typing

72. TuxType
Replaces Maevis Beacon Teaches Typing, Typing Instructor
For younger kids who are just learning their way around the keyboard,
this app uses fun games to help students learn where the various
letters are. Type quickly to help Tux the Linux penguin eat all the fish
or save the earth from falling asteroids. Operating System: Windows,
Linux, OS X.
73. TypeFaster Typing Tutor
Replaces Maevis Beacon Teaches Typing, Typing Instructor
Unlike TuxType, TypeFaster is a full-featured touch-typing learning
system. It includes a 3D typing game, and it has a teacher mode that
allows instructors to track students' progress. Operating System:
Windows, Linux.
74. Klavaro
Replaces Maevis Beacon Teaches Typing, Typing Instructor
Humble Klavaro describes itself as "just another free touch typing
tutor program," but it's actually quite unique for its keyboard- and
language-independent operation. In addition to the familiar "QWERTY"
keyboard layout, it supports six other built-in layouts, or you can use
the keyboard editor to create your own. It's also available for 25
different languages, making this a good choice for users outside the
U.S. Operating System: Windows, Linux.
75. WriteType
Replaces Microsoft Word
This option doesn't teach students to type—it makes typing easier,
particularly for younger users. It includes features like word
completion, read aloud mode, grammar checking and auto-correction that
make it easier for young people to create documents on their own.
Operating System: Windows, Linux.

1. Introduction

The aim of this article is to provide an overview of the GNU R
programming language. It starts a series of articles devoted to
programming with R. Its objective is to present, in an organized and
concise manner, the elementary components of the R programming language.
It is designed to help you understand R code and write your own. It is
assumed that the reader has already some basic programming knowledge of
R. If you are not familiar with any of R features it is recommended that
you first read A quick GNU R tutorial to basic operations, functions and data structures.

2. Expressions

An R expression is an elementary component of R code. Expression in R can be:

assignment statement;

conditional statement;

arithmetic expression.

Examples of R expressions:

> y> if (1==1) 1 else 0
[1] 1
> 100/5
[1] 20-100>

R expression are constructed from objects or functions. It is common
to separate them with a new line, however, you can also separate
expressions with semicolons as below.

4. Symbols

A symbol in R is a variable name. So if you assign an object to a variable name you actually assigning an object to a symbol. An environment in R, then, is a set of such symbols that are created for a given purpose.Example of symbols in R:

> x> y
Above, x and y are symbols.

5. Functions

A function in R is an object that takes as arguments
other objects and returns an object as a result. Did you know that the
assignment operator '

6. Other properties of R language

6.1. Objects are immutable in R

In R objects are immutable. This means that R will copy the object
not just reference to the object. Consider the following example. We
define a function that sets the ith element of vector 'x' to 4 as below

> f
Let's see what happens if we define a vector 'w' and pass it as argument to the function 'f'.

> w> f(w,1)
> w
[1] 1 3 6 7-c>

What we just observed above is that the vector 'w' was copied when it
was passed to the function so that the function does not modify this
vector.

6.2. Everything in R is an object

Everything
in R is an object. Objects are not only defined to store data as in the
case of vectors, lists or other data structures. Other examples of
objects in R are functions, symbols or R expressions. For instance,
function names in R are symbol objects that point to function objects as
indicated below

> functionname> functionname
function(x,y) x+y-function>

6.3. Special values in R

There is a number of special values used in R. These are:

NA, used to represent missing values, means "not available";

Inf and -Inf, resulting in a calculation when the output number is too big or too small or when dividing by zero;

NaN, resulting in a calculation that is not possible to compute such as division of zero by zero, means "not a number";

NULL, used often as an argument in functions, means that no value was assigned to that argument.

6.4. Coercion

R often coerces values from one type to another. For instance, when
you call a function with an argument of a wrong type, R will try to
convert this argument to a different type so the function can work.
Another example might be when we define a vector with numeric values, R
will assign it a type "integer" as below

> x> typeof(x)
[1] "integer"-c>

Now, when we change the forth element of vector 'x' to four, R will
automatically change the type of the vector to 'double' as indicated
below

> x[4]> typeof(x)
[1] "double"-4>

6.5. The R interpreter

An interpreter is a program that executes the
written code. There is no need to compile R code into an object language
as in the case of C, C++ or Java. This means that R is an interpreted
language.
R interpreter evaluates R expressions in few steps. First, it parses
an expression changing it into an appropriate functional form. Let's
call the quote() function to see how this happens.

> typeof(quote(if(1>3) "one is greater than three" else "one is less than three"))
[1] "language"

The R expression above returned a "language" object. To see how R evaluates an expression we produce a parse tree.

> as(quote(if(1>3) "one is greater than three" else "one is less than three"),"list")
[[1]]
`if`
[[2]]
1 > 3
[[3]]
[1] "one is greater than three"
[[4]]
[1] "one is less than three"

Let's also apply the typeof() function to the elements in such list, which shows how the expression is interpreted by R.

As you can see some parts of the if statement where not included in the parsed expression. That is, the else element. Additionally, it is interesting to note that the first item in the list is a symbol, which points to the if() function. Even though the syntax for the if
statement differs from the function call, the R interpreter translates
the expression into the function call with the name of the function as
its first argument and other arguments as in the list above.

7. Conclusion

This article is an introduction to the R programming language. In the
forthcoming articles we will focus in detail on the defined here
elements of the R language.
Make sure you tune in to our RSS and Linux jobs portal to stay informed about the latest opportunities in the field. Also visit our Linux Forum if you want to share your Linux experiences with us or require additional help.
-function>
->
->-1>
->->-3>

Whether it is on your desktop or server installation of Ubuntu, there
will come a time that you may need to work with Apache and
certificates. We will go into full certificates from Certificate
Authorities (like Verisign or Entrust) as well as exploring some of the
‘Open Source’ Certificate Authorities (read: free) in a later article.
Today we are discussing how to prepare Apache to answer HTTPS requests
in the VHOSTS as well as installing and configuring the pieces. Finally,
we will install a self signed certificate and access our system over
HTTPS to verify it all works.Assembling All the Pieces
The first thing we need to do (assuming that Apache 2 is already
installed) is to make sure we have all the pieces. Let’s install the SSL
package and module we need:sudo apt-get install openssl
This will install the parts needed to generate our certificate and the module that apache needs to enable SSL support (NOTE: Apache 2 must be installed prior to running this command in order for the module to be installed in the proper location). Once that is installed, apache needs to be told to enable the module:sudo a2enmod ssl
If this is the first time you have used apache’s shortcut scripts,
you may not be familiar with the most common one’s – you will want to be
familiar with the following scripts when using apache in any Debian
based distribution (including Ubuntu):

a2enmod: Shortcut for ‘Available To Enabled Module’, takes
an installed module and creates a link from
‘/etc/apache2/mods-available’ to ‘/etc/apache2/mods-enabled’ so that
when apache is restarted, the module will be enabled in the live
configuration

a2dismod: Shortcut for ‘Available to Disabled Module’,
removes the link from ‘/etc/apache2/mods-enabled’ created when the
module was enabled (see above), the module will then be disabled after
apache is restarted

a2ensite: Shortcut for ‘Available to Enabled Site’, takes
an installed site (vhost) and creates a link from
‘/etc/apache2/sites-available’ to ‘/etc/apache2/sites-enabled’ so that
when apache is restarted, the site will be enabled in the live
configuration

a2dissite: Shortcut for ‘Available to Disabled Site’,
removes the link from ‘/etc/apache2/sites-enabled’ created when the site
was enabled (see above), the site will then be disabled after apache is
restarted

Generating the Certificate
Our module is installed and active (at least the next time we restart
apache), so now we need to generate a certificate and then note their
locations (we will place our resulting certificate files and keys in the
most ‘standard’ locations, but they can be changed to whatever is
appropriate in your circumstance as long as you note where they are
during our vhost setup later). First, we need to generate the
‘Certificate Request’ file:

Creating Our Certificate Request

As you can see, you will be asked for a number of pieces of
information. During the generation and installation of a ‘Self Signed’
certificate, like we are installing, this information is unimportant.
However, in our next article on the topic, it will be important since
some of these items will generate the key embedded in your certificate
and that is used by the Certificate Authority who issues the final
certificate to validate your identity and secure your site. At this
point, as you can see I did, you can put in almost anything you want.
A couple more things to complete the creation of our certificate and move the files into place. See the following screen shot:

Creating Our Certificate and Moving Files

Informing Our Web Server
A couple more steps and we are ready to test. We need to grab a vhost
for apache to use for SSL requests and then we need to make some
changes to it in order to use our key and certificate file. The easiest
way to do this is to copy the following:sudo cp /etc/apache2/sites-available/default-ssl /etc/apache2/sites-available/my-ssl
Then edit the ‘my-ssl’ file and be sure to add the following (comment
out or remove existing entries if they are different than this):

NOTE: change the path for the certificate and/or key if you saved or named them something else in our earlier steps.
Finally, we need to restart apache so that the new virtual host is picked up:sudo service apache2 restart
You can then go to your browser and enter ‘https://localhost’ to see
your new certificate in action. You WILL get a browser security warning
since this is a SELF SIGNED certificate and not issued by a Certificate
Authority. This is normal and in our next article, we will cover
installing a CA signed certificate which will fix that.Final Thoughts
This will give your system a bit of extra security. For testing or local
use, a self signed certificate is enough protection to secure basic
authentication, database traffic or other web traffic from anyone
sniffing your connection. Total time to implement this is about 20
minutes and is worth the effort.
Leave me a comment with any questions or problems and I will help out if I can.

Commonly known as .htaccess, AllowOverride is a neat little feature
that allows you to tweak the server’s behavior without modifying the
configuration file or restarting the server. Personally, I think this
is great for development purposes. It allows you to quickly test
various server configurations without needing to mess with restarting
the server. It helps you be more (buzzword alert!) agile.
Beyond the obvious security problems of allowing configuration
modifications in a public document root there is also a performance
impact. What happens with AllowOverride is that Apache will do an
open() call on each parent directory from the requested file onward.
To demonstrate this I used a program called strace which checks for
system calls and gives you a list of each system call that is made.
First we’ll take a look at the strace with AllowOverride set to None.

You can clearly see the additional open() calls being made to try and
discover the .htaccess file. In this case the calls are completely
superfluous because we have nothing there. But even so we have a
significant impact on static file throughput.
AllowOverride None

The requests where AllowOverride was turned off were executed at 60% of the time of the ones where AllowOverride was turned on.
And remember, this is just the impact of file operations and does not
take into account the time to reconfigure Apache during the course of
these requests.
So the data would clearly show that there is a negative impact to
having AllowOverride turned on in a production environment. Instead it
will generally be better to take those changes in .htaccess and place
them in your httpd configuration file.
[UPDATE]
In fact Mike Willbanks says you should never do it. I agree with
him, but I wouldn’t make as big a stink in dev as I would in prod.

Sunday, February 24, 2013

Summary

Shell scripts are often running as background processes, doing useful things without running in a visible shell.
Think, for example, of cron jobs or scripts that are fired from a program on a web server.
To write such scripts can be quite painful, as all errors occur out of sight as well.
Off course you can make use of a log file, but the ideal level of logging is hard to find.
You often log way too much when the script is running fine and way too little when it unexpectedly fails.
While log files can hold a lot of information, finding the relevant information is a bit trickier.
My solution is to log only the errors with all the details to a small database.
This database contains tables for the message, the corresponding stack trace and the important environment variables.
I have chosen for an SQLite database in this howto, but the same principle works with other databases as well.

The database

SQLite needs some settings to work as I expect it to, and these settings can be put in an initializing script.
These settings include the error behaviour of SQLite itself and its foreign key handling:

.bail ON
.echo OFF
PRAGMA foreign_keys = TRUE;

Off course, we also need a database and I do not want to rely on one
to exist.
Therefore, the first thing the bash script will do is to run an SQL
"revive" script on the database file: if the database did not exist, it
will be created and if it did, it will do nothing:

The mechanism

Now we have the database defined, we can focus on the error handler itself.
Bash has a trap command that, among other
things, can be used to trap non-zero exit codes of commands.
These non-zero exit codes usually denote an error.
Within a "trapped" section of a script, you can still react to a
non-zero exit code without the error handler to take over by using the
"or construct":

false || echo "non-zero exit"

The above line will not invoke the error handler, but any of the following lines will (the command false will always exit with a non-zero exit code, the command true will exit with an exit code of zero):

false
false;true

If we define a function that will log all error data, we can pass that function to the trap command.
From that function, we can access the call stack by invoking the caller command.
The caller command will return an array containing the line number, the subroutine name and the file name.

I
will read the environment from a predefined list of variables that I am
interested in, because my (Xubuntu) system has so many pre-defined
variables that logging them all would drown me in useless information.
I am also interested in the messages that are written to STDERR, so I
will define a file location that I can use to send the STDERR stream to.
After logging the error, I will still write the contents of that file to
STDERR, so the user is not left in the dark with scripts that do run interactively.
Now I have all the information I want to log.

Caveat

The commands logged are the commands at the level of the trap command.
This means that if you put a trap statement in your main script and call
a function, that function will complete, and only after its completion,
the error handler may be invoked if the function ended in an error
state.
If you want error handling within the function as well, put the trap command also inside it.

Invoking the error handler

To make my error handling script available as a kind of library, I will call it from the scripts I want to monitor, with the source command.
This will effectively include it in the scripts I want to monitor.
The calling scripts need to define a few settings before including the error handling script:

ERROR_CLEANUP_ACTION

Optional. A command to be executed after the error handler has run, for example to clean up temporary files.

ERROR_ENVIRONMENT_VARIABLES

A list of variables to log. This list will be augmented with EXITCODE BASH_COMMAND BASH_LINENO BASH_ARGV

ERRORDB

The location of the database.
The database file does not need to exist, but the location does need to be writeable by the user running the script.

ERROROUTPUT

The location of a file to catch the contents of STDERR.
Optional. Defaults to "/var/tmp/$$.err".

SQLITE3_EXECUTABLE

The location of the SQLite3 command line client.
Optional. Defaults to `which sqlite3`.

Because escaping all possible error messages for an INSERT query can be quite a challenge, I define two auxiliary functions Error_Hexit and Error_Hexit_File.
This allows me to "escape" all exotic strings to hexadecimal strings.
SQLite interprets all hexadecimal strings as binary objects, so I cast them to TEXT in my INSERT statements.