Summary

There are many examples of the shunting algorithm that takes infix and converts to postfix (or RPN). None of them did everything I wanted. I wanted something full featured sort of like excel. I wanted:

Negative numbers need to be handled and they are often ignored in converters.

Support single and multi character operations (i.e. <<, >>).

I wanted support for multiple argument functions and even no argument functions.

I wanted solid error messages and an indication where the error was.

I wanted an optimization step where constants are evaluated and reduced prior to execution.

Motivation

The motivation was a project where we need a think PC client to process an infix equation and translate this down to a set of opcodes and data for a small processor (memory/horsepower).

The first step is getting the conversation to work in high level constructs.

The second step which will happen later was to convert the high level language constructs to a set of opcodes and data.

Background

There are numerous examples of equation processing but I never found any that did everything sort of like excel did.

This one is pretty good: http://interactivepython.org/runestone/static/pythonds/BasicDS/InfixPrefixandPostfixExpressions.html

Maybe someone else did a full implementation but I never found it.

Tools

PyCharm and Python 3.6.

Definitions

operator – This is a symbol such as +, -, *, /. It can be more than one character. The ( is also an operator technically.

function – This is a piece of operation in the form of FUNC(…) with a certain number of arguments defined for that function. Each argument is separated by a comma. If there are no arguments (for example, RND for random), then no parenthesis are allowed. Functions can have zero to “n” arguments.

variable – This is a textual name for an variable (or input) in the equation. This can be one or more characters.

constant – This is a number in the infix equation.

Steps

As we work through the steps, refer to this equation:

2 * (1 + -3) + abs(-round(rnd, 2))

In text, this is 2 times the sum of 1 and a -3 + the absolute value of a random number rounded to the 2nd decimal place and negated.

The first step is to convert the infix to a set of stack items. We are effectively splitting on operators. Then each stack item is evaluated for the type of information between the operators. After this step, the infix stack looks like this.

Notice there are two subtractions operators that really should be negation. So the second step is to determine if there is a valid operator in front of the subtraction and if a function, constant, or variable is immediately after it. After this check, here is the infix stack. (I used the ! as a negation symbol.)

The next 3 steps are to check the validity of the infix equation. First, we check to make sure that we do not have operators next to each other. Next, we check for mismatched parenthesis. Finally we check for the correct number of arguments in functions.

if anything failure, the python script will tell where the error occurred and what the error was. This can certainly be beautified into whatever final UI is chosen. The information is there.

If there are no errors, the stack is shunted into a postfix stack and the commas and parenthesis are removed.

Executing this yields a result between negative 4 and negative 3 such as -3.81 or -3.66.

Future

This code will likely be translated into C++ with the functions in C so that it can be shared on the embedded system. The functions used for optimization and execution must be the exact same and this is a case where we will want to share source.

The python script could be better organized into several modules but this is a sample for the final run which will be in likely C/C++.

Executive summary

Here is how to interact between Raspberry PI 3 and Lego EV3 using Bluetooth. There will be a python 3.6 script presented and sample EV3 programs. All of this is stock stuff on both platforms.

The key is how to interpret the bytes. The script presented takes care of all of that. Here is the EV3BT Script.

The fun stuff

So, I could possibly have a family and I could indeed possible like to enjoy time with them. I play with Lego in the interest of my family. I am a good dad… unless explosives are involved.

Don’t judge.

So the project was to determine if we could use a Raspberry PI with open CV and a camera and drive a Lego robot around the floor using the EV3 trying to do cool things.

There are several communication options but I didn’t want to use any sort of wired or wireless Ethernet nor is USB an option from what I can tell. I also didn’t want to change the EV3’s operating system.

I decided to play with bluetooth. They are probably other and perhaps better ways to do this but this is what I chose.

Enough disclaimers about the effectiveness of my decision making skills especially about the explosives and related minors.

If you are unfamiliar with pairing with Raspberry PI3 and the EV3, google it. It is a bit cryptic but once pair, you can connect to the EV3 and it appears as a serial device (i.e. /dev/rfcomm0).

The trick and the purpose of the

Sending to the EV3

The program above simply waits for a Bluetooth message and in this case, a text message, prints it on the LCD screen and beeps. The python below shows the sending of a message to the EV3. The trick of the EV3BT class is to simply encode and decode the bits. The serial module takes care of the transmission.

You have options for what sort of messages you can send. These are Text, Numeric, and Logic. In programmer speak, strings, floating point values (IEEE 754), and Boolean (literally a 1 and a 0). See below.

The other key is the mailbox. The ‘abc’ is the mail box and is shown on the block (highlighted in red). You must pass the mailbox in a form that matches the block so that the block receives the message. In case it isn’t obvious, you can send messages to multiple mailboxes in your script and control several EV3 loops just but using different mailboxes.

Reading from the EV3

Reading is just a little more complicated in that you probably need to have some sort of loop with blocking logic. As I say that, every experienced developer reading this is rolling their eyes (back in old school Linux days, should we use select or poll?).

In this example, the EV3 will wait for a touch sensor to be activated and send a message. To be precise, the message ‘Raspberry PI’ will be sent to the mailbox ‘abc’ in the Raspberry PI.

The key is the decodeMessage. You tell the method, the string to parse, and the message type expected (as this effects the length of the transfer and there is no identifying mark in the message about the type). It will return you the mail box, the value, and any remnant bytes left over. The buffering is not stellar in this example but it gets the point across.

Moving a robot

So to put it together, I could make a robot move under python control in the Raspberry PI with this script and program.

The EV3 was made as a tank with two treads. A left and right. Essentially, the script is sending a numeric to each tank tread. A positive number moves forward, a negative number moves back. The larger the number, the faster the rotation.

Areas of improvement

When reading, there is no identifying mark in the package concerning the type. You also can not assume that number of bytes on the wire will work. A 3 byte with null looks like a float.

Deriving application specific code to handle the case where you want the EV3 to send a float and a Boolean should be simple to add. A string? Not sure the worth of that really from the EV3 to the Raspberry PI.

This gives a large enough scrollbar surface to reliably use a finger on. It isn’t the exact same finger sliding on a mobile device but it works well enough. An since it is merely a scrollbar, your pages will work the same between a PC browser and the 7″ display.

We want to scroll by sliding our finger.

We found that larger sized scrollbars are usable as mentioned above. It isn’t exactly the same as drag scroll, it is intuitive.

Real Estate.

Again with CSS, you can do many things

I found font-size: 1.Xem !important; to be very useful. Apply something similar to all your tags. For example:

You will have to wrap your checkbox in a parent entity that is a bit larger and “on click” on the parent, update the checkbox (by making the state the “opposite”. Keep in mind that a click on the checkbox occurs, you will want to stop propagation so that the parent doesn’t receive the same click and effectively change the state back.

What about Keyboard?

Thanks to the work by Jeremy Satterfield and Rob Garrison, the virtual keyboard worked very well. Since it is a jQuery plugin, it is very easy to use. For example.

The python priority queue describes messages in absolute terms. The most important element will always be popped. The least important element may never be popped.

Sometimes when a message based data path is serialized to something (like a serial port based piece of hardware), all messages have some importance. Unlike the priority queue, we would rather not use such absolute terms. We would rather say that most of the time, the important messages are sent. However, we guarantee that some of the time, the less important messages are sent.

The language is more gray and the queue needs to be a bit more… “cooperative”. (We could say “round robin”.)

Let’s implement our own. But first, a quick back story. I have some knowledge of Motorola’s (at the time) Time Processor Unit (TPU) on some of their embedded processors. The TPU had an event driven model where certain sections of microcode would execute based on priority but lower priority microcode would still operated periodically. Let’s base our design on this philosophy. (See section 3 for more information on the TPU scheduler.)

Python has a pretty good queue framework that you derive your queue from and implement several methods. If you ever look at the priority queue code in python, it is pretty simplistic as it relies on the heapq. There is much work done in the Queue class including thread synchronization.

The _qsize and _put methods are basic. The _init method requires a priority order. This specifies the priority guaranteed to pop from the queue for the corresponding _get call. The _orderIdx is used to know which step of the priority order the algorithm is on. It increments or resets on every _get call.

The _get call is where the magic happens. The first step is to increment or reset the _orderIdx value. Then we are going to get our filterPriority from the _priorityOrder list based on the _orderIdx. Then we iterate through the queue. If we find an entity that matches our priority, return it.

However, what happens if we don’t find a message with our priority? That is ok, we simply return the highest priority item first found in the queue. The highest variable is a reference to the first item in the queue with the highest priority and is returned if we don’t find a match.

The only thing we require from the entities in the queue is that they implement a priority member (or method) that returns the priority number. This deviates from the priority queue in that the queue wasn’t concerned about the priority number, just that an entity in the queue could compare to others. In our case, we need to know the priority to filter the entities the way we want to.

Like the priority queue, the lower the priority number, the more important it is.

We can create our round robin queue like this:

self.q = RRPriorityQueue([1,2,1,3,1,2,1])

In this case, priority 1 messages are guaranteed to pop 4 out of 7 times. Priority 2 message are guaranteed 2 out of 7. Priority 3 – 1 out of 7.

You can make whatever pattern with whatever priorities you want. Just understand that if a priority of an entity is not in the priority order, it can be starved. It will be popped if it is the most important message when a message of the current filter order isn’t found.

When using SQLite3 and Python (2.x), there are two important lessons that are not obvious (at least not to me).

1. Dictionaries and TimeStamps

Ideally, I would like to do two things. First, access data from a dictionary and not a list. It is far more intuitive to access by column name (or query name substitution) than by list index. Second, the datetime values should be correctly coerced even though SQLite has no implicit timestamp type. I believe these two simple requirements are expected by most people, especially those family with Microsoft SQL and ADO.NET.

Good news, Python supports this! However, there are some switches to set so to speak.

After importing sqlite3, the following connect statement will suffice for both needs:

The PARSE_COLNAMES returns a dictionary for each row fetched instead of a list.

The PARSE_DECLTYPES empowers python to do a bit more type conversion based on the queries provided. For example, if you do this:

cur.execute('select mytime from sometable')

You will get mytime as a string. However, if you do this:

cur.execute('select mytime as "[timestamp]" from sometable')

You will get mytime as a datetime type in python. This is very useful.

One step further; let’s say you want a timestamp but substitute the column name in the query. Do this:

cur.execute('select mytime as "mytime [timestamp]" from sometable')

Not only with the data be returned as a datetime object, the dictionary will contain column name substitution provided in the query. Beware, if you don’t do this on aggregate functions, the python sqlite library will attempt to add to the dictionary with an empty string key. (Not sure why this is but beware.)

The adapters and converters live in the file dpapi2.py in your python library installation directory for sqlite3.

So I wanted a super cheap but effective Kiosk. The Beaglebone Black (BBB) was a great choice with a 7″ screen from 4D Systems. It is a ARM Linux machine with 4GB of internal storage. I have a Rev C board that I got from Element 14.

For my project, I wanted a web server (sitting in front of a physical system I was controller) and a dedicated browser on the front panel.

Some Initial Steps

When I got the BBB Rev C, I plugged in the USB. No connection. Then I watched my router while I plugged in the network connection to see what IP address was served. Nothing. Then I found on the forums that the BBB may not have shipped with the intended OS installed. That was my case.

This guy’s site is pretty goodat describing the procedure even though he was using Ubuntu. You can get the latest debian image here. Install the OS and everything mentioned in the prior paragraph will magically be fixed.

Swapping In Lighttpd and PHP

The first step was to stop apache and bonescript so that we could install lighttpd and php.

Get a root shell on the BBB. This can be done using PuTTY on Windows or a Linux box. I used both because I am that sort of guy.

Put your pages in /var/www or wherever you prefer (as changed in lighttpd.conf). You can manually restart lighttpd but I just reboot because I am that kind of guy and the BBB boots in 10 seconds.

Front Panel

As mentioned before, our front panel was a 7″ display from 4D systems.

You can choose to remove X from running by disabling lightdm (associated server and sockets) using systemctl. I would rather keep the front panel and have a kiosk that is a web browser.

Side note: I tried to swap nodm for lightdm but ended up getting some error message that I don’t remember and got verify frustrated as X wouldn’t start except from command line. I suspect it had something to do with the order systemctl started certain services. I gave up because it wasn’t important to me to figure it out and reflashed the OS. I wanted to see of nodm and matchbox were better (i.e. smaller and faster) than lightdm and LXDE but oh well.

So, in this case, we will continue to use lightdm, have it auto logon to the debian account and start the kiosk.

This was really simple. There are two steps.

First, have chromium start when LXDE starts. Edit /etc/xdg/lxsession/LXDE/autostart. Comment out the lines beginning with @lxpanel and @pcmanfm. You don’t need the desktop panels and file manager. Add the following to the file: @chromium –disk-cache-dir=/dev/null –app=”http://localhost”. Of course, substitute your path if different. The disk-cache-dir set to /dev/null prohibits chromium from caching pages. This is critical if update your pages periodically.

If you have reviewed any chromium documentation, there is also a –kiosk flag. However, two very undesirable aspects arise. First, chromium balks about not having Google API keys installed. I don’t care but chromium is relentless. Second, if the browser is stopped hard, on the next startup, chromium asks if you want to restore your session. Again, I don’t care but chromium is relentless. Using the “app” flag guarantees these relentless messages do not appear. Also, you will not see any navigation bar, etc.

Second, the screen isn’t full and you get window decorations. Both need to go. Edit the /home/debian/.config/openbox/lxde-rc.xml file. Under the <applications> tag.

Future Considerations

First, the scroll bars are way too small for a finger. I didn’t get around to changing this. Scroll bars should be avoided. We will elaborate more.

Second, I have been in several companies now where they think that a browser on a PC will look the same as a browser on a 7″ display. Nope. Two problems. Real estate and the size of a finger versus the control of a mouse. Buttons have to be big and pages must minimize scrolling.

Third, in this day and age, we want to scroll by sliding our finger like a cell phone. Unfortunately, this is not the way this works in this case. Your finger is really a mouse and the screen is not a “real” touch device like a phone or tablet.

I want to play with Android for BBB but that will be a future article.

There are cases in the Windows world where you only want a single instance of an application. So, if we launch the application either from the Start menu or command line, or we launch the process from another application, we only ever want one instance – the same one.

If an instance of the application already exists, we want the application to come to the foreground. Not only that but we want the view in that single instance to change based on how the second instance was invoked.

Much has been written on this including using .NET remoting. I chose a far simpler way involving memory mapped files, passing command line information from the second instance to the first. (I freely acknowledge that there is a hack component to it.)

We will begin with the essence of the operation in the main function of your application.

/// <summary>
/// The main entry point for the application.
/// </summary>
[STAThread]
static void Main(string[] args)
{
// The helper object for single instances
ApplicationManagement applicationManagement = new ApplicationManagement();
// If we have arguments, pass them to the original instance.
// If we are creating the first instance, the same memory mapped file mechanism is used for simplicity.
if (args.Length > 0)
applicationManagement.SetArguments(args);
// if a previous instance exists..
if (applicationManagement.DoesPreviousInstanceExist())
{
// Exit stage left
applicationManagement.ShowPreviousInstance();
return;
}
// in the case where we are the first instance and we don't have a previous one, then
// default to the default view
if (args.Length == 0)
applicationManagement.SetArguments(new string [] { frmMain.commandLineFlagForm1 });
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
// pass the application management object to the main form
frmMain main = new frmMain();
main.applicationManagement = applicationManagement;
Application.Run(main);
}

Here is the operation truth table with this code.

command line

new instance created

previous instance exists

no arguments

show default view

do nothing (but set to top of Z-order)

has arguments

set view to the argument

set view to the argument

Notice that if a previous instance exists, we want to bring the previous instance window to the top of the Z-order. The previous instance itself can not go to the foreground (without attaching to thread input of the top order window but we are not doing this). While a new process is being created, until the first window of the application is created, the new process can send any other window of any other process to the top of the Z-order. This is a peculiarity of Windows.

Now a hacky part… In the main form, there is a timer that periodically checks for new arguments to appear.

Essentially, the form’s timer will periodically check to see if there are new arguments. In this example, the command line can specify a form 1 or a form 2. If form 1, it can pass additional information in as well.

So now we will get to the handling of the previous instance and bringing it to life and then we will deal with passing command line information.

For single instance, the applicationManagement class that you see handles the single instance. The constructor determines if a previous instance exists.

You can see the methods where we determine if a previous instance exists and also showing the previous instance. The ShowWindow, EnableWindow, and SetForegroundWindow API calls can be optained here: http://www.pinvoke.net/.

Finally, the here is the code in the applicationManagement class that handles passing through the memory mapped buffer.

/// <summary>
/// The map name used for passing arguments through the MMF
/// </summary>
private const string mapName = "MySecretMapName";
/// <summary>
/// The argument stream length in bytes
/// </summary>
private const int argumentStreamLength = 512;
/// <summary>
/// The argument reference that will be used to pass arguments through a memory mapped file.
/// </summary>
private MemoryMappedFile m_arguments = null;
/// <summary>
/// The accessor to pass arguments through the memory mapped file
/// </summary>
private MemoryMappedViewAccessor m_argumentViewAccessor = null;
/// <summary>
/// Creates the argument stream.
/// </summary>
private void CreateArgumentStream()
{
m_arguments = MemoryMappedFile.CreateOrOpen(mapName, argumentStreamLength, MemoryMappedFileAccess.ReadWrite);
m_argumentViewAccessor = m_arguments.CreateViewAccessor();
// Set up the buffer with nulls
ClearArguments();
}
/// <summary>
/// Clears the arguments by setting everything to nulls which is still a valid string but has no meaning
/// </summary>
private void ClearArguments()
{
byte[] buffer = new byte[argumentStreamLength];
m_argumentViewAccessor.WriteArray<byte>(0, buffer, 0, buffer.Length);
}
/// <summary>
/// Gets the arguments from the memory mapped file and convert back to a string array.
/// We are using a less than pristine synchronization method of looking for any nulls in the string.
/// This method does however work as we are looking for the last null to be removed which should happen
/// in SetArguments.
/// </summary>
/// <returns></returns>
public string[] GetArguments()
{
byte[] buffer = new byte[argumentStreamLength];
m_argumentViewAccessor.ReadArray<byte>(0, buffer, 0, buffer.Length);
// get the string of arguments from the buffer
string args = Encoding.Unicode.GetString(buffer);
// if there are any nulls, then we do nothing but return an empty string
if (args.IndexOf('\0') != -1)
return new string[0];
// if there are no more nulls the we have something to evaluate
args = args.Trim(); // remove the spaces
ClearArguments();
// split the string based on our marvelous separator
return args.Split(new string[] { argumentSeparator }, StringSplitOptions.RemoveEmptyEntries);
}
/// <summary>
/// Sets the arguments from the command line. This is a poor mans way of doing this. A more "pristine" way would be to
/// serialize and deserialize the command line argument string array.
/// </summary>
/// <param name="args">The arguments.</param>
public void SetArguments(string[] args)
{
// puts the nulls into the MMF
ClearArguments();
// Removes all the null when it is finally written to the MMF
string arguments = String.Join(argumentSeparator, args).PadRight(argumentStreamLength, ' ');
byte[] buffer = Encoding.Unicode.GetBytes(arguments);
m_argumentViewAccessor.WriteArray<byte>(0, buffer, 0, buffer.Length);
}

This is a really simple information passing with some inherent synchronization although it is a bit hacky. Essentially, the memory mapped buffer is full of nulls. When the previous instance is going to put the command line arguments into the MMF, it copies all the command line arguments converting from a string array to a joined single string with a clever separator (lame). Then it pads to the length of the buffer with spaces.

The primary instance will wait until there are no more nulls in the buffer before evaluating. In this way, we achieve some form of synchronization. After the complete string is read and split to the original string array, the buffer is cleared out. (This is also a clear last one in wins scenario.)

So there it is. If there are questions, post a comment. I would love to see something other than “buy Nike shoes written in Chinese”.

A friend pointed out that while this was cool (ok, I may have change the tone there), it doesn’t internationalize well because a) it assumes the US keyboard layout and b) it assumes a ‘.’ is always the radix point. Bad, bad, bad.

Instead he turned me onto letting Windows do its job and checking the validity of the float to begin with and enabling the user to press “OK”. Like this…

For systems where you must make a decision to validate or preempt keystrokes, validation will always be best. By validation, I mean that you do not let the user continue unless all the data is correct. This is true with web page development as well as Javascript will parse floats based on locale.

I frequently have the need to spin up a file server, either for temporary use or for longer term storage.
Recently I was migrating a system from one place to another and needed a large amount of fast temporary storage.
At home I have a file server for mass storage of music, videos etc.

There are lots of solutions to this. Could always just use an old Win XP box, or set up a Linux server using your favorite distro. Then there are free software systems, FreeNAS is a good example.

One I like is Server Elements.
They have a line of products including one that boots from a floppy! Hardware requirements for all of the products are very modest. Basically take some old PC, stuff it full of old disks, create a bootable CD, or Thumb Drive and off you go. Prices range from $10 – $35 for the 64bit product with some media streaming capabilities.

For quick set up – Server Elements is really great and worth the small price.
If you want more features, FreeNAS is a very good choice.