The way that we’re specifying the array in numpy actually helps add some clarity here. The 2 × 3 × 4 matrix is really just a list of lists of lists of elements.

I thought you said it added clarity?

Well, it does. Rows are made up of elements, 2D matrices are made up of rows, and 3D matrices made up of 2D matrices.

Anyway, the point of this post is, what do you call the measure of that third dimension? If the measure of the first dimension (the row) is called “columns”, the measure of the second dimension “rows”, what should the measure of the 3rd dimension be called?

I’m not a huge fan of “aisles”, despite it getting the most upvotes. It just doesn’t seem intuitive enough. When I first think about an aisle, I’m thinking about the narrow passage ways in the stores you walk down, not the sides of those that contain the product (which is where I think that commenter was headed.) That could just be me, however.

I personally also rule out “slices” and “tensors” out, slices having somewhat specific meanings in python, and tensors having specific meanings in linear algebra and mechanics (which don’t align completely to the concept of the measure of the 3rd dimension).

I also rule out “depth”, if only because its not easily pluralized like row → rows, column → columns. A phrase like “number of columns” has some issues when you move to “number of depths”. That being said, the “depth” of a 3D matrix makes a lot of sense to me. Just as you might gain a “depth” perception when you go from a “flat” 2D Cartesian coordinate system to a 3D one.

So to that end, “layers” and “pages” both fit my intuitive concept of the measure of the 3rd dimension. You have a stack of “pages”, each with a 2D matrix printed on it — fine.

Between the two, I think “layers” is probably the better term — it seems more general.

However, there is an interesting “natural” relation that springs up if you go with “pages”. Consider:

Columns

Rows

Pages

Books

Series

“Series” is pretty weak, I admit, but the first four degrees are pretty solid. Nevertheless, I’m still going with “layers”.

People familiar with languages like C, where there are two distinct methods of passing variables to functions (pass-by-value and pass-by-reference) often ask how you can emulate these methods in Python.

So what’s happening here?

Note: This is a simplified discussion, see the Techincal Notes section at the bottom for more information.

In both functions, the value of x is 10 after the *= 2 operation. However, only the second function causes the change persist once the function exits. This is the basic difference between pass-by-value (PBV) and pass-by-reference (PBR).

In PBV, the function receives a “copy” of the variable, and operates on that. Any changes made to the copy of the variable aren’t reflected in the original variable.

In PBR, the function receives a reference to the original variable. Any changes made through the reference are made directly on the original variable and will “persist” once the function ends.

So this explains why, after the function pass_by_val() exited, that the value of v remained unchanged at 5. It also explains why after the function pass_by_ref() exited, the updated value of 10 persisted and was reflected in the final line of the output.

So C is both Pass-By-Value and Pass-By-Reference?

Well, no. I kind of hand-waved in the previous section to illustrate the difference between PBV and PBR. In fact, strictly speaking, C is only pass-by-value. You can emulate a pass-by-reference behavior by passing the function a pointer, but even that is still pass-by-value — in this case, that value is of the memory address the variable is stored in.

Safe Deposit Boxes

Let me try to explain that a bit further.

A scenario

Let’s first think about a bank with a wall of safe deposit boxes. One day, you decide you want to store your Personal Documents, so you go into the bank and reserve a safe deposit box. An unreserved safe deposit box is labelled and set aside for your use. Somewhere in the bank, there’s a log that now contains an entry linking you, your purpose, and the number of the box.

Date

Customer

Use

Box No.

2013-09-22

John Smith

Personal Documents

114

For the rest of this discussion, lets assume that we’ve chosen a very unscrupulous bank to store our things, where the only thing you need to gain access to a safe deposit box is the number. Thats it! Just having the box number will grant you access to the contents inside the box.

Let’s say a few months later you want to store some valuables as well, but your current safe deposit box is too small. So you go into the bank and reserve a second box, this one for your larger valuables. The entries of bank’s log corresponding to you might now look like:

Date

Customer

Use

Box No.

2013-02-14

John Smith

Personal Documents

114

2013-09-22

John Smith

Valuables

237

Now, sometime down the road our spouse wants our birth certificate. But for some odd reason, the only way we’re able to communicate with them is over a fax machine. (Bear with me here).

Anything we attempt to give them will first be scanned, digitized, sent as bits over the phone line, and reconstructed on the other side.

So how can we get our spouse our birth certificate?

We could go into the bank, access our safe deposit box, take out the birth certificate, and fax it to our spouse.

Or, instead we could simply fax our spouse a piece of paper like the following:

Shade E. Bank
1402 Main Street
Box 114

Leaving it up to them to retrieve it from the safe deposit box.

What does this have to do with anything?

Well, this scenario is very much like what happens in your computer when you declare variables. In C, any time you declare a variable, the system reserves a memory location of sufficient size to store that variable. This is just like you taking out a safe deposit box at the bank.

Just like the safe deposit boxes have box numbers, the memory locations that you’ve reserved have addresses. But instead of Box 114 and Box 237, the memory addresses are a bit longer and often expressed in hexidecimal, for example 0x02f41ae7.

Just like safe deposit boxes are uniquely identified by the box number (that is, there are no two boxes in the bank with the same box number), memory addresses uniquely identify memory locations in your computer.

So every time you declare a variable, it’s like you’re reserving a safe deposit box in the bank. And every time you define that variable, or change its value, it’s like you’re changing the contents of that safe deposit box.

And the two options we discussed about how we could get our spouse our birth certificate?

Well the first, where we faxed the original, can be thought of as pass-by-value. We’re giving them a copy of a document, and no matter what they do to it, it won’t affect the original that remains in our safe deposit box.

The second, where we faxed them the location of the original, can be thought of as pass-by-reference. Instead of faxing them the document which they can use directly (albeit a copy of the original), we give them the location of the original itself. In other words, we give them a pointer to the document.

Just like we had two methods of passing our birth certificate, we have two methods of passing a value to a function. But remember that C is always pass-by-value, so that anything we pass will first be “copied”, eliminating our ability to modify it and have those modifications persist once the function ends.

So since we’re always passing a copy of the variable, we can either pass a copy of the value, or we can pass a copy of a pointer to the value (the address of the value).

Just like our birth certificate scenario, passing a copy of the value gives us no way to edit the original. But passing a copy of the address of the original allows us to (through a little bit of extra work) access and edit the original.

So when we pass the function a copy of the value directly, this is pass-by-value.

And when we pass the function a copy of the address, this is also pass-by-value, but here, that value is a pointer. And using this method allows us, by “visiting the address” (or “dereferencing the pointer”), to access and edit original. This behavior is sometimes called pass-by-reference-by-value, but pass-by-value(-of-reference) might make a bit more sense.

Pass-by-value(-of-reference) is a technique used in other languages too. For example, Java passes everything by-value, but for objects, but that value is a reference.

Quick Aside: What’s True Pass-By-Reference?True pass by reference allows you to reassign the variables that are passed to it, and have those changes be reflected once the function exits.

For example, for this function to work, exactly as coded, the variables would need to be passed by reference:

function swap(a,b)
{
t = a;
a = b;
b = t;
}

Note that pointer you could write a very similar function in C, with pointers:

void swap(int *a, int *b)
{
int t = *a;
*a = *b;
*b = t;
}

But you’re not actually passing the things you’re swapping — you’re passing pointers / memory addresses and swapping the values at those addresses. This is the reason that C only emulates pass-by-reference.

While working on a web interface for a project we’re doing, I got really into reading up on HTML5.

One (of the many) cool additions is the ability to specify different tailored input types. Whereas we may have previously just resorted to <input type="text"> as a catch-all input for everything, we can now give the browser some context about what we expect to be entered in the field.

For example, I’ve written many-a-form with something like the following

And I’d hope that the user would enter something like Jones.McGillicutty@foobar.org in the field before submitting.

Of course, if I wasn’t sure that would happen (any production code), I could do some validation. This would always be server-side (as it should be), but sometimes I’d even throw in client-side validation as well for that extra spice.

In the end, the real validation was whether the user was able to receive the confirmation email at the address they specified, but catching errors / omissions immediately, before asking the user to wait around for a confirmation email they’d never get was, of course, a better approach.

But as it turns out, validating email addresses isn’t as easy as it might appear to be. As Mark Pilgrim notes,

Without any negative side effects? you may ask. Indeed. As it turns out, browsers as early as IE6 (circa 2001) handled inputs with unexpected type attributes as having the text type. So what this means is, that by changing your input type from text to email, the worst that will happen on any modern (or not so modern) browser is that it will be silently changed back to text in the DOM.

Fine, no big deal. And in the browsers that do support it, you may even be rewarded with free client-side input validation. In fact, Apple takes this a step further and slightly modifies the iPhone and iPad’s soft keyboard accordingly. So when an iPhone/iPad user enters an input field of type email, the space bar shrinks a bit and buttons are added for the characters @ . _ (at sign, dot/period, and underscore). Screenshots of the iPhone’s automagically adapted keyboard can be found here in Mark’s article.

If you want to test it out for yourself, to see how your browser stacks up, I’ve thrown together a few things you might find helpful:

A tool to show you which of these new types your browser supports — Using the heuristic method described in Mark’s article, it just generates a new input element of each type, and sees whether the element type “sticks”. The pertinent javascript is contained within the file itself, so just view the source if you have any questions. The text under the table is just your User Agent string, which I tacked on there so include in the screenshots I post below.

An example of each of the 13 new input types — Just click the link, then click on one of the input types on the left. A form will load on the right, along with a note indicating whether your browser seems to support it. The types email, url, and search seem well-supported, so you may want to start with them. If your browser supports it, try submitting the form with invalid contents — see if you get a 100% code-free client-side error preventing your submission. Again, all pertinent javascript is included in the file itself, so feel free to view source.3

Screenshots

Firefox (23.0.1)

Google Chrome (29.0.1547.66 m)

Internet Explorer (10.0.9200.16660) (UA string truncated)

Interesting Reads

Mark Pilgrim’s Dive Into HTML5 — an online companion-of-sorts to Mark’s HTML5: Up & Running (O’Reilly), updated by a team of his friends.

Richard Clark’s article focusing on the new input types — an illustrative look at the new input types and how some various browsers render the various widgets and the associated error messages one gets when attempting to submit invalid input.

Modernizr — a well-polished Javascript library that among (many) other things, you can use to determine what capabilities users’ browsers have. My heuristic detection method was based off the one in Mark’s article, attributed to Modernizr.

If you noticed the “Mozilla” in Internet Explorer’s UA string, this article has a interesting and humorous history of the Browser Wars and the origins of “Mozilla” in IE’s User Agent string.

Proper server-side validation is still very good practice; for a number of reasons including security and the fact that not all browsers yet support this built-in validation. ↩

I could come up with some edge-cases where this would affect your code (e.g. using a CSS selector like input[type="text"]), but it seems extremely unlikely. ↩

There is some server-side logic involved in validating the query string and displaying an input of the appropriate type, but it’s pretty trivial. ↩

Quick sidenote here: This failure may be very confusing, especially if you’re not aware of the same-origin policy restriction. The AJAX query will fail (so the fail and error methods will be called), but it will not indicate why it failed. More puzzling is that Firebug shows that the response header as 200 OK, but no data is passed back. Just a heads up in case this happens.

One option, if the server is setup to handle it, is JSONP. You can read more about it on the Wikipedia article, but unless the server is already configured to handle JSONP requests, or you can modify it to do so, this will not be an option for you.

When using third-party data providers, like in my case, these are not options.

What I needed was some way to asynchronously fetch the contents of a URL, but since I couldn’t do it directly, I did it indirectly.

To workaround this issue, I first created an extremely simple PHP script[2] that I called proxy.php:

<?php
$file = file_get_contents($_GET['requrl']);
echo $file;
?>

All this script does is fetch the URL that is specified in the requrl GET parameter and return its contents.

For example,

http://www.mysite.com/proxy.php?requrl=http://google.com

would just display the contents of google’s main page.[3]

And because this file, proxy.php, was hosted locally on the same domain, I’m able to send AJAX requests to it without issue.

If we take the javascript example above, it could be re-written to use the proxy as follows:

Now, the client will fetch proxy.php via AJAX, the PHP server will fetch the specified URL and return it in response to the client’s AJAX query.

If you notice, we’re even able to pass GET parameters to the proxy and those parameters will be used by the webserver when fetching the page. This is because jQuery’s param function will URL encode the query string after creating it from the map, eliminating any ambiguity about where the query string parameters should be sent to.

The reason this doesn’t break the same-origin policy is because the same-origin policy is meant to prevent the client from making unintended/unexpected requests (and usually, exploiting the fact that the client’s cookies are sent and can be used to imitate a user’s intentional actions). In the case of this proposed workaround, it is the server that is making the request and your cookies will be safe.

With any code samples you find on the internet, this one included, you should read up on the functions you’re considering using before putting them in any (especially production) code.

Notes:

This snippet, and the other javascript snippets require jQuery. Also, I specify the response dataType as json so that jQuery will parse the returned json and return an object. If you don’t want this, you can remove or edit this line.

This is just a proof-of-concept proxy script and shouldn’t be used in practice. Without first checking things like referrer or requested URL against a whitelist, anyone could use the proxy for whatever they wanted. It’s just not a good idea to leave production code like this.

Because google uses relative paths for things like CSS and images, the page you fetch via the proxy may look a bit different, specifically it may be missing styling, images or scripts. This is almost never an issue if you’re using a JSON API or even just scraping pages for the text data.

With PuTTY and Xming I use X Forwarding to do a lot of work on my local linux boxes on my Windows box.

After a reformat of both, I ran into the issue where a command like wireshark would work fine but running it via sudo (sudo wireshark) failed with the error:

X11 proxy: wrong authorisation protocol attempted

For me, the only thing that was required was to properly define the $DISPLAY (which was fine) and $XAUTHORITY (which wasn’t set) environment variables and make sure they’re passed through to root via sudo.

The first can be done via:

export DISPLAY="localhost:10" # For example, this might be different for you -- this is probably already set
export XAUTHORITY="/home/<username>/.Xauthority" # For me, this wasn't set. Also, make sure to use the full path and not ~

Once that is done, ensure that these two environment variables are “kept” via sudo by checking /etc/sudoers and finding the lines that look like:

These lines define what environment variables persist for the root user when sudo is invoked. By default in my installation (Fedora 16, x64) both DISPLAY and XAUTHORITY persist (first and last lines). If your file doesn’t have this, the environment variables won’t persist and you won’t be able to use X Forwarding as easily/transparently, so you should consider adding them.

Lastly, since $DISPLAY is already set correctly, I just added the $XAUTHORITY definition (shown above) in by ~/.bashrc file and I was good to go.

Subclipse is a great subversion plugin for Eclipse allowing very tight integration into the IDE. Installing it on a 64-bit JVM is a bit tricky, however, and not as straightforward as it probably should be. Today’s post will walk through setting up Subclipse on a fresh Eclipse install.

We’ll be doing this the “right way”, using public key authentication (“passwordless”), but I’ll also point out the steps that can be skipped if you want to enter a password each time with an asterisk*. Note: entering a password for every transaction gets very tedious and I would recommend using public key authentication as I outline.

First, I’ll list the versions I’m working with:

Java: 1.7.0_03(x64)

Eclipse: 3.7.2 (Indigo) (x64)

SilkSVN: 1.7.5.1 (x64)

Subclipse: 1.8

You’re free to use a newer version if this post becomes outdated, but things may change which may or may not invalidate this walkthrough.

Prerequisites

A 64-bit JVM installed. Installing subclipse is presumably a lot easier if you have a 32-bit JVM installed.

Install

Update: I originally outlined the steps necessary to install a version of Subclipse that included packages that were shortly thereafter outdated. Mark Phippard, the project manager for Subclipse and developer for Subversion stopped by and updated us with the current status of the project, including fact that the updated packages make this install a lot easier. As a result, I’ve struck out the steps (4,5) that were made unnecessary in the latest update. Thanks Mark!

Ensure you have MSVC++ 2010 (x64) Redistributable Package Installed. If you think you have this installed, you can skip this step. If you encounter an Eclipse error dialog stating: no msvcp100 in java.library.path, you’re missing this.

Install TortoiseSVN (or just download TortoisePlink.exe). Note that TortoisePlink.exe is different than the PLINK.EXE that is provided by the PuTTY package and using that version will not work.

Download Puttygen.exe* (available here, either as part of the entire PuTTY package or alone). Note that this step will be necessary if (a) you want to use passwordless login and (b) you don’t already have a private/public keypair (if you’re unsure about this, you probably don’t have one and generating another one is no big deal).

Install public key on remote computer.* Note: this might depend on the OS of your subversion server, but generally the steps are the same: (1) copy public key to subversion server, (2) add key to ~/.ssh/authorized_keys. Something like:

cat id_rsa.pub >> ~/.ssh/authorized_keys

Configure Subclipse to use plink. Open %appdata%\Subversion\config and in the [tunnels] section, add the line

Notes: (1) change paths and username as appropriate, (2) The path separator should either be a single forward slash or a double (escaped) backslash. If you decide not to use public key authentication, drop the last (i) option.

Launch Eclipse

Open SVN Configuration (Window – Preferences – Team – SVN) and ensure JavaHL (JNI) 1.7.5 … is selected under (SVN Interface – Client) and an error dialog isn’t displayed. If you get an error dialog and the dropdown contains JavaHL (JNI) Not available, you may have forgot to exit and restart Eclipse or something went wrong while installing SlikSVN.

Results

At this point, one of four things should happen:

Your repository opens up in the Repo Browser. Congrats! You set up Subclipse successfully. Code on!

A password dialog opens up. Typically this is because either (a) you erred when specifying your private key on the ssh=… line, or (b) you didn’t, or didn’t correctly, add the public key to the authorized_keys file.

You get a red console error:Network connection closed unexpectedly
svn: Unable to connect to a repository at URL ‘svn+ssh://path/to/repo’
svn: To better debug SSH connection problems, remove the -q option from ‘ssh’ in the [tunnels] section of your Subversion configuration file.
This means that you were able to connect to the server, but for some reason the connection later failed. This could be because you specified an incorrect username. Ensure your TortoisePlink.exe options are correct.

You get a red console error:The system cannot find the file specified.
svn: Unable to connect to a repository at URL ‘svn+ssh://path/to/repo/’
svn: Can’t create tunnel: The system cannot find the file specified.
This means that Subclipse couldn’t locate TortoisePlink.exe. Ensure your path is correct and your path separators are appropriate.

and run it, either in the emulator or on a device (with USB debugging enabled), we would see in LogCat an entry tagged with JNIDemo. In this case, we’d expect something like 3->81, since 3^4 = 81. But we’ll do a little bit more to see the performance of the library directly on the UI.

Instead of walking through the specific steps of creating the UI, I’ll simply post the pertinent files:

You can enter a number in the text field and press the calculate button which will result in the squared and ^4 calculations:

Of course this was a pretty silly demo library because a squared function could have been trivially implemented in java without the need for c code, cross compiling or dealing at all with the Java Native Interface, however it still illustrated the steps necessary to compile a native library against the Android NDK and how to import and use it in an Android Project.

Today I’ll be posting a quick walkthrough of how to create and build a simple android project that includes native code using the Java Native Interface (JNI). As a note, there are sample projects included in the Android NDK, but this will walk you through building your own. After going through this, it’s suggested you review these sample projects.

Prerequisites

As a prerequisite for this tutorial, you’ll need:

Eclipse installed and configured to create Android projects. There are a number of tutorials out there about how to do this if you need help.

A JDK installed, as I don’t believe the standard JRE contains the javah command that will be needed.

Create the Android Project

For this tutorial we’re going to create a new project. However there is no reason you couldn’t integrate this into an already-created project.

To create the project, right click in Eclipse’s Package Explorer → New → Android Project. Give your project a name and select an API. For this tutorial I choose the latest Gingerbread API, 2.3.3.

Add jni folder, Android.mk makefile

Once your project has been created, you’ll need to create a new folder inside the top level of the project. To do this right click on your project name → New → Folder. Name this folder jni.

Inside this folder, create a new blank text file. To do this right click on your newly-created jni folder → New → File. Name this file Android.mk. Leave this file blank for now, we’ll come back to it later.

Your project should look something like this:

Create java source

For this tutorial, we’re going to have a simple c program — squared — that accepts an int and returns the square (e.g. 2 → 4, 3 → 9, etc.)

In order to accomodate that, we first create a java source wrapper. The wrapper’s job is to load the library, expose any native functions we wish to use directly, and provide any functions that we want to be able to utilize private native functions.

For this tutorial, we’re going to expose directly the native squared function as well as provide a to4 “derivative” function.

To expose the native function directly, we just declare it public. Alternatively, we could declare it private and limit it’s availability to other functions of the class.

Create C header

After we outline the native methods we’ll be using, we can use this java source to create a c header file with the function prototypes for the native methods we used. To do this, we first have to compile the java source into a class file. You can do this manually via the javac command, e.g.:

Note that the -d switch specifies the output directory for the class file — in this case, I’m just throwing it into /tmp.

Now that we have the class, we can create the c header file., e.g.:

cd /tmp
javah -jni org.edwards_research.demo.jni.SquaredWrapper

Note the need to specify the fullly-qualified class name (including package) and not the .class file extension.

The resulting header file in our case is /tmp/org_edwards_research_demo_jni_SquaredWrapper.h, but we can rename it to whatever we want. In this case, we’ll rename it to squared.h and place it in the jni folder in our project directory.

Note we have to give the parameters names and I arbitrarily chose je andjc, and chose base to replicate our java source parameter name.

In this case, the c source is very simple, but this tutorial is meant to illustrate how to include native code into your Android app and more complex c functions could be substituted with few modifications.

Create Android.mk

After we create our c source file, we have to create our Android.mk file. This file serves the as a sort of makefile for the Android build tools. There are a number of sample Android.mk files in the samples/ directory of the NDK and we’ll actually be using almost the exact lines from the hello-jni sample project.

At this point, we have laid most of our groundwork for setting up and compiling the library. The next steps are actually creating the shared library, and implementing some simple UI code to show that our native function (squared) and derivative function (to4) work as expected.

So if you’re ever writing a command in a terminal, only to realize you forgot to do something first (e.g. switch to the right directory, change permissions etc.), you don’t have to delete the entire row, do what you forgot to do and retype.

To some, this might be obvious, they might suggest holding down the left arrow until you reach the beginning of the line and “commenting it out” by inserting a # in front of the text.

But there are actually two quicker solutions, one somewhat obvious, and the other not so much. The reason I’m writing this is just to point out the second one.

The first quick solution would be just to press the “Home” key and then insert the # symbol and press enter. This is the pretty obvious method.

The second method, which is actually a keystroke shorter, is to press Escape then the # symbol. These two keys have the same result as the three keys in the first method.

Saving a single keystroke is really not the motivation for posting this tip, but rather the fact that some terminals, especially when first configuring your client / initialization scripts don’t accept the home key correctly (or quite often your client isn’t sending it correctly).

Anyway, in that situation, its helpful to have a backup option and the “Escape #” method comes in very handy.