news

Like the CV says, I left my position at TRI a few months ago and started up as a Computational Physics Programmer with AECL back in the Old Country. Well, I guess the U.S. is now the Old Country; making Canada the Current Country. One of the hazards of being a dual citizen I suppose, Current vs. Old Country is all about the timing.

I enjoyed my time with TRI but this was an opportunity to grow professionally. I did my share of programming at TRI but it wasn’t my job as such. At AECL it’s all right there in the job description, and for whatever reason I was interested in trying something completely outside my experience. That, and the chance to work with clusters running my code was too good to pass up.

Unfortunately this means that visible work on NDIToolbox will probably slow down, at least in the short term. TRI continues work on the project so I’m hopeful anything they come up with will eventually make it into the open source repositories. In the meantime if you have any questions or run into any trouble feel free to contact me.

Today we forked NDIToolbox into a new project, NDIToolbox Labs. We’ll be using Labs for experimental features before they go into the main NDIToolbox repository. Read on for more background.

One of my main responsibilities at TRI is working with Subject Matter Experts (SMEs), basically gurus in one particular field or another. A big part of my job is in helping SMEs write code from scratch or port it from MATLAB, R, C/C++, etc. to Python. The SME works out an algorithm, I code it up in Python, rinse and repeat.

I started the Labs fork because in many cases the SMEs aren’t familiar with Python or unit testing, but I didn’t want to slow their efforts down by insisting on tests for inclusion in the main NDIToolbox repository. Labs will be the unstable branch of NDIToolbox – stuff might break but it’s where all the cool new features will be. Once the SME is more or less satisfied with how their code is working in Labs, I’ll add the requisite tests and whatnot and port it to NDIToolbox stable.

One of the first new additions will be “Automated Defect Analysis,” a suite of code designed to read data and automatically locate anomalies in the sensor data. Instead of having an inspector scroll through 100 miles of pipeline inspection data for example, you’d let ADA read the data on its own and let it present you with a report of where it thinks cracks and pits were found.

Update Fri Aug 31 13:18:28 CDT 2012:Computational Tools‘ first alpha of the ADA Toolkit has been added to NDIToolbox Labs. Although it is functional and can run ADA Models (specialized NDIToolbox plugins), it’s still in the early stages of development. I’ve also put together a Windows binary if you’d like to check it out and don’t have Python installed. Be sure also to download the ADA Model ZIP from the same page to see ADA Toolkit put through its paces, or copy the URL to the clipboard and download/install from ADA Toolkit itself.

Next up will probably be some Probability Of Detection (POD) models that you’d use to simulate an inspection to find out if the inspection would actually be able to detect the anomalies you’re interested in finding. Going back to the pipeline inspection example, a POD model might tell you whether a lower resolution scan might suffice to find damage; saving you time and money in the inspection.

So to summarize: if you don’t need the latest and greatest, I’d recommend sticking with NDIToolbox stable. If you need the latest and greatest, try NDIToolbox Labs but expect bugs.

Courtesy PyInstaller, I’ve made an NDIToolbox binary distribution for Windows users that’d just as soon not install Python and friends just to try out one program. Download nditoolbox_windows_binary.zip; extract to a folder of your choice, and run nditoolbox.exe from the nditoolbox folder. Also included are a few sample data files to play with if you don’t have anything handy, taken from honest-to-goodness ultrasonic scans. The binary release is intended to be a demonstration of NDIToolbox and will probably lag behind development so if you like NDIToolbox and plan on using it I’d recommend installing Python and taking it from there.

For any fellow PyInstaller users out there, if you have any problems getting h5py to work, just add hook-h5py.py to your hooks folder. It should already be in recent PyInstaller builds, but if you’re using an older release like I am (1.5.1) you can just copy it over manually.

In other NDIToolbox news, this release is the first to come with some early documentation I’ve been working on, including a quick-start guide and a few words about running and writing NDIToolbox plugins. The documentation is available from NDIToolbox’s Help menu, or under the docs folder in NDIToolbox’s folder.

Finally, we’ve more or less completed our project name change from “a7117” (the internal TRI project number) to “nditoolbox” so default folder locations, etc. have been updated. Your old settings and files aren’t deleted, but NDIToolbox is now looking for nditoolbox.cfg rather than a7117.cfg so if you’re happy with your current setup just rename a7117.cfg to nditoolbox.cfg and you’re set.

Just released this week is early support for slicing 3D data. Now when you attempt to plot an image plot of a 3D dataset, NDIToolbox will ask you what z index to use. Same goes for previewing 3D datasets as well.

What’s more interesting is what’s coming up: better support for a few common plot types in Nondestructive Evaluation (NDE). That last link gives a good explanation of the various types of scan but for those of us that didn’t start out in ultrasonic NDE here’s the abridged version.

A scans are waveform plots, voltage vs. time. In a 3D dataset an A scan is the data at a given position (x, y).

C scans are plots of 2D planes taken in the z plane of a 3D dataset. In the simplest case they’re just a slice of data at a given z (i.e. depth into structure) position, e.g. the voltage, time pairs at for every point (x, y) in the plane z=200. In ultrasonics you’ll frequently see C scans created from some feature in the 2D array of A scans, e.g. maximum amplitude for all A scans between z=200 and z=250. If you’re conducting a raster scan of a part and recording voltages at regular (x, y) positions, you’re basically creating a C scan.

B scans are slices through the C scan dataset (at least in our presentation). We’re using the cursor position for our B scans, generating the horizontal and vertical slices through the current C scan dataset.

NDIToolbox already handles A scan and C scan plots but this week I’m working on a four-panel plot that does A, B, and C scan plots of a 3D dataset. Here’s a screenshot of the new type of plot I’m adding:

This is an actual ultrasonic scan of an NDE standard (basically a block with a regular pattern of known flaws at various depths) we made in the lab; when you click on a position in the C scan it will retrieve the A and B scans for that position and update the other plots. You can browse through the 3D dataset by choosing your slice index at the top to update the C scan layer. In this early version you’re limited to viewing a single slice in Z, but the data handler underneath supports running functions against a subset of data in Z. For example, returning the peak-to-peak value between Z=220 and Z=230 and returning that as your C scan data. I’m hoping to have that implemented in the near future.

If you’re not a fan of the four-panel plot, no problem. The aforementioned data handler is a simple Python utility class that when initialized with a 3D NumPy array will return A, B, and C scans on request. So you can slice, dice, and present your 3D NDT data as you see fit.

So far this new plot is fairly responsive; on my machine I’m able to comfortably handle 2 billion data points or around 500MB worth of data in memory without any lag. I mentioned it’s freely available, right? Go get your copy.

Update Mon May 14 14:29:23 CDT 2012: just pushed out the “megaplot” to the bitbucket repo. There’s still more work to be done with plugins and exporting data subsets but the basic plot functionality is ready to go. Here’s another slice through the same ultrasonic data (Linux Mint 12), this time using a different colormap to better highlight the flaws in the test article.

I’m happy to report that I’ve been given the go-ahead to put out the initial release of NDIToolbox, and it’s up over on my Bitbucket page if you’re interested.

This isn’t the same version of NDIToolbox seen over on the About page, this is a completely new codebase I started up in January 2012. It’s basically the same concept – signal and image processing for Nondestructive Testing – but cleaner, fewer dependencies, and hopefully more functional in the long run.

This initial release is really just the user interface, but here are the features so far:

Import of DICOM / DICONDE data files

Extensible with plugins – write your data analysis as an add-on without having to worry about making a user interface

Documentation’s nonexistent at this stage but basically NDIToolbox should be able to handle most ASCII-delimited data files for basic 2D and image plots. There are some examples of plugins included with fully-commented source that will hopefully get you started. While I work on the docs, feel free to drop me a line if you need any help.

Developing the front end (PODToolkit) was interesting for me in that in was one of the first times I’ve really gone through a formal mock-up process. Ordinarily I’ll sketch a few ideas for the user interface on paper or on the whiteboard, but this time around I needed to make sure that my ideas matched pretty closely with the end user’s. After a few phone calls, emails, and one big fax we ended up at this for our mock-up:

Which after a while ended up looking like this:

All in all, I’m pretty pleased with how it turned out. There’s a lot of information that has to be presented when you’re working with POD in Nondestructive Evaluation (NDE) and I like to think we did a good job of putting it together. Hopefully we’ll get the chance to do some more work on it in the future.

Finally completed the move from a hand-rolled PHP template I wrote a long time ago to something more than a little easier to use. Whenever I had anything to put up on the site it used to feel like an all-day affair sometimes, having to update all those files by hand. I doubt I’ll miss that.

I’ve taken down the older content – if there’s something from the old site that you want or desperately need let me know and I’ll get it going again.