MacOS X Yosemite: How to mount a disk on startupMacOS X Yosemite removed few capabilities that existed in previous versions of MacOS X. Notably the StartupItems feature which allowed...MacOS X Yosemite removed few capabilities that existed in previous versions of MacOS X. Notably the StartupItems feature which allowed you to run scripts when the system boots up.

On my machine I want to be able to mount an internal disk as soon as the machine boots up, but before a user logs in. A common solution in the past involved a custom StartupItems script, but this no longer works on Yosemite.

To solve this problem I created a LaunchDaemons script. My disk's volume name is BigDisk and is accessible via /dev/disk0s2. To mount it on startup I created a file named com.jollyturns.disks.plist in /System/Library/LaunchDaemons with the following content:

When the machine starts up, launchd will load the file since we have the RunAtLoad set to true. It then finds that /Volumes/BigDisk does not exist, since the disk is not yet mounted, so it executes the diskutil command to mount the directory.

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000490.html
Appleovidiu2015-01-28T13:52:34-08:00JollyturnsJollyturns - a ski/snowboarding mobile application I worked on for more than 2 years, is finally live! The focus for...Jollyturns - a ski/snowboarding mobile application I worked on for more than 2 years, is finally live!

The focus for the first release was on providing as much ski resort information as possible. We produce the ski resort information ourselves, and it includes detailed information on ski lifts, ski runs, restaurants/lodges, snow and weather information.

Future releases will focus on better statistics and improved social interaction.

Please check it out and let me know what you think by sending email to support@jollyturns.com.

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000488.html
Workovidiu2014-01-13T11:37:46-08:00BeagleBone Black vs Arduino UnoI just received my BeagleBone Black from Adafruit. It's a very sweet board and the price is just right. US...I just received my BeagleBone Black from Adafruit. It's a very sweet board and the price is just right. US $45 will get you a 1GHz ARM processor with 512MB RAM and 2GB flash, built-in Ethernet (10/100Mbps), video graphics card with HDMI output port, 65 I/O ports, 4 hardware UARTs. A very capable board for driving the various small projects I have in mind.

The first project I will work on is to replace a Guruplug+Arduino-based project that was driving my projection screen. The setup I had was quite nice, but it appears the Guruplug died after running great for the past few years.

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000487.html
BeagleBoneovidiu2013-05-06T17:44:02-08:00Rsync over ssh: the dreaded "writefd_unbuffered failed to write 4 bytes to socket" error I use rsnapshot to implement a backup solution for the computers in my home. Rsnapshot runs as a cron...

I use rsnapshot to implement a backup solution for the computers in my home. Rsnapshot runs as a cron job on a ReadyNAS Pro Business Edition system with 6 2TB drives inside (a total of 7.4TB in an X-RAID2 configuration). It backs up the data from a Mac Pro using rsync over ssh. I would have used TimeMachine but its inability to reliably run on non-Apple hardware and with volumes larger than 2TB drives me nuts.

Every once in a while, especially when I have some new fresh data to be backed up, I see rsync start up but it mysteriously dies after a short while (in in /var/log/rsnapshot.log). Running the same command in a terminal gives the following error:

I upgraded rsync both on the source machine (the ReadyNAS), as well as on my Mac Pro to 3.0.8, by manually compiling the latest version. However the error still persisted.

Googling around didn't reveal any solution to the problem, which apparently goes back all the way to at least 2008. On one forum a poster suggested removing the compression from rsync and let ssh handle it. This works for small files but it tends to break on the larger files (larger than 15GB in size).

What seems to work best however is remove compression altogether. Another thing I've done is to use rsh instead of ssh. Here is the new command line (note the new paths to rsync 3.0.8 on each machine).

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000486.html
Open Sourceovidiu2011-06-17T14:48:47-08:00Arduino Tiny Web Server - part 3Update (September 3rd, 2011): The code is on Github: https://github.com/ovidiucp/TinyWebServer A zip file of the latest version of Arduino TinyWebServer...Update (September 3rd, 2011): The code is on Github: https://github.com/ovidiucp/TinyWebServer

What is it?

Arduino TinyWebServer is a library implementation of a small web server running on an Arduino Duemilanove or Uno using the new Ethernet Shield released at the end of September 2010. The new Ethernet Shield adds a microSD card right on the board, in addition to fixing a number of hardware bugs from the previous iteration.

The library is meant to provide a Web interface to your Arduino project. It is not meant to be a full Web server implementation, just the minimum amount to let you get started quickly.

To save precious memory space on your Arduino, TinyWebServer encourages you to develop your web applications as AJAX applications. This means that you should delegate the page generation and complex UI interactions to the Web browser, instead of trying to handle it on the Arduino side of your project. The only thing the Arduino should implement is retrieving and displaying the internal state, as well as modifying it using HTTP POST requests.

Changes from the previous release

As you're probably aware, a new Ethernet Shield for Arduino was released at the end September 2010. I bought one and I'm pleased to report that it fixes an annoying hardware bug present in the old shield. In addition it has a new microSD slot that is actually accessible through a dedicated SPI SS pin. This means that you can use an Arduino Duemilanove or Uno with the Ethernet Shield with no modifications whatsoever.

Compared to the previous release, the new Arduino TinyWebServer release removes the dependency on the EthernetDHCP library. As a result you need to configure the IP address of the Ethernet shield manually inside the examples to match your network. Of course you can still add EthernetDHCP support to your own sketches if you wish.

The library now uses the SdFat library instead of the Fat16. This allows the use of microSD cards larger than 2GB, formatted with FAT32, not only cards less than 2GB formatted with FAT16.

The new release no longer uses a modified Ethernet library. It instead uses the existing Ethernet library that comes with the Arduino IDE.

The code was updated to work with the latest Arduino IDE, version 021.

I fixed a bug related to memory allocation and how the Ethernet Client objects were returned through the API.

In the new release I've added a new example that shows how you can control an LED from a regular Web browser. The video below demonstrates how it works.

Documentation

Arduino TinyWebServer is a library and as such requires you to write code customized for your own application. Some almost ready-to-go examples can be found inside the distribution in the examples/ directory. Follow the instructions in the README.txt files for each of the examples.

An explanation of the library's API can be found in this older post. Some things like the EthernetDHCP code no longer apply, so please look at the new examples for the updated code. Still most of the code is still applicable.

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000484.html
Arduinoovidiu2010-12-08T22:36:40-08:00Transcode AVCHD to MPEG using ffmpegI recently bought a Panasonic Lumix DMC-LX5 camera, which records video in both MPEG and AVCHD formats. The camera has...I recently bought a Panasonic Lumix DMC-LX5 camera, which records video in both MPEG and AVCHD formats. The camera has some nice improvements over the previous LX3 camera. I really like the extended zoom range, 24-90mm versus the 24-60mm in the old camera. Another feature I really love is the ability to zoom while shooting a movie. The sounds quality improved significantly as well, some videos I took with the old camera had some strange sound issues.

One thing that drove me nuts however was the inability of iMovie 7 or Final Cut Express to import the AVCHD movies I made with the camera. They would import only the first few minutes of the movie and invariably stop after that. The MacOS X software that comes with the camera doesn't handle movie files, and the Windows version sucks badly.

So I was stuck with no way of converting the AVCHD movies to an MPEG file format that can be viewed on MacOS X. Adobe Photoshop Lightroom 3, the program I use to catalog the photos and videos I take, does not understand AVCHD either.

You can however convert the .MTS files from the AVCHD directory created by the camera using the open source ffmpeg tool. To do this, make sure you have MacPorts installed on your computer, then install ffmpeg like this:

sudo port install ffmpeg +gpl +lame +x264 +xvid +mp3 +aac

Once you have ffmpeg installed, you can convert the MTS files like this:

ffmpeg -i <file>.MTS -vcodec mpeg4 -f mp4 -sameq <file>.mpg

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000483.html
Open Sourceovidiu2010-12-08T10:54:08-08:00How to build an Arduino project from the command line For the projection screen controller I'm building I decided to use an Arduino hooked up to a GuruPlug. This...

For the projection screen controller I'm building I decided to use an Arduino hooked up to a GuruPlug. This way I offload the communication with the outside world to the GuruPlug, which runs a full Linux server, complete with Ethernet and WiFi connectivity. The Arduino is now used to control the motors and relays, and to receive input from an IR sensor, limit switches and infrared proximity sensors. This solution replaces the relatively cumbersome solution involving an Ethernet shield hooked up to the Arduino.. The software on the Arduino becomes a lot simpler too, without the Arduino TinyWebServer running on the Arduino board itself.

However it does have its own complications. Since the Arduino is now hooked up to a remote computer, I need to be able to build the C++ code for the Arduino on my host development machine (a MacOS X), push it to the GuruPlug running Linux and have the GuruPlug program the Arduino with the new binary.

The build process for the Arduino is handled internally by the Arduino IDE. The logic needed to build a typical Arduino project is hidden inside the Java program, which knows about the Arduino built-in libraries, as well as any additional user-defined libraries. I was hoping I could write a simple makefile to do the actual build, but the library dependencies are determined implicitly from the C header files included by your project's .pde file. It could be done using a combination of shell and awk, and a recursive invocation of make, but it's slightly more complicated. A major problem I hit was with file names containing spaces, which abound at least on MacOS X. Make uses a lot of built-in functions that use space as delimiter between the elements of a list, and it's impossible to use any built-in make functions.

Because of the space character problem, I gave up on make. I was looking for a build system that uses a more programatic approach, and so I found SCons. It's a nice build system built entirely in Python. I was about to write a build file for it when I found arscons, a small build file for Arduino-based projects. The funny thing is that the project was checked in Google Code only two weeks ago. Very timely indeed!

In classic open-source fashion - which I haven't done since I joined Google, back in 2003 - I made few improvements to the code and sent them to the author. I can now build my Arduino project on MacOS X, have it pushed to the GuruPlug and from there uploaded to the Arduino board connected to it.

Prerequisites

For this to work there are a few prerequisites for your development system. Make sure you have Python 2.5 or newer installed. Once you have Python installed, install SCons and the Python Serial module. On a Linux system you can install these by doing:

sudo apt-get install python2.5 python-serial scons

If you're on a Mac, you can get these slightly easier by installing MacPorts. If you do this, make sure you're running the python that comes with MacPorts, and not the default one in the MacOS X distribution. If you decide not to use MacPorts, download and install the packages manually, the process is pretty straightforward.

If you want to push the Arduino binary to a remote machine where the Arduino is connected, you also need to install the Python Fabric package on the host computer. On the remote computer (the GuruPlug in my case), you need to install Python Serial and avrdude. Again if your remote computer is a Linux box like the GuruPlug, you should be able to run apt-get to install the packages.

Once you have these packages installed, download the SConstruct file and put it in your Arduino project's directory. If you plan on remotely pushing the binary, download the pulsedtr.py file and place it in the same directory.

Building your Arduino project

After you install everything, you should be able to build your project using this command:

scons

This will only build the project. To build it and upload it to an Arduino connected to the same computer, run this:

scons upload

If you have a single Arduino board connected to your host computer, scons should be able to identify it automatically. If for some reason it doesn't, run scons like this:

scons ARDUINO_PORT=/dev/<arduino-tty-device> upload

If your Arduino board is not a Duemilanove, you might need to specify the board type using ARDUINO_BOARD. To see a list of possible values for this argument, run scons like this:

scons ARDUINO_BOARD=

Beside the parameters defined above, a few others can be defined in the command line:

ARDUINO_HOME used to specify the location of your Arduino IDE package

ARDUINO_VER the version of the Arduino IDE

EXTRA_LIB if you have additional libraries that you'd like to use for the build, specify that directory using this argument.

For additional parameters that you can specify, look in the SConstruct file.

Deploying to an Arduino board connected to a remote computer

The most complex scenario is when you develop on a machine and push the binary to an Arduino connected to a remote computer (in my case that's a GuruPlug). This usually happens after the development phase, when you deploy your Arduino/GuruPlug project. At this point you don't want to take out the Arduino board from inside your project's box. This option allows you to reprogram the Arduino remotely, directly from your development machine.

To do this, you need to make sure you can login to the remote computer (GuruPlug in my case) using ssh without being asked for a password. You can do this by creating a new RSA key and placing the .pub key in the ~/.ssh/authorized_keys on the remote computer. Let's say your new key is in ~/.ssh/guruplug. You should be able to do this from your host computer, and be logged in to the remote machine automatically (obviously you need to replace user and remote-machine accordingly:

ssh -i ~/.ssh/guruplug user@remote-machine

You can now run scons and have it build, push the Arduino binary to the remote system, and upload the binary to the Arduino board connected to the remote system. Here's how:

Possible issues

I ran into a strange while trying to push the binary to the remote system using fabric. From the stack trace the error was inside paramiko, the library on which fabric is based. This problem was happening even after I created a new RSA key without a password. Each time I would run scons I got this error:

SSHException: invalid key

The error turned out to be inside ~/.ssh/known_hosts which apparently contained an invalid key, or a key paramiko couldn't parse properly. I removed all the suspect keys from it, and the error disappeared.

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000482.html
Arduinoovidiu2010-09-30T21:40:37-08:00How to build a plastic box for a small projectI needed a small box to house some electronic boards for a remotely controlled projection screen project I'm building. The...I needed a small box to house some electronic boards for a remotely controlled projection screen project I'm building. The GuruPlug computer I'm using for the project has WiFi on board but no external antenna. A metal box would severely limit the radio waves, if not shield them completely, so I decided to go with a plastic box instead.

I couldn't find any plastic boxes that would fit my project, so I decided to build my own. Luckily there is a TAP Plastics store right in downtown Mountain View, and they carry a large selection of plastics. On their web site they have a lot of videos showing what can be done with plastics.

It turns out you can build a box using acrylic panels very easily. Acrylic is very easy to cut holes into and you can glue the panels of the box almost as easily as you can glue paper. Check out this video showing how.

Below is the finished plastic box. If you don't have the machines at home to cut the panels to length, you can have TAP Plastics cut the pieces for you. I just got a bunch of scrap plastic panels they had on sale for $1 a piece, cut them on my table saw, drilled the holes using a Dremel, and glued them together. The total cost for the box below was $2 (excluding the tools and the time used to build it :)

Pictures that document the building process of the box can be found in this Picasa album.

Here are few tips when building your own box:

Draw a diagram of how everything fits together and make sure you take into account all dimensions, including the thickness of the panels.

Measure everything using the metric system, the inches are too coarse grained to achieve a good precision.

Before glueing make sure the pieces fit the way they're supposed to.

Use eye and ear protection. Drilled plastic flies around and it's really hot, you don't want this nasty stuff to get into your eyes. The Dremel is loud and I prefer to maintain my hearing to listen to music.

When cutting with a Dremel the right attachments will make your life a lot easier.

If you don't already own a Dremel, this Dremel kit has the basic tool plus a lot of goodies that you'll find useful later on.

The circle cutter attachment for cutting circles. If you have the Dremel kit above, this accessory is included in it.

The Dremel router attachement for cutting straight lines. The circle cutter attachment above has a little metal piece that allows you to cut straight lines with it as well. The router attachment however is much easier to move in a straight line while you push on it.

Here are additional accessories I found very handy when building the box.

When gluing panels together at a 90° it's a lot easier to align and hold if you use an adjustable clamp like this.

If you cut the plastic yourself on a table saw, get a high quality blade for cutting plastic. With this blade I didn't need to route the edges of the plastic pieces I cut prior to gluing.

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000481.html
Hardwareovidiu2010-09-24T08:56:29-08:00Panasonic LX5Whoa, it looks like I missed a slew of new camera announcements. The one I'm particularly interested in is the...Whoa, it looks like I missed a slew of new camera announcements. The one I'm particularly interested in is the announcement of the new Panasonic LX5 camera, the successor of the LX3. I own the LX3 and it's a really great camera, especially in low light.

The LX5 camera extends the zoom range to 90mm from 60mm, it is now 24-90mm f/2.0-3.3, versus the LX3 which had a 24-60mm f/2.0-f/2.8. Sensor size is the same but it's a brand new one which supposedly has an increased dynamic range and even less noise. It looks like the sensor changes will improve movies in low light, since the minimum illumination is as low as 3 lux in movie recording mode.

Another addition is the new AVCHD Lite recording format for movies, which allows longer movies to be shot. The movies are no longer limited to the maximum 2GB file size. I haven't heard about the AVCHD Lite version before, but I find AVCHD a pain to work with. At least on Mac, you can't see the movies by opening them in QuickTime, you need to convert them first using an editing program.

Overall it looks like the LX5 is a nice incremental improvement over the LX3. I can't wait to see some side-to-side image comparison of the two cameras. For my usage, I think the LX3 is good enough. I don't shoot movies that long using the camera, so the new movie recording format doesn't affect me. With a 16GB SDHC card, I have plenty of space to shoot movies and pictures for a whole day.

Judging by the performance of its predecessor, I heartedly recommend buying the LX5 camera. Its low light performance is astounding for a compact camera.

Here are some pictures I've taken at various concerts using the LX3.

Umphrey's McGee at the Warfield, San Francisco:

]]>http://www.webweavertech.com/ovidiu/weblog/archives/000480.html
Photoovidiu2010-09-23T09:52:26-08:00Arduino and GuruPlug Few months ago I attempted to make an Arduino-based project accessible via a web browser and/or a phone using...

Few months ago I attempted to make an Arduino-based project accessible via a web browser and/or a phone using an Ethernet shield (see here and here). The results were a bit disappointing, as the Ethernet shield tends to behave strange when the files to be transferred are large. Everything seems to be fine if the file to be transferred is just below the Ethernet MTU (1492 bytes by default), with extremely fast transfers all the time. When the file exceeds that threshold, there are a bunch of inexplicable timeouts which make the file transfer to be seconds instead of milliseconds.

I then discovered the GuruPlug. For $100 you get a full computer running on a Marvell 1.2GHz processor, with 512MB RAM and 512MB flash as the hard drive, 1 Gigabit Ethernet, 802.11 b/g and 2 USB 2.0 ports! The best part is that the GuruPlug consumes less than 5W, so you can have it on at all times.

The company making the device is positioning the GuruPlug as a general purpose computer, but I think that's a mistake. I'm not going to trust such a puny computer to run my file server. However it's powerful enough to run a bunch of dedicated little applications. I've been running my home's internal DNS server for more than a year now on the previous generation SheevaPlug.

Since the GuruPlug has 2 USB ports and has both Ethernet and WiFi, you can easily connect it to an Arduino and have the web server run on GuruPlug, while having the actual interface with motors and sensors done by Arduino. The communication between the two can easily happen over USB using a custom made protocol. More on the software side of things in a different post

Since I want my project to go inside a single box, and not have lots of wires running around, I disassembled the GuruPlug and removed the main board. Doing so voids the warranty, so do this at your own risk! You need to provide 5V at 1A to the board. I removed the USB connectors on both the GuruPlug and the Arduino board, and ran some wires directly between them. This way I don't need to run a dedicated USB cable between the two boards. On the downside, I lost a USB port which could have been exposed outside of the box.

Right now I'm in the process of building a small box for the project. Since the GuguPlug doesn't have an external WiFi antenna, I needed to build the box from plastic instead of metal. Another thing to keep in mind is that the metal part that's on top of the GuruPlug board is a heat sink that runs very hot. I'll most likely put a fan right on top of it, even though the original GuruPlug doesn't have any. More on the building of the actual box in a later post.