Thoughts from a combined Apple/Linux/Windows geek.

As soon as you add a USB host port to your microcontroller project, a lot of possibilities suddenly open up. You can add support for plugging in a flash drive for firmware updates, diagnostics, and all kinds of miscellaneous data transfer. One task I’ve needed to complete in the past is the ability to save an Excel .xls spreadsheet. After doing some research, I found that this was actually a fairly difficult task on a microcontroller with limited memory. Most of the C/C++ Excel libraries I’ve found aren’t prepared to work in a microcontroller environment, and they generally do dynamic memory allocation because, well, Excel files are dynamically sized.

I’m going to assume that you already have USB mass storage support figured out for your particular project. This will usually involve using a USB library that provides the USB functionality. Many USB libraries also provide support for accessing mass storage devices. On top of this, you will also need a filesystem library for accessing a FAT32 filesystem, which is arguably the most common filesystem in use on flash drives today. One library I would recommend for working with FAT32 filesystems is FatFS. Note that FatFS doesn’t provide any of the USB functionality — you will have to hook it up to your USB library in diskio.c.

Now, back to the actual file generation. One thing you can do is simply generate a .csv file instead. That strategy works pretty well, and it may end up being all that you need for your particular project. One downside is that in certain cases, it will bring up a dialog box that forces the user to specify options for importing the file. LibreOffice is a particular offender in this regard.

I wanted a solution that generated a real .xls file that Excel and LibreOffice could open without any prompting and didn’t require the use of malloc(). I ended up writing my own library called MicroXLSWriter. MicroXLSWriter is a simple library that should fit on any embedded system. It generates files in a very old .xls format (BIFF2, from Excel 2.0). Modern versions of Excel (tested through Office 2007) can still open BIFF2 files without any problems. My code doesn’t completely follow the BIFF2 format perfectly, but it’s good enough that Excel and LibreOffice don’t seem to complain.

MicroXLSWriter is very limited. It doesn’t support any cell formatting or styles. You can’t customize fonts, colors, borders, or anything like that. The only thing it lets you do is set the width of each column and put text or a number into each cell. For what I needed, this is completely fine. BIFF2 files aren’t typically supported by .xls manipulation libraries, so if you’re generating something that won’t simply be opened by a user with Excel or LibreOffice, you should probably look elsewhere — if you can find a better alternative that still fits in a microcontroller.

If you’re interested, try it out. The source code is on GitHub and it has a very permissive 2-clause BSD license that allows you to use it in whatever commercial or open-source project you want, as long as you follow the simple requirements about preserving the copyright notice in materials distributed with your project (e.g. documentation). If you get a chance, let me know if you use it! I’m curious to hear about various applications where it ends up being used.

For various reasons, you might end up needing to hook up your iOS device directly to an Ethernet network without using Wi-Fi. For example, maybe you’re in an area with bad Wi-Fi reception. Or perhaps you are trying to do something that requires high bandwidth and your Wi-Fi access point doesn’t have the best speed.

Well, it turns out that iOS has supported Ethernet for a while. Obviously your iOS device doesn’t have an Ethernet port, so you have to use a USB-to-Ethernet adapter. As of iOS 10.2, there is even a settings screen that appears when you plug in a supported adapter. Older versions of iOS apparently still supported Ethernet, but didn’t provide a screen for setting it up, so you were probably stuck with the default settings (grabbing an IP address from a DHCP server). Now that there is a setup screen, you can configure it exactly how you want.

I tested it out with three devices tonight, and they all worked flawlessly: iPhone 6, iPhone 7, and iPad Air.

I recently ran into a problem upgrading a computer from Windows 7 to Windows 10. I received an error message with the following error code: 0x800703ed

It turns out this system was a dual-boot Linux and Windows system. I was actually booting to a different hard drive which had GRUB installed, and then GRUB was booting to my Windows drive. I fixed the problem by switching my BIOS boot order to boot directly to the drive with Windows 7 installed and then repeated the update process. This resulted in a successful Windows 10 upgrade. After the upgrade was complete I was able to change my BIOS boot order back to the way it was before, and now everything works fine.

After upgrading my Ubuntu server from 14.04 to 16.04, I noticed that legacy AppleTalk clients connecting to my netatalk (afpd) server showed an empty volume list; I could no longer access my “Home Directory” share that showed up by default in 14.04.

After some research, I discovered that the 14.04 to 16.04 upgrade brought netatalk from version 2.2.2 up to 2.2.5. It turns out that 2.2.5 has a bug which causes AppleTalk clients to not see anything from /etc/netatalk/AppleVolumes.default; only /etc/netatalk/AppleVolumes.system is read. Here is a link to the bug report I posted.

You could rebuild netatalk 2.2.5 with the latest unreleased patches as I mentioned in my first comment of the bug report above, or you can do what I did: just put your volumes into /etc/netatalk/AppleVolumes.system instead. I’m unsure if the “~” share works properly when it’s in /etc/netatalk/AppleVolumes.system, so I hardcoded it to my home directory path just in case. It works fine!

Recently I had an idea: what if I maintained a local server to host my Ubuntu install ISOs so I could load them through the PXE boot capability available on all of my computers? I would never need to create an Ubuntu install USB drive ever again — instead, I could just tell the BIOS or UEFI to boot through the network card, pick the ISO I want from a list, and boot right into it. Every x86/amd64 computer I’ve ever used (with the exception of Macs) has had an option in the BIOS/UEFI for booting through the Ethernet port, but it’s something I’ve never been able to try, and tutorials on it are usually old and/or incomplete. I finally decided to try to figure it out. I got it working after a lot of trial and error.

I found plenty of tutorials, but none of them had everything that I needed to know in order to get this setup working. Most of the tutorials that were Ubuntu-specific applied to really old releases (10.04 and below), and generally didn’t support both UEFI and BIOS simultaneously. I can verify that the process I figured out will work for both legacy BIOS and modern UEFI systems, and it will properly boot into an Ubuntu 16.04 desktop live ISO. Specifically, I have tested it with an Ubuntu MATE live ISO, but it should work with all other variants too.

dnsmasq

These instructions will use dnsmasq, which is a nice little DNS/DHCP/TFTP server program that you can install in Linux. If you are using something like DD-WRT, there is a good chance you already have dnsmasq! However, for these instructions it’s apparently important that you use the latest version as of this writing, which is 2.76. I didn’t test with earlier versions, but according to sources online, version 2.76 contains bug fixes for PXE booting with UEFI. So to start, you need to make sure you have dnsmasq 2.76. If your server is an Ubuntu 16.10 server, you can simply install dnsmasq with apt–it’s already the latest version. Otherwise, you may need to manually grab the .deb from 16.10, or compile it yourself. Keep in mind that desktop versions of Ubuntu run dnsmasq in the background for DNS, so you may have to set up something special if you want to run this server from a desktop install of Ubuntu without interfering with the existing dnsmasq. I’m not 100% sure–I did this on a server install so that wasn’t an issue.

If you already have a different DHCP server program, you can probably adapt these instructions to work with it, but you’re on your own to figure out the required syntax.

Proxy DHCP or not?

There is a decision you will have to make which will affect how you configure dnsmasq. Do you want to run dnsmasq as a normal DHCP server, or a proxy DHCP server? Let me explain the choices. As a forewarning, this may take a while to fully explain, but it’s all useful information.

When I say “normal DHCP server”, what I mean is that dnsmasq will replace your existing DHCP server. So you would set up dnsmasq to assign IP addresses to all of your computers and devices on your network, and turn off the DHCP server built into your home router. This new DHCP server provided by dnsmasq will recognize when your computer is trying to netboot, and will tell it to use TFTP to download a file from a server to boot. As you will see shortly, this is probably your best option. It’s understandable, though, if you don’t want to mess around with the DHCP server that you already have which is working just fine with no problems whatsoever. If you’re in that camp, there is another option available: a proxy DHCP server.

If you already have a DHCP server on your network that you can’t configure for PXE but you still want to use (e.g. the DHCP server in your home router), you can set up dnsmasq as a proxy DHCP server that works alongside your existing DHCP server. Your existing DHCP server will still be responsible for assigning IP addresses to all of your computers and devices, but the proxy DHCP server will provide the necessary information that PXE clients need for booting (the TFTP server IP address and file path). You will not have to change any configuration on your existing DHCP server; the proxy just provides additional information that all PXE-enabled BIOS/UEFI implementations are supposed to support. PXE was designed to support this use case, and every computer I’ve tested so far works with it just fine.

With both of these options available, the choice may seem obvious to you: use it as a proxy DHCP server! It’s a low-risk choice because you’re not messing with the configuration of anything else already on your network. In theory, this would be a great idea. In practice, though, there are problems with booting in proxy mode with UEFI. It’s not actually dnsmasq’s fault at all. It’s not the PXE protocol’s fault, and it’s not even the fault of your computer’s UEFI implementation. The fault lies in shim and grub. Let me explain further.

shim is a bootloader that serves a single basic purpose: to load grub. The reason it exists is because it’s possible to have it signed by Microsoft so that it can load on a UEFI computer that has Secure Boot enabled. So for Ubuntu, Canonical provides a version of shim signed by Microsoft that trusts binaries signed by Canonical. This signed shim can then load a version of grub that has been signed by Canonical, and grub will load a signed kernel, and so forth. The trick with shim is that it needs to know where to find grub. It attempts to auto-detect how it was loaded, and looks for a file called grubx64.efi next to itself. This works great when it was loaded from a hard drive or SSD. If it was loaded through netboot, it will try to load grubx64.efi from the same TFTP server and directory from which it was loaded, which it determines by looking at the original information provided by the DHCP server. Unfortunately, shim doesn’t currently detect this information properly when it is booted from a PXE proxy server. UEFI provides the information necessary to figure it out, but shim doesn’t currently look for it.

You could be like me and say, “who cares about Secure Boot?” Instead, just disable Secure Boot on your computers, and boot directly to grub instead of using shim. The problem is that grub has the exact same issue, and it needs to know how it was booted in order to figure out where to get grub.cfg. There is probably a way to work around this problem by embedding an intermediate grub.cfg into your grub binary (using grub-mkstandalone) that knows where to look for the real grub.cfg, but you won’t be using shim so you won’t support clients with Secure Boot enabled.

The easiest thing to do is just not use proxy mode for now if you want to support UEFI clients, especially UEFI clients with Secure Boot enabled. I will still explain how to set up proxy mode at the end of this post if you’re interested, but just know that if you do it, you will need to spend a bit more time figuring out how to get grub to grab its config file and any other supporting files over TFTP when netbooting a UEFI computer. I didn’t bother to figure this out, though, because…

There is actually a third option that may work OK for you if you were hoping you could use a proxy server: use dnsmasq as a normal DHCP server alongside your existing DHCP server, but tell it to assign IPs outside of the range that your existing DHCP server provides, and also tell it to ignore non-PXE clients. That way, it will allow your other server to handle all normal DHCP leases, but it will still be able to respond with the boot info when it sees a PXE client. Your other DHCP server will also respond to the PXE client’s boot request, but I believe the PXE client will ignore your other server because it doesn’t provide any PXE boot info. It works OK in my testing, anyway!

You may be thinking, “Doug, you’re crazy! Two DHCP servers on the same subnet?” I know it sounds weird, but it’s safe to have two DHCP servers on the same network as long as they don’t assign any overlapping IP addresses. Some people even recommend using such a setup intentionally for redundancy. For example, if your existing DHCP server assigns addresses from 192.168.1.100 to 192.168.1.199, you could safely tell your new DHCP server to assign addresses from 192.168.1.200 to 192.168.1.249 with no conflicts at all–normal DHCP clients will use whichever server responds first. It would definitely be wise to ensure your new DHCP server assigns the same default gateway and DNS servers that the existing server assigns, but if you’re configuring it to only respond to PXE clients, it’s probably not a huge deal because the PXE boot process probably isn’t going to need them on a simple home network.

Do initial setup

Regardless of what type of DHCP server you’re going to run, there is a bunch of common setup to do. Let’s get started!

TFTP directory

First of all, let’s create a TFTP directory on your server, and change it so it’s owned by you (replace “doug” with your username):

sudo mkdir /tftpboot
sudo chown doug:doug /tftpboot

Download and extract the live install ISO

Now, we need to download the install ISO. You can store it wherever you want, but the location you choose won’t end up mattering; we’re going to extract the contents of this ISO into a directory on your computer. Start by downloading the Ubuntu .iso that you want to make bootable. In my case, I downloaded Ubuntu MATE 16.04.2 LTS:

Now, we need to do several things. First of all, we need to put the kernel and initrd into your TFTP directory so TFTP clients will be able to access it. I like to make a directory in the TFTP server for each ISO I’m hosting:

Note that even though the kernel is named with a .efi extension, it will still boot a normal BIOS system just fine. The bootloader will know how to handle the file either way.

Now, you need to save a copy of all of the contents of the ISO somewhere locally that you can serve with NFS. I prefer to make a directory on the root of my drive for each ISO, but do what you want. Afterward, you can unmount the ISO.

At this point, you are serving the contents of the ISO inside the directory /ubuntu-mate-16.04.2-desktop-amd64 with NFS, and you no longer need the .iso file. Note that some people might prefer to just keep the .iso file mounted loopback at all times instead of extracting the files from it. If you prefer to do it that way, you can probably set something up in /etc/fstab to automatically mount it, and then set up /etc/exports to share the mounted directory instead. Whatever you prefer!

Now, we need to set up some bootloaders that the PXE clients will load. We will use PXELINUX for BIOS clients and shim/grub for UEFI clients. Later on, we will set up dnsmasq to decide which bootloader to serve based on how the client identifies itself.

PXELINUX

For legacy BIOS clients, we are going to use PXELINUX. This is a very popular PXE bootloader based on SYSLINUX. It only supports BIOS clients, so this will only handle the BIOS half of things.

Start out by downloading SYSLINUX, which includes PXELINUX. Extract pxelinux.0 and a few libraries from it and place them into the TFTP server directory:

pxelinux.0 will be the file that is sent to BIOS PXE boot clients. After loading, it will be responsible for grabbing the other library files (ending in .c32) from the TFTP server and then it will display a list of menu choices, which we will define in a config file. Create the directory that will contain the config file:

mkdir /tftpboot/pxelinux.cfg

Now, create the file /tftpboot/pxelinux.cfg/default, which will contain the commands needed to boot into the live environment from the NFS share of the Ubuntu ISO contents. In the example below, your NFS server is 192.168.1.2:

You can add as many “LABEL, MENU LABEL, KERNEL, APPEND” groups as you want to provide multiple selectable options.

shim and grub

For modern UEFI clients, we can’t use PXELINUX. Apparently, SYSLINUX also supports UEFI netbooting, so you could use it, but I read that the current version as of this writing (6.03) is very buggy with UEFI PXE booting. Instead, let’s use grub. Using grub will also allow you to use shim and support Secure Boot if you care about that.

Download Ubuntu’s version of shim-signed, and copy it into your TFTP directory. There are many ways to obtain it, such as downloading the dpkg file and extracting shim.efi.signed out of it; here is an easier way that works as of this writing:

You can add as many “menuentry” items as you want to provide multiple selectable options.

Set up dnsmasq

We are now finished setting up the TFTP directory. All bootloaders and config files are in place. Now, we just need to set up the DHCP/TFTP server and tell it to boot BIOS clients using pxelinux.0, and UEFI clients using shim.efi. First, install dnsmasq 2.76 or greater using whatever method is most convenient for you. Based on which strategy you chose earlier, go to the subsection that matches the setup you want, and set up your dnsmasq.conf based on the given example.

Replace your existing DHCP server

Here is a (mostly) complete dnsmasq.conf file that sets up a full DHCP server. Note you will probably need to add additional options for the default gateway and DNS servers, but I’ve only supplied the parts of the config file necessary for PXE booting to work properly. Find a normal dnsmasq tutorial for info on the options to add for fully setting up a DHCP server. In the file below, it’s assumed that the server running dnsmasq is 192.168.1.2 and it is assigning IP addresses in the range of 192.168.1.50 to 192.168.1.99. Also, note that the configuration includes support for EFI32, which is a pretty uncommon architecture and will probably require a different build of grub and shim, if anyone has even made such a thing. You probably won’t have to worry about EFI32, so you can completely remove the lines referring to it if you want.

Another interesting tidbit is that this configuration doesn’t actually use PXE; it just uses the normal TFTP server and filename fields that are a part of DHCP. All PXE clients I’ve ever tested (both BIOS and UEFI) support this without any problems though. The reason I did it this way is because I discovered during testing that UEFI clients don’t seem to work properly when you actually use PXE with this setup. I never figured out why, but since it works to just set the DHCP boot server and filename, there’s no need for PXE.

# Don't function as a DNS server
port=0
# Log lots of extra information about DHCP transactions
log-dhcp
# Enable the built-in TFTP server
enable-tftp
# Set the root directory for files available via TFTP
tftp-root=/tftpboot
# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients
dhcp-no-override
# Inspect the vendor class string and match the text to set the tag
dhcp-vendorclass=BIOS,PXEClient:Arch:00000
dhcp-vendorclass=UEFI32,PXEClient:Arch:00006
dhcp-vendorclass=UEFI,PXEClient:Arch:00007
dhcp-vendorclass=UEFI64,PXEClient:Arch:00009
# Set the boot filename based on the matching tag from the vendor class (above)
dhcp-boot=pxelinux.0,,192.168.1.2
dhcp-boot=net:UEFI32,shim.efi,,192.168.1.2
dhcp-boot=net:UEFI,shim.efi,,192.168.1.2
dhcp-boot=net:UEFI64,shim.efi,,192.168.1.2
# Give out IPs from 192.168.1.50 to 192.168.1.99
dhcp-range=192.168.1.50,192.168.1.99

If for some reason you’re directly booting into grub instead of shim, replace all instances of shim.efi with grubx64.efi instead. This won’t work with Secure Boot enabled, so I’d definitely recommend using shim for maximum compatibility.

Set up a proxy DHCP server

This config file sets you up as a PXE proxy server. Like the above example, it is assumed that the dnsmasq server’s IP address is 192.168.1.2. I am also assuming that the other DHCP server is 192.168.1.1. Unlike the above example though, this example does actually use PXE, and it works properly with both BIOS and UEFI. Unfortunately, as I explained earlier in this post, shim and grub won’t play nicely because they won’t be able to automatically detect that they were loaded from a PXE proxy. Because of that problem, this configuration is mostly useless if you need to support UEFI. I’m just providing it for reference, especially to show the difference between setting up normal DHCP TFTP server/filename booting and actual PXE booting. If shim and grub are ever fixed to support PXE proxies properly in the future, this configuration will be much more useful.

The “Test PXE” and “Testing PXE” text can be set to whatever you want.

# Don't function as a DNS server
port=0
# Log lots of extra information about DHCP transactions
log-dhcp
# Enable the built-in TFTP server
enable-tftp
# Set the root directory for files available via TFTP
tftp-root=/tftpboot
# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients
dhcp-no-override
# Set the boot file and server IP based on the architecture
pxe-service=x86PC, "Test PXE", pxelinux.0, 192.168.1.2
pxe-service=BC_EFI, "Test PXE", shim.efi, 192.168.1.2
pxe-service=X86-64_EFI, "Test PXE", shim.efi, 192.168.1.2
# PXE menu. The first part is the text displayed to the user. The second is
# the timeout, in seconds.
pxe-prompt="Testing PXE", 1
# We are proxying for the DHCP server 192.168.1.1
dhcp-range=192.168.1.1,proxy

Add a second DHCP server with a different IP range (and ignore non-PXE clients)

This method should allow you to preserve your existing DHCP server while still supporting PXE. Because of the problems with shim and grub mentioned above, I’d personally recommend using this method instead of the proxy method if you don’t want to mess with your existing DHCP server. Use the same configuration from the “Replace your existing DHCP server” section, but add the following lines to the config file, just after the dhcp-no-override line:

# Set a tag if it's a PXE client
dhcp-match=set:IsPXEClient,60,"PXEClient"
# Ignore non-PXE requests, to allow the existing DHCP server to handle them instead.
dhcp-ignore=tag:!IsPXEClient

And of course, update the dhcp-range setting to make sure it doesn’t overlap with your existing DHCP server’s IP address range.

As described earlier, the dnsmasq server in this configuration will ignore non-PXE clients, so your existing DHCP server will be fully in charge of assigning IP addresses to your computers and devices, with the exception of PXE boot attempts. Both your existing server and the dnsmasq server will respond to PXE requests, but your server should “win” because the existing DHCP server won’t respond to the PXE request with any of the required PXE information. I can’t guarantee this will work with every PXE client, but it seems to work fine with every computer I’ve tried to netboot.

Run dnsmasq and try it out

At this point, you have set up dnsmasq using one of the three strategies listed above. Now it’s time to try it out! Ensure that dnsmasq will start up and run. While testing, I like to leave the service stopped and instead run it manually with the -d flag so I can see all of the debug output:

sudo service dnsmasq stop
sudo dnsmasq -d

Now, tell a computer on your network to boot! Usually your computer has a boot menu that lets you pick various boot options. Sometimes the F12 key works, sometimes the Esc key works. Or, try making a VMware virtual machine and booting it without a hard drive. In many computers, you have to enable the PXE boot ROM or the “Networking Stack” in order to make it work. There’s almost always an option somewhere! Remember, this only works over Ethernet; if you have built-in Wi-Fi, it’s probably not going to work unless you have a really fancy BIOS that knows how to connect to a Wi-Fi network — I’ve never heard of such a thing.

If everything works OK, you should end up at a menu screen served by PXELINUX on a BIOS computer or grub on a UEFI computer. Pick the Ubuntu option you added to the config file earlier, and press enter to begin booting!

Sometimes, downloading the kernel and initrd can be slow. If you see a black screen and nothing’s happening, don’t panic. Open up Wireshark and see if anything’s happening, and be patient. In some cases, such as a VMware VM set up for EFI, grub takes forever to load the kernel and initrd over TFTP and sits with a black screen for a long time. dnsmasq doesn’t actually print out anything about a TFTP file transfer in progress; it only prints out a message after the file transfer succeeds. At some point, Ubuntu will load and attempt to mount the ISO contents over NFS.

Once I know that everything is working OK, then I kill the debug dnsmasq with ctrl-C and run it as a service instead (assuming I have it installed that way):

sudo service dnsmasq start

You might want to remove some of the debug options from the dnsmasq config file such as log-dhcp once you know everything is working OK, but keep in mind if you ever need to do any troubleshooting, it will be useful to turn it back on.

After installing, fix /etc/network/interfaces

When you install Ubuntu from an NFS mount, it does something weird: it sets up /etc/network/interfaces to manage your Ethernet card instead of allowing NetworkManager to handle it. It’s really easy to fix though. After booting into your new Ubuntu install, edit /etc/network/interfaces and remove the lines relating to your Ethernet card. The only two non-comment lines left when you’re finished should be:

auto lo
iface lo inet loopback

This is just a minor annoyance that is easy to solve. It’s probably possible to customize the Ubuntu install scripts to fix this automatically, but I spent way too much time figuring out how to netboot. I don’t feel like solving that tiny problem now. Post a comment on the blog if you figure out how to do it and I’ll add instructions to this post.

All done!

Not too bad, right? I hope this post can serve as a reference tool for sysadmins, developers, power users, etc. who want to netboot install modern Ubuntu systems. Maybe I didn’t search hard enough, but I couldn’t find any reference online with all of this information in one place. Let me know if you have any suggestions on ways I can improve these instructions, or if things change as time goes on! Maybe someday, shim and grub will support PXE proxy servers and I won’t have to tell people to steer clear of that option.

Special thanks

It wouldn’t be right if I didn’t give credit to the following sites that helped me figure out how to make this work and also helped me understand why things weren’t working properly:

As someone who works with a lot of development boards, most of which come with a USB-to-serial chip of some kind, I deal with an annoying problem pretty often in Linux (currently, Ubuntu 16.04). When I plug the development board into my computer, I’m blocked from doing anything with the USB serial port it creates. The first error I usually get is that permission was denied:

$ cat /dev/ttyUSB0
cat: /dev/ttyUSB0: Permission denied

This is a common problem which is easy to solve — the problem is that the port’s permissions and ownership don’t allow me to access it:

I usually just add myself to the dialout group after I install Ubuntu to get around this issue. But…this isn’t what this blog post is about. The problem is that after I fix this, log out, log back in, and plug the board back into my computer, I run into this next problem:

$ cat /dev/ttyUSB0
cat: /dev/ttyUSB0: Device or resource busy

What’s interesting about this problem is that if I wait for 15-30 seconds and try again, it works. The problem goes away on its own. I did some sleuthing with the “lsof” command and figured out why this happens. The reason is that ModemManager opens the port when you plug it in, checking to see if a modem is attached to the serial port. I personally find this really annoying.

Luckily, it’s pretty easy to fix with a udev rule. udev already comes with several default rules which prevent ModemManager from looking at some USB serial devices (see the files /lib/udev/rules.d/77-mm-usb-device-blacklist.rules and /lib/udev/rules.d/77-mm-usb-serial-adapters-greylist.rules). Following the pattern in these files, you can create your own blacklist rule which you can put in /etc/udev/rules.d. In this example, I’m going to prevent an imaginary USB-to-serial adapter with USB vendor ID 0x1234 and product ID 0x5678 from being checked by ModemManager.

Create the file /etc/udev/rules.d/99-sample-usb-blacklist.rules and enter the following content, all on a single line:

That’s all there is to it. This rule checks for a device in the subsystem “usb” with a DEVTYPE of “usb_device” and a matching USB PID/VID combination. If it finds a matching device, it sets an environment variable called ID_MM_DEVICE_IGNORE to 1, which will tell ModemManager to ignore it. Reload the udev rules with the following command:

sudo udevadm control --reload-rules

Now unplug and replug your development board (or USB-to-serial adapter or whatever it is) and you should notice that ModemManager didn’t interfere.

Luckily, this problem is not as common today as it used to be. Modern versions of ModemManager come with a default rule that “greylists” all USB devices with the FTDI vendor ID so they are only checked if you manually scan for modems–see the 77-mm-usb-serial-adapters-greylist.rules file I mentioned earlier. Most development boards that I’ve seen use FTDI chips, so that rule takes care of them. I still occasionally run into this problem with non-FTDI chips or older versions of Ubuntu, though, and the rule pattern I gave above should fix it.

Note that the ID_MM_DEVICE_IGNORE environment variable needs to be set on the USB device, and not the tty device. In other words, your rule needs to say SUBSYSTEM==”usb” and not SUBSYSTEM==”tty”. I have spent an embarrassing amount of time trying to troubleshoot a rule that didn’t work because of this requirement. If you already have another rule that sets custom permissions on the tty device, you need to create a separate rule to add the ID_MM_DEVICE_IGNORE environment variable to the actual USB device rather than the tty device.

Recently, my dad’s TracFone Huawei H110C cell phone (with over 900 minutes and 200 service days) was a victim of mistaken identity. It was loaned out to someone else, and their dog thought it was a chew toy.

I was called to the rescue. The phone still worked, but the display was all screwed up so I couldn’t see anything. My first thought was to transfer the service (phone number, minutes, service days) to a new phone, but I discovered that you need to enter and read back codes on the old phone in order to transfer your service, which you obviously can’t do with a broken display. I also read on forums that it can be time-consuming to work with TracFone to recover your minutes from a broken/lost/stolen phone, so I didn’t really want to mess around with that.

I tore apart the old phone to try to see if it was possible to replace the display. I was able to remove the LCD display module, but I couldn’t find that particular display available for purchase anywhere online. It may have been custom-made for Huawei. A brand new H110C costs less than $10 at Wal-Mart, so I decided the easiest approach would be to buy a new H110C and remove its display, then plug the new display into the old, broken phone and hope that it would work long enough to get the minutes transferred off of it.

It worked! I was able to enter the codes needed to transfer the service to another new H110C, which I also bought at Wal-Mart. Rather than reassemble the first new phone and transfer the service to it, I decided it would make more sense to buy a second new phone because some components in the phone are glued together, and you have to sort of mangle a metal shield in the phone in order to get to the display. So this process required me to buy two new H110Cs at Wal-Mart for a total price of $19.76. Not bad! Other than setting off the alarm at Wal-Mart when I left because the employee who helped me forgot to do something to the phone boxes, buying the phones was a painless process.

I hope I can help someone else out who runs into this problem, so here’s a tutorial on removing/replacing the display in a Huawei H110C.

Get the phone out of the box

Remove the phone from the box. Leave the battery out (or if it’s already installed, remove it).

Remove the six screws from the back

There are six screws on the back of the phone. Remove them. They are Torx T5 screws, so you will need an appropriate screwdriver. One of them (right center in the picture) is covered with a white sticker as proof that you haven’t opened it. We’re going to void our warranty here anyway, so no big deal.

Separate the two halves

Removing the screws will not automatically separate the plastic pieces that hold the phone together. Carefully slide an old credit card or similar tool in between the two halves to release the latches that hold it all together. Work your way around the entire phone until the two halves are separated, and then lift the top of the case away.

Remove the button board

The buttons are on a small flexible white circuit board. It is glued (and/or taped?) to the metal shield underneath it. Carefully peel it up off the shield. There is a connector on the bottom of it that connects to the main phone circuit board below. The connector should easily disconnect as you peel the button board away. It’s difficult to not bend the board, but try to keep it as straight as possible because we will need to use it again when we turn on the old phone.

Pull the bottom half of the phone out of the plastic case

At this point you should remove the main phone board and display from the bottom case. If you push it around enough it should come out. I struggled with it a little bit, but eventually I was able to snap the circuit board out.

Remove the metal shield

In order to get to the connector that connects the display to the board, you will need to remove the metal shield. This shield is probably an EMI/RF shield blocking interference to/from the phone. There are four tabs that hold it in place–two on each side. Peel them back, and then pop the metal shield off. This may take some force, so be careful. I used a small set of needle-nose pliers. On the first phone I opened (the broken one), I mangled the shield pretty badly. I did a better job on the second one, but I wouldn’t count on necessarily putting it back together when you’re done unless you’re really desperate to only buy one new phone instead of two.

Disconnect the display from the main board

The display is connected to the main board through a small gray connector attached to a ribbon cable. It is easy to remove. Gently pry it up (I used a small screwdriver).

Remove the display

Removing the display is pretty difficult because it is also glued/taped down. I couldn’t figure out exactly what they used, but it seemed to be some sort of really sticky tape. I carefully stuck a thin screwdriver between the phone board and the display, and very carefully pried it up a little bit at a time to break the bonding. Eventually it lifted off the board. Be very careful here, especially with the brand new phone, because you don’t want to damage the new display.

Repeat this process on your broken phone

Do the same thing to remove the broken display from the old phone. As you can see, the old one was damaged by a dog bite.

Put the new display into the old phone

Basically, just do these steps in reverse. Plug the new display and one of the button boards into the old phone’s main board. I left the metal shield off because I knew I would only have the old phone turned on for a few minutes. I also covered the bottom of the button board with Kapton tape to ensure it would stay insulated from any metal underneath, although I don’t know how important that was.

Write down everything about the old phone

You will need the MEID of the old phone during the transfer process. TracFone will probably already have your old phone’s MEID on file, but just in case, write it down or take a picture of it (it’s inside the battery compartment, on the back of the main circuit board).

Turn on the old phone

Put the old phone (with new display and button board) into a case, insert a battery, and turn it on. You can press the little buttons on the button board that correspond with the phone’s keys. Cross your fingers! Here’s what I saw after a second:

Begin the transfer process

Now, go to TracFone’s website and start the process of transferring the old phone’s service to the second new (unopened) phone. The TracFone site will walk you through entering some codes on the old phone and typing the result codes into their webpage to determine the number of remaining minutes and service days.

While you have the old phone turned on, be sure to write down the old contacts and settings so you can put them into the new phone.

All done!

After that, I was instructed to dial a number on the new phone to activate it. Activation worked painlessly and the new phone worked great. At that point I was able to turn off the old phone and get rid of it. Remember, lithium-ion batteries are dangerous if they get damaged, so don’t put them in the garbage–take them somewhere that can dispose of them properly. In all honesty, the dog was probably lucky that he didn’t puncture the battery.

The tape/glue/whatever was pretty annoying, and the metal shield was a pain to remove, but other than that, this phone wasn’t very difficult to take apart. It was definitely worth the less-than-$20 price to buy a brand new phone and another new LCD donor phone in order to recover all those minutes and service days from the old phone. I hope this little tutorial helps someone out there!

I recently repaired a Macintosh IIx I bought on eBay. It wouldn’t power on at all–both the rear power button and the keyboard power button did nothing. I was already aware that this system needs fresh batteries in order to boot, and I had removed the old batteries and replaced them with new ones, which didn’t make a difference. The real culprit: all of the surface-mount aluminum electrolytic capacitors had leaked electrolyte all over the logic board. This is a very common problem in older computers. These capacitors have a nasty habit of leaking after about 20 years or so. The electrolyte ate away at traces and several traces near capacitors were broken. Several IC pins were corroded really bad as well and had lost contact to their solder pads.

It was quite a journey figuring out how to fix it. The power button didn’t do anything, so I didn’t have a clue where to look. Luckily, an important figure in the Classic Mac community, Gamba, created a schematic of the power circuit. Sadly, Gamba passed away, but his site is still operational. I think there is a small error in his schematic, so I went ahead and fixed it. The fixed schematic is depicted below. The only change is the positioning of J18-15 and TS1 on the right side:

I began looking at this schematic and realized I didn’t have a clue what was going on in any of it. All of those NAND gates, diodes, and transistors looked intimidating. I buckled down and did my best to read the schematic and figure out what every piece does. After lots of experimentation and research, I feel like I understand this circuit pretty well now. I’ve decided to write this post to help explain the circuit and maybe help people in the future who are trying to figure out why their II/IIx won’t start up. Note: I am not an electronics expert. I’m trying my best, but I could be wrong about some details. Corrections would be appreciated!

Note that the diagram’s component numbering is for the Mac II. The IIx is slightly different. I would provide more detail, except my IIx logic board is a prerelease one that has the same component numbering as the II. So my IIx’s logic board’s component numbering matched this diagram. Other IIx systems do not match.

Let’s get started!

Overview

Here’s a broad overview of the different pieces of the circuit:

The ADB portion of this circuit is unrelated to this discussion, so I will skip it. Don’t worry–I have included the keyboard power button as part of my description of the power on circuit.

When I first looked at this diagram, for some reason I thought the top half was for the keyboard and the bottom half was for the rear power button. I was totally wrong. The keyboard and rear power button share most of the same power on circuitry. Now, I’m going to cover each section in more detail.

Batteries

The two batteries B1 and B2 (3.6V 1/2 AA lithium) are in series with each other. This means they combine together to create a 7.2V voltage across ground and the positive terminal of B1. The three diodes seem to serve two purposes:

Drop the voltage a little bit

Protect against reverse polarity (putting the batteries in backwards)

In particular, D5 and D6 take the 7.2V voltage from the batteries and drop it down to about 7V or so. Normally these diodes would have a higher voltage drop, but you will see that while sitting idle, there isn’t much current being drawn from them, so the diode voltage drop is pretty low. D4 drops the 3.6V voltage a little bit, but I think its main purpose is to protect against reverse polarity. If you put B1 and B2 in backwards, D4 and D5 will prevent negative voltages from making their way to any other part of the logic board.

The 1000 μF capacitor is (I believe) the capacitor on the battery replacement board, and it can be found on both the II and the IIx, despite what Gamba’s diagram says. These computers originally came with the two batteries soldered directly to the logic board. When they needed to be replaced, there was a little circuit board that a service provider would solder on to replace the original batteries. It had two battery holders and a 1000 μF capacitor. I don’t fully understand the purpose of the capacitor, but it might be to make life easier on the batteries when the power button is pressed.

So this portion of the circuit takes two batteries as input, and as output provides a 3.3V-ish voltage rail and a 7V-ish voltage rail. I say “-ish” because I measured anywhere from 6.8V to 7V when I was debugging this circuit. But it’s somewhere in that ballpark. The 3.3V voltage is for powering the PRAM and clock while the computer is turned off. The 7V voltage is for turning the power supply on when you press either of the computer’s power buttons.

If you’re troubleshooting this portion of the circuit, make sure the cathode of D6 has 7V or so, and the cathode of D4 has 3.3V or so. If not, there’s something wrong with some of the traces and/or components pictured above. Check continuity between everything in the diagram above, and if needed, test D4, D5, and D6.

Clock/PRAM

This portion of the circuit is responsible for powering the clock and PRAM. On most (all?) of the Macs that came after the II/IIx, this is the *only* thing the single 3.6V battery is used for, at least as far as I know.

Recall that I already explained that there is about 3.3V on the cathode of D4 coming from battery B2. The 4.7KΩ resistor R17 is limiting the amount of current that will be supplied to UB6, the clock/PRAM chip. So…when the computer is turned off, the battery B2 is powering UB6. D3 is preventing the battery voltage from going into the +5V rail (which will be 0V when the power supply is turned off).

When the power supply turns on, the +5V rail actually has 5V on it. D3 will allow the 5V to go through into UB6, so the +5V rail will power UB6. I believe the 4.7KΩ resistor (R17) makes it so that the 5V rail does most of the work powering UB6 instead of the battery when the power is on. D4 is also (I believe) preventing the circuit from sending +5V into the battery circuit when the computer is powered on.

This part of the circuit is pretty simple! I don’t think a problem in here would necessarily prevent the computer from turning on, but it might prevent settings from being remembered while the power is off.

Power On

OK, here is where things really get interesting and more complicated. This is also where it will become obvious that both batteries must be installed in order for the II/IIx to boot. Recall that the cathode of D6 has about 7V on it or so. It powers UB2 (which is a 74HC132 containing four NAND gates–three are used, one is unused). UB2 doesn’t draw much current, so that’s why the diodes mentioned in the battery portion of the circuit don’t have much of a voltage drop. Without the batteries, these NAND gates will not be powered at all, and therefore the power circuit won’t work. It’s a good thing UB2 doesn’t draw much current, because otherwise the batteries would run out of juice too quickly.

Supplying 7V to the 74HC132 is a bit risky — it’s within the allowed range, but the datasheet I have recommends a maximum of 6V. Whatever–it seems to work.

The 7V rail also goes to the emitter of transistor Q3, which is a PNP transistor. We’ll see in a minute, but when the power button is not being pressed, the transistor is turned off, so it shouldn’t be drawing any current.

Additionally, the 7V rail is directly connected to one of the inputs on each of the three NAND gates. In effect, it means each one of these NAND gates is really acting as an inverter (e.g. a logic 0 at the other input becomes a logic 1 at the output, and a logic 1 becomes a logic 0). This is because:

1 NAND 1 = 0

0 NAND 1 = 1

I’ll go into this in a bit more detail when I describe what happens when you press the power button.

Finally, the 7V rail goes through the 100KΩ resistor R18 to the second input of the leftmost NAND gate. This is a pull-up resistor that keeps the second input of the NAND gate pulled up to a logic 1 by default. The 100Ω resistor R3 is also connected to this part of the circuit, but the other end of it is floating when both power buttons are not being pressed, so it doesn’t really affect the circuit until a power button gets pressed.

So in the default state when the system is plugged in but no power button has been pressed:

The leftmost HC132 NAND gate has both inputs at 7V. This means its output will be 0V.

The output of that first NAND gate is fed into the other two NAND gates.

Each of those NAND gates has one input at 0V, and one input directly wired to 7V, so they are both outputting 7V.

Since 7V is going into the left side of R14, there is no current between the emitter and base of the transistor. The transistor is turned off.

Because the transistor is turned off, the power supply control pin (J18, pin 15) is left floating (remember, the +5V output is not supplying 5V yet, since the power supply is off).

Thus, the power supply stays off.

When the keyboard power button is pressed, the left side of R3 is grounded. This pulls pin 2 of the HC132 NAND gate to ground through the 100Ω resistor R3. So:

The leftmost HC132 NAND gate has one input at 0V, and one input directly wired to 7V.

The output of that NAND gate is thus 7V, which is fed into the other two NAND gates.

Each of these NAND gates has 7V going into both input pins, so they are both outputting 0V.

Since 0V is going into the left side of R14, current is flowing from the emitter of the transistor to the base of the transistor. The transistor is turned on and allowing current to flow from the emitter to the collector.

Because the transistor is turned on, the power supply control pin is driven to 7V or so (just assume that TS1 is a short circuit for now–it will be described in more detail in the power off circuit description)

Thus, the power supply turns itself on.

As soon as the power supply turns itself on, it begins outputting 5V. 5V goes through D2 and R10, pulling the power supply control pin up to +5V (minus the diode drop) through the 220Ω resistor. As soon as you release the power button, this mechanism keeps the power supply control pin high. Otherwise, the power supply would immediately turn itself back off.

The rear power switch does essentially the same thing as the keyboard power switch. When you press it:

It completes a circuit to allow the battery to begin charging up capacitor C3, which is discharged when this process begins.

The left side of R3 starts at 0V and slowly starts raising up as C3 charges.

Since C3 starts at 0V, the same steps are followed as when the keyboard power button was pressed, and the power supply turns on.

After the power supply turns on, 5V is available through R9, so C3 will charge up to 5V.

C3 charging up to 5V is an important detail. If the button is later used for turning the computer off, the rear power switch won’t try to turn on Q3 at the same time it tries to turn the power supply off. It also gives you some time to release the power button before the system tries to turn itself back on again.

Wow! I may try to add some more diagrams in the future to depict it a bit more clearly. It’s a lot to digest.

You may be wondering: what is the point of the three NAND gates? At first glance, all they are effectively doing is repeating the input value that came out of R3. The purpose seems to be to allow better current drive capability for controlling the transistor, and a clean output that ensures that the signal controlling Q3 is either on or off, and not somewhere in between as capacitors charge up.

The three NAND gates essentially combine to create a buffer with double the current drive capability of a single NAND gate’s output. As long as all of the NAND gate pins are connected together and the NAND gates are working, you can assume that a low value at pin 2 of UB2 will turn the transistor on, and a high value at pin 2 will turn the transistor off.

Power Off

Just like the power on part of the circuit, the power off part of the circuit uses NAND gates that make it look way more complicated than it really is. These NAND gates are powered by the power supply’s +5V line. The two leftmost NAND gates form an SR latch. The SR latch’s top “set” input is connected to UB11, pin 13 (and the power button’s second pole). I believe this line is pulled up to +5V by a pull-up resistor inside the 6522 VIA (UB11), keeping it high (deasserted) while the computer is running. UB11 pin 13 is the /POWEROFF signal. At startup, C5 is fully discharged to 0 volts, so the SR latch’s “reset” input is low (asserted), which sets the SR latch’s stored “Q” value to 0. Then, C5 charges up, so the “reset” input goes high (deasserts itself) and stays deasserted while the computer is powered on. So the entire time the computer is running, the SR latch’s “Q” value is 0.

The !Q output of the SR latch is connected to the next two NAND gates, which act as an inverter just like in the power on circuit as I described earlier. So the output of the dual-NAND-gate inverter connected to R15 is really just the noninverted Q output of the SR latch, but with twice the maximum current. Thus, the voltage on the left side of R15 is 0V while the computer is on, keeping the NPN transistor Q2 turned off. Remember that the power supply control pin (J18, pin 15) is being pulled up to +5V in the power on circuit, keeping the power supply turned on.

Let me explain TS1 now. TS1 is a temperature switch. During normal operation, it acts like a short circuit. If the temperature gets too hot, it becomes an open circuit. If it becomes an open circuit, the power supply control pin is no longer pulled up to +5V, so the power supply shuts off (I’m pretty sure the power supply has a weak internal pulldown on that pin). During normal operation, TS1 will act like a wire.

When you tell the computer to shut down, the computer drives UB11 pin 13 (/POWEROFF) low. This causes the SR latch’s “set” input to go low, so it’s now asserted. This makes the !Q output low, which makes the inverter output high, so Q2 turns on, connecting R11 to ground. Now there is a 27Ω pulldown pulling the power supply control pin low, and a 220Ω pullup pulling the power supply control pin high (in the “power on” portion of the circuit). This is a voltage divider, so the voltage at the power supply control pin will be approximately 0.5V. This is low enough to tell the power supply to turn off.

If you press the power button on the back of the computer, the same thing happens — it connects the SR latch’s top “set” input to ground, and the exact same thing happens.

Troubleshooting

OK, so I have tried my best to explain what’s going on in each portion of the circuit. If you have a broken power circuit, how do you fix it?

The system will not power on

First of all, there are a few traces that have a tendency to go bad because they run nearby leaky capacitors:

Of course, make sure you have removed, cleaned around, and replaced all of the electrolytic capacitors on the logic board.

Make sure that all of the connections between R3, UB2, R14, and Q3 are OK. If you can, use a voltmeter to verify that the NAND gates are operating as expected when you press either of the power buttons. You can do this with the logic board out of the computer as long as the batteries are installed.

Check the power supply control pin (J18, pin 15) to see if it’s correctly getting a voltage when you press either of the power buttons. If it is, there’s a chance that the power supply is at fault.

UB2 is somewhat near a capacitor. There’s a good chance that some of its pins have lost connectivity due to corrosion. Check every pin on UB2 and make sure it has connectivity as shown in the schematic.

The keyboard power button works, but the rear power button doesn’t

There are only a few parts of the circuit that could cause this. Make sure R9, C3, the rear power button, R3, and the keyboard power button pin of L2 are all connected as shown in the schematic diagram. For the rear power button to not work when the keyboard does, one of the connections between those components has to be broken.

The system powers on, but it won’t power off

UB1 is extremely close to a capacitor. There’s an extremely good chance that it has lost some connectivity due to corrosion. Verify all connections.

See if the output of UB1 pin 8/11 is correctly going high when you press the rear power button while the system is on. If not, follow the circuit and see where the broken trace is.

Final thoughts

I thought this would be a good idea to explain the circuit in more detail. This has turned into quite a long-winded explanation, and it doesn’t have quite as many pictures as I would have hoped. Maybe this would be better as a video or an animation that shows what the circuit is doing. For now, I hope this is enough to help someone out there troubleshoot their Macintosh II/IIx power circuit.

In my particular case, a trace was broken in the R3-R18-C6-UB2 group of pins that should be connected together, and UB1 was completely covered in capacitor leakage, which had very badly corroded its pins and solder pads. I ended up replacing both UB1 and UB2 to be safe. A trace between a via and C5’s positive end was also broken and needed a patch wire. I had to run several patch wires to repair broken traces:

By the way, this wasn’t the only problem with my IIx. This was actually the last part of the logic board that I fixed. The whole power on circuit can be bypassed by tapping a 3.6V battery between ground and the power supply control pin, so I did that for testing until I repaired this circuit. The power supply would turn on, but nothing would happen. It turns out that there were a ton of other traces that were corroded. The 53C80 SCSI chip (again, near a capacitor) was very badly corroded, so I removed it, cleaned the solder pads, and replaced it with a new one. One of the solder pads had a break from its trace, so I had to run a patch wire. I also found several other broken traces on solder pads near two chips on the board.

When I wrote Parallel Port Tester, I discovered that detecting parallel ports and their I/O addresses in Windows is a pain in the butt. I have received several inquiries asking about how I did it, so it’s obvious that others agree with me. Hopefully, this post will help everyone learn how to automatically detect their parallel port addresses. I certainly don’t claim to be an expert, and there may be better ways to do it, but this method seems to work for me.

Don’t use Win32_ParallelPort

One of the strategies I’ve seen recommended is to do a WMI query on the Win32_ParallelPort class. This is originally how I started implementing Parallel Port Tester, but I quickly discovered that it sucks! Sometimes it gets the I/O address for the wrong port, and sometimes it doesn’t even find your parallel port at all. It seems to depend on the version of Windows and the chipset manufacturer. In particular, I found that it would not detect a parallel port implemented by a Moschip chipset in Windows XP. It also was returning incorrect I/O addresses on a user’s system running Vista. Anyway, I’d highly recommend that you avoid this class. I could not get it to work correctly. Maybe I was doing something wrong…but I’m pretty sure it just sucks. It sucks so bad that I disabled this method of discovering parallel ports in version 1.0.0.1 because of a report that it was returning erroneous information.

Instead, find objects belonging to the Ports class manually

For more reliable results, what I do is search for all devices that are ports (which also include COM ports). The GUID for this class is {4D36E978-E325-11CE-BFC1-08002BE10318}. I only pay attention to returned ports that are LPT ports. On each found LPT port, I request all of the port resources to determine the I/O addresses.

How do you do that? Here’s some sample C# code using .NET. Note that I’m only showing relevant bits and pieces, so you will have to fit them into a class and function as appropriate.

using System.Management;
private static readonly ushort SPPIOLength = 8;
private static readonly ushort ECPIOLength = 4;
SelectQuery portsQuery = new SelectQuery("Win32_PnPEntity",
"ClassGUID=\"{4D36E978-E325-11CE-BFC1-08002BE10318}\"");
ManagementObjectSearcher portSearcher = new ManagementObjectSearcher(portsQuery);
foreach (ManagementObject port in portSearcher.Get())
{
// Grab its name. This will be in the form "Blah Blah Parallel Port (LPT1)".
// We're going to get ugly now...
string name = (string)port.Properties["Name"].Value;
// Skip ports that are not parallel ports. This is UGLY...I'd much rather
// do this a cleaner way, but it seems that different parallel ports do
// things differently.
if (!name.Contains("(LPT")) { continue; }
// and extract the parallel port name by looking for everything inside
// the parentheses. I don't know how else to do it...
int beginLoc = name.IndexOf("(LPT") + 1;
int endLoc = name.IndexOf(')', beginLoc);
name = name.Substring(beginLoc, endLoc - beginLoc);
// Grab the device ID so we can find all the associated port resources.
string deviceID = (string)port.Properties["DeviceID"].Value;
// Replace single backslashes with double backslashes to be suitable
// to use in a query. Note that I also had to escape it myself in
// code...so \\ represents a single backslash.
// TODO: Any other escaping necessary?
deviceID = deviceID.Replace("\\", "\\\\");
// Now search for I/O ranges of this port.
ObjectQuery resourceQuery =
new ObjectQuery("ASSOCIATORS OF {Win32_PnPEntity.DeviceID=\"" +
deviceID + "\"} WHERE ResultClass = Win32_PortResource");
ManagementObjectSearcher resourceSearcher =
new ManagementObjectSearcher(resourceQuery);
// Find the SPP and ECP base addresses
ushort SPPBase = 0xFFFF;
ushort ECPBase = 0xFFFF;
foreach (ManagementObject resource in resourceSearcher.Get())
{
// Grab starting & ending addresses
ushort startAddress =
(ushort)(UInt64)resource.Properties["StartingAddress"].Value;
ushort endAddress =
(ushort)(UInt64)resource.Properties["EndingAddress"].Value;
ushort rangeLen = (ushort)(endAddress - startAddress + 1);
// If we haven't yet found the SPP base address, and this range
// is big enough, use it as the SPP base address.
if ((SPPBase == 0xFFFF) && (rangeLen >= SPPIOLength))
{
SPPBase = startAddress;
}
// If we haven't yet found the ECP base address, and this range
// is big enough, use it as the ECP base address
else if ((ECPBase == 0xFFFF) && (rangeLen >= ECPIOLength))
{
ECPBase = startAddress;
}
}
// Although I didn't include the code here, if I fail to detect an
// SPP address with the rules above, I grab the first returned
// address and use it as the SPP base address. Then I set the ECP
// address to 0xFFFF (which I treat as meaning "unable to find")
// TODO: If SPPBase != 0xFFFF, add it to the list of discovered
// ports -- the name of the port is stored in the variable "name"
}

I hope this code is helpful to someone out there! From this point, you can use something like Inpout32.dll to access the port registers directly.

I’m not a huge fan of the hack where I extract the “LPT#” name from the long name by searching for the “(LPT” string, but I couldn’t find a better way to do it. Suggestions would be appreciated! There also may be a better way to use parameters to do the ASSOCIATORS query instead of ugly escaping and string concatenation, but it worked and that was good enough for me on this particular project. I couldn’t find an easy way to set it up for use with parameters or else I would have done it that way.

I bet you thought I gave up on these lessons! In my last post about microcontrollers, I introduced the idea of an interrupt-safe circular buffer. I briefly mentioned that circular buffers are commonly used in software drivers for peripherals that send and/or receive a lot of data. One example in particular that I gave was a UART (universal asynchronous receiver/transmitter). I’d like to expand further on UARTs in this post.

How they work

First of all, what is a UART? Well, let’s break down the acronym:

Universal: The device can be configured for different modes/speeds.

Asynchronous: The communication does not use a separate clock wire for synchronization; it’s all handled only with a single data wire in each direction.

Receiver: It can receive data…

Transmitter: …and it can transmit data.

That’s great, but it still doesn’t really explain anything about what the UART does. It’s easier to describe a UART by talking about a computer serial port. If you’re geeky like me, I’m sure you’ve dealt with serial ports before. For example, if you’re a Raspberry Pi aficionado, you might have a USB-to-TTL serial adapter that you hook up to the Pi to access its Linux console. On your computer, you use a program like PuTTY or minicom to open the port for 115,200 baud, 8 data bits, no parity, and 1 stop bit. Then, you can turn on the Pi and see all of the messages it sends at boot. After the operating system is fully booted, you can also enter commands on the console.

If you have done this, you have actually worked with a UART. A UART is this whole interface I just described. You configure it for a baud rate as well as various framing parameters (7-bit data? 8-bit data? 1 stop bit? 2 stop bits? parity bits?). Then you send it bytes. It turns each byte into a sequence of 1s and 0s with a prepended start bit and an appended stop bit or two (there’s also an optional parity bit that can be added to the mix to try to detect errors). Finally, it sends this final sequence of bits onto a transmit pin at the configured bit rate. It also listens on a receive pin for data coming in with that kind of format and parses the serial bits back out into parallel bytes.

On the hardware side of things, let’s go through an example of sending out an uppercase ‘M’ with a UART configured in 8N1 mode. This means there are 8 bits per byte, no parity bit, and 1 stop bit. We start out by telling the UART to send the character ‘M’ which is represented as 0x4D according to an ASCII table. 0x4D in binary is “01001101”. A 1 represents a high state and a 0 represents a low state. The start bit is always a 0 and the stop bit is always a 1.

If you transmitted the character ‘M’ and looked at it on an oscilloscope, this is what you’d see. While the UART is inactive, it keeps the pin at a high state. Thus, the start bit is low so you can detect the start of a transmission. After the stop bit, if there is no more data to be sent, the pin stays high until the next start bit begins.

Hopefully this diagram makes sense. You’ll see that the bits are in reverse order because they are sent least-significant first. The horizontal unit on this diagram is time; the vertical lines are each about 8.7 microseconds apart, assuming the baud rate is set to 115200. I got that by calculating the reciprocal of the bit rate. If the bit rate is 115,200 bits per second, then there are (1/115200) seconds per bit.

The receiver looks for this pattern and interprets the bits between the start and stop bit. That’s really all there is to it! UARTs were very common on computers of the past generations. Earlier computers used a separate chip just for the UARTs, like the 8250 or the 16550A. In fact, many UARTs to this day remain software-compatible with the 16550A. Later on, computers began including a Super I/O chip with integrated 16550A-compatible UARTs.

How to use them

In microcontrollers, you typically find a UART (or multiple UARTs) available as memory-mapped peripherals. Sometimes they use the same interface as the 16550A, and sometimes not. For simplicity I’m going to talk about an AVR’s UART in this lesson instead of the 16550A. The 16550A is complicated because it has a couple of built-in 16-byte FIFO buffers for transmitting and receiving along with a fairly strange interrupt setup because of the FIFOs. The FIFOs are important for normal desktop computers because if your computer had to deal with an interrupt on every character, it would get bogged down handling the interrupts. The FIFO allows interrupts to occur less frequently. On a simple microcontroller like an AVR, handling an interrupt on every character is not a huge problem. Because of the lack of a FIFO, the AVR’s software interface will be easier to understand, which makes it appropriate for this introductory lesson.

I’m going to be using the AT90USB646 as an example. This is the same processor that I use on my Mac ROM SIMM programmer. It has a single USART. Notice the extra S? USART stands for Universal Synchronous/Asynchronous Receiver/Transmitter. It just adds the ability to work in a synchronous mode with an external clock pin. For our purposes, we won’t worry about this. We’ll be using it in asynchronous mode.

The AT90USB646 reference manual says that this chip has a single USART, described in chapter 18. It provides some sample code in both assembly and C for talking to the UART. It’s really not that hard to do.

Setting the baud rate

The baud rate is set with two registers: UBRR1L and UBRR1H. They are the low and high bytes of the full register UBRR1 (which you can also access directly as a 16-bit word, at least in AVR-GCC and AVR Libc). The equation, as per the datasheet, is:

UBRR1 = (oscillator_frequency / (16*baud_rate)) – 1;

So let’s use my SIMM programmer board as an example. It has a 16 MHz crystal, and let’s pretend we’re going for a baud rate of 9600. Thus, I should do:

UBRR1 = (16000000UL / (16*9600UL)) - 1;

Note that I used UL at the end of these constants to ensure they’re being treated as unsigned long integers. Otherwise, the intermediate calculations might get messed up due to only being promoted to 16-bit values. Your development environment will probably provide a define for the clock rate. Plus, it’s usually a good idea to create a #define for your baud rate instead of hardcoding a 9600UL randomly in the code. So it’s actually better to do this:

#define MY_BAUD_RATE 9600UL
UBRR1 = (F_CPU / (16*MY_BAUD_RATE)) - 1;

If you do the math (and integer truncation) yourself, you will discover that this will assign UBRR1 a final value of 103. Mathematically, it would be 103.166666666…, but it will get truncated down to an integer. This also means the baud rate won’t be exactly 9600 baud. If we solve the equation above for baud_rate and put our calculated/truncated UBRR1 value of 103 back in, you’ll see that we don’t get exactly 9600 baud:

baud_rate = oscillator_frequency / (16*(UBRR1 + 1));

baud_rate = 16000000 / (16*(103 + 1));

This gives us a baud rate of 9615. The AVR will actually be transmitting and receiving at 9615 baud. That’s an error of about 0.16%, which is definitely close enough to 9600 baud that it won’t be a problem. Usually you’re in good shape if you’re within a percent or two, and we are much closer than that. Keep in mind that oscillators aren’t perfect either, so UARTs are tolerant of baud rates that aren’t exactly correct.

Configuring the framing

We have to make sure that the USART knows what type of data to send and receive. Should there be 5 data bits? 8 data bits? 1 stop bit? 2 stop bits? Parity? No parity? All of this can be configured in some of the registers in the USART. The registers are called UCSR1A, UCSR1B, and UCSR1C. UCSR is short for “USART Control and Status Register”. These registers contain bits for controlling the setup of the USART as well as some bits that signal status of the USART.

The bits that are important for us to configure in order to get basic functionality are the following:

U2X1 bit (bit 1) of UCSR1A. This is a double-speed bit that essentially turns the constant “16” in the baud rate calculation formula above into an “8” — we should make sure it’s disabled since we assumed in the baud rate calculation we did earlier that it was in fact 16.

RXEN1 bit (bit 4) of UCSR1B. This bit enables the receiver so we can receive incoming bytes.

TXEN1 bit (bit 3) of UCSR1B. This bit enables the transmitter so we can send outgoing bytes.

UCSZ12 bit (bit 2) of UCSR1B and UCSZ11, UCSZ10 bits (bits 2 and 1) of UCSR1C. These three bits combine together to determine the number of data bits (5 to 9).

0 –> 5 bits

1 –> 6 bits

2 –> 7 bits

3 –> 8 bits (what we want)

4, 5, 6 reserved

7 –> 9 bits

UMSEL11, UMSEL10 bits (bits 7 and 6) of UCSR1C. These bits determine the mode (synchronous/asynchronous) of the USART.

0 –> asynchronous (what we want)

1 –> synchronous

2 –> reserved

3 –> master SPI (this USART is capable of being an SPI master; I have discussed SPI in the past)

Transmitting data

After you have configured the USART correctly, it’s very easy to send and receive. To send data, you simply write the character you wish to send into the UDR1 register. Here’s an example of transmitting the character ‘A’:

UDR1 = 'A';

You can only transmit if the hardware has finished transmitting the last character. This is because the USART does not have a transmit FIFO. Because of this, you need to check a status bit to ensure the transmitter is not already busy:

if (UCSR1A & (1 << UDRE1)) {
UDR1 = 'A';
}

This will only transmit a character if the transmit buffer is empty. Otherwise, it won’t transmit. If you need to make sure the character is transmitted, you can wait until the bit goes high:

while (!(UCSR1A & (1 << UDRE1))) {
}
UDR1 = 'A';

So you do an empty loop until the bit goes high, after which it’s safe to write to the transmit buffer.

Receiving data

Receiving is very, very similar. First, you need to know if a character is ready to be received. The hardware will tell you if a character is waiting to be read by making a bit high in the UCSR1A register. If the bit is high, you can read the character by reading the UDR1 register:

if (UCSR1A & (1 << RXC1)) {
received_char = UDR1;
// do something with the received character
}

The receiver actually has a FIFO (first-in, first-out) buffer to hold a couple of received characters in case your program is unable to read the first character received before another one arrives. This is a nice little safety mechanism to have, but you still need to read received characters quickly before the small FIFO fills up. Fancier UARTs have a bigger receive buffer to enable high data rates with less processor utilization.

Example code

Here’s a sample program that will echo back any character that is received. If you type the letter ‘Z’, it will transmit the letter ‘Z’ right back to you.

It’s really not that complicated. It waits until a character is received, and then receives it. Then it waits until the transmitter is empty, and transmits the received character. Note that I put semicolons at the end of the while() loops waiting for the bits. This was done intentionally to save space over giving them an empty loop body, but it can be a little confusing to read, so I wanted to explicitly point that out.

Anyway, this whole thing is easy, right?

Interrupts

There’s a catch. If your program has to do anything other than sit in a loop waiting for a character to arrive over the UART, this whole idea starts to fall apart. If you don’t check for received data often enough, you will run the risk of losing received characters. This could happen if the receive FIFO is filled before you get around to checking the UART again. How do we solve this dilemma?

You may have been wondering why I mentioned circular buffers at the beginning of this post. What do circular buffers have to do with anything I have mentioned so far? Good news: you’ve finally reached the point in this post where everything will start to come together and make sense.

The solution to this problem is to handle the UART with an interrupt. Whenever a character is received, the UART will fire an interrupt. This will cause your main program to temporarily jump to an interrupt handler which reads the received character from the UDR1 register and stores it in a circular buffer. Somewhere in the main loop, you can occasionally check the circular buffer to see if it contains any characters and process them. As long as the buffer is big enough so that you will check it before it fills up, it will solve the problem.

On the other end, you can also create a separate circular buffer for transmitting over the UART. This will allow you to put the data into the circular buffer and then move on to other places in your program while the data is sent in the background by your interrupt handler. Basically, it prevents you from having to sit in a loop and wait for all of your characters to be transmitted before you can move onto other tasks.

This a perfect application for a circular buffer. It’s easy to make a circular buffer interrupt-safe as long as you insert from the main loop and remove from the interrupt handler (or vice versa), and inserting and removing from it are both O(1) operations. This is exactly what we need for both the transmitting and receiving end of things.

The AVR’s USART is really handy because it has what’s known as the UDRE interrupt, which stands for USART data register empty. This interrupt fires anytime the transmit data register is ready for you to write another character. It’s based on the same status bit we checked in the non-interrupt-driven code above. So if the USART is idle, this interrupt will fire. Because of this interrupt, there are no special cases when you first start transmitting characters. When you put a character in the TX buffer, you have to ensure that the UDRE interrupt is enabled. In your UDRE interrupt handler, if the TX buffer is empty, you disable the UDRE interrupt. This logic is very simple compared to other UARTs, which often require special cases when you transmit your first character in order to get the interrupts rolling.

Example code

Here’s some sample code I wrote for interrupt-driven UART functionality. It consists of a header file and source file for the UART driver. You will notice the ring buffer functions which I renamed and borrowed from my post about ring buffers. Many AVR microcontrollers have multiple USARTs. This particular one works with USART1 on the AT90USB646.

This particular sample program doesn’t really take advantage of the fact that the code is interrupt-driven, but at least it tests the code out. The interrupt-driven aspect of the code shines when you need to send a 50-byte string and don’t want to wait around until it finishes sending. With this code, if you quickly write 50 bytes, the program will continue doing other things while the 50 bytes are sent in the background as interrupts come in. For receiving, you can let your program do many things without worrying about losing a received character. Every time through your main loop, you should process all characters that are waiting in the receive buffer to ensure that it doesn’t fill up.

Other functions

You could easily add other functions for transmitting a string, receiving a full line, etc. It would probably be nice for the transmit and receive functions to have optional parameters to tell them not to block if the buffers aren’t ready. These are all excellent exercises I leave to you, the reader, to implement.

Conclusion

I hope this explains the UART/USART in enough detail to get you started. This is a very important peripheral that is commonly used for interfacing with displays, sensors, and other microcontrollers. If you’re going to use a UART/USART, I would highly recommend that you do it with interrupts as I described above. This is definitely the type of thing you should stick into your toolbox of peripheral drivers and reuse on all of your projects.