Twitter Watcher, the #twatch using PIC18F67J60 microcontroller

The #twatch scrolls the latest trending topics from Twitter on an LCD screen. It’s a stand-alone network appliance that stays updated without a PC. It was awesome to watch #iranelection, Michael Jackson, and other historic events scroll by while we developed the #twatch. This Instructable documents the #twatch hardware and design.

In addition to a Twitter trend ticker, the #twatch is also a generic ethernet LCD backpack. It’ll show playlists, PC stats, and other info with programs like LCD Smartie. It’s also software upgradable, so it’s never outdated.

The #twatch is open source, so you can download our designs and build your own.

See this article with its original formatting at DangerousPrototypes.com, more discussion in the #twatch forum. We’ll send a free #twatch PCB if you’re the first to tweet the #twatch!

Concept overview

The #twatch grabs the latest trending topics from Twitter, then loads a few tweets for each. The trending topics and tweets scroll on a screen. The #twatch grabs fresh trends and tweets every five minutes so you always see the latest trending topics.

We also added a generic ethernet backpack mode, so the #twatch can also show PC stats from a program like LCD Smartie, more about this feature in part 2.

Step 1

Using it

The #twatch is easy to use.

Connect it to a home ethernet network with Internet access. The #twatch requires automatic network configuration (DHCP), this is the default setting on almost every modern home network.

Power it up. The #twatch requires a 6-7volt DC power supply. It uses a 2.1mm DC power plug, the most common type. Universal DC power supplies should include a 2.1mm plug.

Adjust the contrast. LCD screens change with temperature and age, use the adjustment screw to tweak the screen contrast.

The #twatch will configure the network settings and begin scrolling the latest trending topics and a few tweets from each. You may need to adjust the contrast again for the cleanest scrolling effect.

The Microchip PIC 18F67J60 is perfect for this project because it combines an ethernet network interface and a 41MHz microcontroller (10MIPs) in a small package for only a few dollars. It only comes in 64pin+ TQFP packages, but we didn’t have a problem soldering it by hand to a professional PCB.

The PIC requires a 3.3volt power supply. The ethernet portion is really power hungry, so we used a giant TO-220 LD117-3.3volt regulator (VR1). We chose a big regulator because it might have to dissipate a bunch of heat depending on the input power supply. The regulator requires a small input decoupling capacitor (C15) and a large 10uF output capacitor (C3).

There’s a big catch with these chips: they can only be programmed about 100 times. That makes development difficult, so we also designed a development version of the #twatch based on a different chip. More about that design in a future article.

Each PIC power pin gets a 0.1uF decoupling capacitor (C17-C23). The PIC has an internal 2.5volt regulator for the microcontroller and ethernet cores, the regulator requires a 10uF tantalum capacitor (C1).

The PIC is programmed through a 5pin ICSP header. The MCLR reset pin is held high with a 10K pull-up resistor (R21), an additional resistor (R4) and capacitor (C16) recommended by the datasheet provide protection against various accidental reset conditions.

We used an ethernet jack with integrated magnetics (J2). The jack is a HanRun HR911105A, provided to us by Seeed Studio – be sure to get the same jack, a compatible jack, or adjust the PCB for a jack you can find. The ethernet interface requires a termination circuit (R30-33, C10-11, L1) and 2.28Kohm 1% bias resistor (R7, not shown).

HD44780 character LCD

The #twatch supports a ‘standard’ 4line by 20character 5volt HD44780 LCD with a +5volt backlight. You can usually find them for about $10 on eBay. Be sure to verify that your LCD matches the #twatch pinout before attaching it. Most LCDs are the same, but not all of them.

Almost all character LCDs operate at 5volts, so we provide a 5volt power supply from a common 7805 regulator (VR2, C14, C2). The LCD with backlight could potentially use a bunch of current, so we used another large To-220 regulator. C12 is a decoupling capacitor for the LCD power supply, but LCDs already have on-board decoupling. C12 need not be populated, we just included it in case of stability issues.

For maximum refresh speed, the LCD is controlled through the full 8bit interface.

Most LCDs are 5volt parts that require about 4.25volts+ to register a high level on the data pins, but the PIC 18F65J60 is only a 3.3volt part. Fortunately, the PIC has a bunch of 5volt tolerant pins so we can hold the signal at 5volts with a 10K pull-up resistor (R10-R19), and then ground it by changing the PIC pin direction setting. This is usually called an open drain output.

Some newer LCDs run at 5volts, but still work at 3.3volt interface levels. The #twatch will support this mode if you leave out R10-19 so no pull-up voltage goes onto the pins, and change the firmware to switch the LAT register instead of the TRIS register in HD44780.c.

LCD screen contrast is controlled by a bias voltage, usually generated with a 10Kohm potentiometer. The #twatch PCB has footprints for an inexpensive 3mm SMD pot (R2), and a second space to use a larger, through-hole pot (R2A). Only one should be populated!

Just in case there’s noise in the power supply from all the ethernet stuff, we filter the bias voltage through a small ferrite bead (L2). We also included a capacitor for additional filtering(C13), but we didn’t use it since neither element is actually needed.

The #twatch can control simple +5volt backlights up to 400mA or so. The PIC switches a transistor (NPN1) through a 240ohm current limiting resistor (R3, not shown). We used a transistor that can handle 800mA+ with a gain of 250hfe+, so the PIC can switch a large load with it’s 20mA maximum pin output current.

R1 is a current limiting resistor for the LCD backlight, if needed. We used a through-hole resistor so it can dissipate a lot of heat with big backlights, and because it’s the easiest size to find locally and solder yourself. If your backlight doesn’t require a resistor, just replace R1 with a piece of wire. Our LCD required a 3ohm resistor for a 240mA backlight power supply.

Some backlights use a lot of power, so we put the supply pins right next to the power supply and reinforced the ground plane with a bunch of VIAs. Some fancy LCD backlights require special drive circuits, so be sure yours uses a simple +5volt supply to avoid damage.

Power supply

The #twatch requires a 6-7volt power supply through a 2.1mm power supply jack (J1). 2.1mm plugs are the most common size, and should come with every universal power supply.

The higher the supply voltage you use, the more heat that has to be dissipated from VR1 and VR2. Remember that the #twatch is a prototype learning board, it’s not a complete and tested commercial product. Take appropriate safety precautions and don’t run it unattended.

Step 3

PCB and partlist

We used the freeware version of Cadsoft Eagle to make the schematic and PCB. Download the latest files from the project Google Code page.

The PCB is a 2-layer design with small traces and separation (10mil) around the 64pin TQFP PIC chip. We prepared gerbers and sent them to Seeed Studio’s PCB service for open source works. Extra PCBs from our order are available in the Seeed Studio shop. If you buy our extra PCBs be sure to get the HanRun ethernet jack that fits the board.

Since the mounting holes on 20×4 LCDs vary, we didn’t try to fit the PCB to the LCD holes. We made it as small as possible, like SparkFun’s serial LCD backpack, so it stays out of the way of the original holes. As a side effect, it’s not too terribly awkward behind some smaller screen like this 16×2 LCD at Adafruit.

Microchip’s ‘free’ TCP/IP stack provides all the network functions we need to exist on a home network and grab data from Twitter. The stack is open source and free-as-in-beer, but the Microchip license prohibits distribution. Due to licensing issues, we’re only putting our public domain source code in the project Google Code SVN, learn how to download and compile the source here.

The stack has a Dynamic Host Configuration Protocol client that automatically configures the network settings using the DHCP server on your local network. The #twatch requires a DHCP server, but the vast majority of networks and routers have this enabled. The IP address, mask, gateway, and first DNS server are displayed on the LCD screen until valid Twitter data is available.

The stack also includes Microchip’s announce server. When the IP address is acquired by DHCP, the #twatch announces its IP address with a broadcast packet to all computers on the local network. Use the MCHPDetect.exe utility in the project archive to view these packets.

Finally, we included a ping (IMCP) server. Use any ping client to check if the #twatch is alive on the network.

Twitter TCP client

The Twitter trend following program is a simple TCP client, similar to a web browser, that pulls data from web servers. Twitter’s API will give us data in a variety of formats. We used the light-weight JSON format because it’s easiest for the low-power PIC chip to decode, check out JSONView if you use Firefox.

After the #twatch automatically configures network settings, the Twitter TCP client takes control and grabs the current trending topics. It searches through the this JSON datafeed and looks for the “name” tag. Up to 10 trending topics are copied into a 225byte buffer. A separate array stores the end position of each topic in the buffer so we can retrieve the topics in the next step.

Next, the #twatch searches Twitter for 2 tweets for each topic. It appends each topic to the end of the Twitter JSON search url, special characters like spaces and punctuation are URL encoded.

The TCP client parses through the search results and looks for tweets that follow the “text” tag. Tweets have multiple layers of encoding. We decode HTML reserved characters like ampersand (&) and quotes (“) because the LCD screen can display them. We remove UTF8 international characters because the HD44780 LCD doesn’t have them in its character set.

The parsed, decoded tweets are stored in a 2100byte buffer, an additional array marks the beginning and end of each tweet in the buffer. RAM space was a big problem on the 18F67J60 chip, it only has about 4000bytes total, but the 2100byte buffer seems big enough to handle 20 average-sized tweets. We took special care to protect against out of memory problems, and we tested the client under reduced RAM conditions to ensure that it fails gracefully during errors.

Twitter is well known for its occasional down-time. If the #twatch can’t connect to Twitter, it displays a connection error message and the retries twice. If it can’t connect after three tries, it waits five minutes before trying again. This gives Twitter a chance to fix their problems without being hammered by #twatch queries.

The #twatch grabs fresh trend and tweet feeds every five minutes. Twitter puts a limit on the number of queries a client may make, so be careful about refreshing more often. Twitter allows 150 trending topic updates per hour, and “significantly more” search queries.