Pages

Friday, January 3, 2014

More Than One Way to Skin a Cat

I know what you're thinking: "DeKay hasn't posted in a while. Maybe he's dead?" Fear not. For while the frequency of my posting has dropped, I remain very much alive. I've just been a little busy at work.

Thankfully the holiday season rolled around and I was able to take a couple weeks off. I took that time to relax and catch up on all the things normal people like to do: enjoy some good food, visit with friends, and hack on microcontrollers with integrated RF transceivers.

A small FIFO means that you better have something real time or close to real time listening to the module. Otherwise its buffer overflows and you lose data. Something like an Arduino works well because it isn't doing other stuff like running an operating system. But worse than this is the hardcoded sync bytes. This pretty much limits the RFM12B to talk to other RFM12B's. And that ain't cool.

The RFM69 fixes all these problems and adds some nice features while it is at it:

it has a 66 byte FIFO, a fully programmable sync pattern up to eight bytes in length, and a variable length preamble

address checking and CRC validation is optional

it supports OOK, FSK, or GFSK

it can do AES encryption

other stuff

In other words, it looks like it can do what other transceiver modules can do and more.

Intrigued, I picked up a Moteino with integrated RFM69W module and a couple standalone modules while I was at it. Then I got busy (see above). Then I got some free time (see above). Then I started hacking (see below).

Devoted readers of this blog will know that I am strangely compulsed to receive wireless transmissions from my Davis weather station using all kinds of strange hardware platforms. So why stop now? I've been looking for a platform like this that is available and easy to use so that more people can take a crack with this stuff (it isn't like you can walk into a Toys-R-Us and pick up an IM-ME anymore). I also don't want to be glued to the Arduino for everything. While it is great for running as a remote node, I like the idea of something like my BeagleBone Black connected directly to a transceiver module running some home monitoring and control software (HouseMon?) with Internet connectivity.

But, baby steps. What I decided to start with was my RFM69 equipped Moteino and the LowPowerLab RFM69 Arduino library. Because once you've got the wireless protocol figured out, hard hard can it be? Turns out, harder than I thought. But after slamming my head against a brick wall a time or two, I got this.

Success!

This is the data from my Davis weather station out in my yard and received on my RFM69-equipped Moteino. The first field is the channel number, the data is the eight data bytes
sent by the ISS on every transmission (see here for what those bytes mean), RSSI is the Received Signal
Strength Indication, and the CRC is calculated from the first six bytes
in the data packet. Every packet is sent with a CRC in the seventh and
eigth bytes, so those two values will agree with the calculated CRC if
the packet is good. I've got some problem somewhere where the first packet I receive is bad, as is any packet on Channel 42. But those are details. I'm off to a decent start here and I'm sure I can sort those issues out in time.

Want to check it out? My DavisRFM69 library is on GitHub and the data above was collected using the ISSRx sketch in the Examples folder. The approach that I've taken is to subclass my driver from the LowPowerLab one, overwriting just a couple of methods that are specific to this application (namely the initialization and the interrupt handler). That reduces duplication and lets me automatically pick up any improvements Felix incorporates into his main library. I have made a couple minor tweaks to his main library for some register definitions that I've issued a pull request for. Hopefully these will be incorporated into his main library soon. In the meantime, you'll want to use my fork of the library, so everything will compile cleanly. This and more is all in the README.

This work can be taken a lot further:

do a full emulation of the Davis indoor console for use by weather software like WeeWx and Cumulus

hang an RFM69 directly off a BeagleBone Black and write a driver to work under node.js

why just receive?

Pull requests gladly accepted :-)

I'm glad to now have this stuff under my belt. I could have taken the entire break to just sit in front of the TV eating fruitcake. That would be easy because My Lovely Wife makes a rocking fruitcake. But what I really find satisfying is that sense of accomplishment when you set a challenge for yourself and make it happen. This was a good way to kick off 2014, and the year is just getting started.

24 comments:

As always a very interesting article! I tried to load and compile your RFM69 sketch but "SPIFlash.h" seems to be missing and therefore compilation breaks with error "ISSRx:31: error: 'SPIFlash' does not name a type". As I am a Ardiuno freshman, this is certainly my beginners fault. However a hint how to get around this would be very helpful.

Sorry for spoiling this article, I just found out that it needs the SPIflash Library from LowPowerLab (https://github.com/LowPowerLab/SPIFlash) also included. Now it compiles fine and I have to wait for the HW to arrive to give it a practical shot.

I tried some math with the frequency values given in your example. 1) oscillator frequency of RFM69 is: Fxosc = 32 MHz2) available frequency step size of RFM69 are: 61.0351563 Hz (32 MHz / 2^19)3) programmed frequence deviation for Davis is: Fdev = 4760 Hz4) frequence for index 0 packets is expected to be 902.5 MHz5) when I do the math on the index 0 entry in the frequency table of your code "{0xE1, 0x98, 0x71}" it results to: 0xE19871 * 61.0351563 Hz = 902.381.897 Hz But that is quite a large deviation from the expected value, even when I add Fdev.

Any idea what is wrong in my math? Unless I understand this it will be hard to do the trick for the Europe frequencies ;-)

Thanks for your interest. Your math is actually correct. Here is where you are going wrong...3) Fdev is not a part of this calculation. It is needed for figuring out the modulation, but not the carrier frequency4) I'm not expecting Index 0 packets at 902.5 MHz. Check out the post http://madscientistlabs.blogspot.ca/2011/03/davis-frequency-hopping-sequence.html and look at the embedded spreadsheet. My example was for channel 1, not channel 0, and the sniffed RF value was 902.381925 MHz. Although that spreadsheet does have an "RF nominal" column, that doesn't mean anything today. I thought at the channels at the time would be evenly spaced, but now I know for a fact that they are not.

I thought later that I code up the European frequencies in here too. If you beat me to it, send me a pull request. I'm trying to knock off another item from my todo list right now. My little spreadsheet that figures this all out says the European frequency array should be...

Thanks for helping me understand. To be honest, I was not aware that more data comes when I do scroll horizontal ;-)

When I take your frequency data above it results according to my math to what is listed below (deviation in Hz to expected values as listed above) in brackets:51: 868066711 (14)52: 868297119 (6)53: 868527466 (59)54: 868181885 (40)55: 868412292 (33)As all these deviations are below the Fstep of abt 61Hz these look to be the correct values.

For the moment I don't pull, as I don't have the HW yet, which I need to produce real results and not just noise like now. When adding the lines, you might have a look at the US frequencies as well, as there seems to be missing one. I had expected 51 value triples (#0 - #50), but only #0 - #49 there.

This stuff is great. I'm considering buying the barebones VP2 without its ugly and (for me at least) useless console. I'd need the console just to be able to hook the it up to the web anyway. I'm also in Europe so I'm gonna need the 868 MHz version too. If this seems doable I'll prolly buy the VP2 and test out your solution and integrate it with WeeWX or something similar (haven't decided on the SW yet).

Found the problem with the very first packet. I don't know why, but the chip was reporting that the FifoNotEmpty and FifoLevel bits in the ReqIrqFlags2 register after powerup and reset for reasons I don't understand. This register can be cleared by writing to the FifoOverrun bit. Did this in my init routine and my first packet problem went away.

I was also seeing packet errors during normal reception and that was caused by the LowPowerLabs code being overly chatty to the radio. Fixed that too.

I'll give that a try but I suspect the problem will always be the first frequency. It occurred to me late last night that I am not programming the frequency registers when I initialize all the rest of them, but do it a little later. Unfortunately I had very little time to play around with this stuff last weekend. Gotta leave some time this weekend!

BTW, are you able to put up your source where you were trying to emulate the console? There is a thread on wxforum where you said you were making progress on this and then ran out of RAM. The Moteino has double the RAM, so it might be doable. I noticed that your source files for this stuff are no longer on your website.

I've bought a VP2 ISS EU version (without console), and currently trying your code for reception on a Moteino/868. I've replaced the FRF array with your calculated one above for 868 MHz and adjusted the hop count, but I'm getting only garbage so far:

Something's missing for this frequency to work. Since I don't have the console I'll need to get it working. I'll ultimately end up emulating a Davis console for alternative weather apps. Do you have any suggestions or directions to go? Anything comes to your mind I'm more than willing to test on the ISS.

Well, I have some moderate success. I modified the code to only hop when the crc is right. The band seems to be crowded here. I get consistent readings but I have many interemittent gaps and I had to move the ISS near the receiver (they're now about 50cm/20" apart only, very unpractical). It seems that some receiver parameters are not optimal for reception. How would one go optimzing them? I tried to shift frequencies a little but it didn't really help.

Alright, it turned out that the noise picked up from the PC was the problem. I ran the Moteino from a battery and made it so every successful reception blinked the led. It was blinking every 2.5 secs continuously, so far soo good. Moving to phase 2. Thank you for your great lib and all the info you gathered, without this it wouldn't have been possible :]

Your find is interesting. I have been getting intermittent lost packets and maybe my PC is the cause? I'll have to try battery power sometime.

I am actually well along the road to emulating the Davis console and hope to have something that works with Cumulus over the weekend. In the meantime, get onto Ebay and order up a DHT22, BMP085 or BMP180, and a DS3231. Stay tuned...

Cool. I actually already have a couple of DHT22's and a BMP085. I plan to build a few sensor nodes similar to the EmonTH (or actually buy a few).

But first, I need to finish my quest on the internet-connected ISS but I'm stuck again. Looked high and low but can't figure out the formula to convert the raw temp data on my metric unit. I commented on it here too: http://www.wxforum.net/index.php?topic=18718.msg208650#msg208650