When my boss' father, john sinclair, mentioned that he could use some help with
building and installing a rain gauge with online data collection
on Fraser Island (for fido.org.au) I naturally
said 'sure'; my experiences with designing and building two remote wind stations for the CHGC were good,
and I find things like that interesting projects to tackle. The rest of this
article documents the good, the bad and the ugly sides of the exercise - if
you want just the goodies skip to the page bottom :-)
Some discussion about the budget and the desired capabilities later we settled on building something around a davis vantage vue sensor+console, a beaglebone black single-board computer and a bog-standard gsm/gprs usb modem.
the capabilities of the vantage vue looked quite good in relation to its cost, and I had found quite a bit of online info about the ease of interfacing with it (as it has a serial connection and a semi-sensible comms protocol). the vantage sensor pod goes on a pole, the console somewhere nearby and the beagle gets attached to the console, talks to it and uploads the data to a server on the intertubes, which stores and displays the goods. Really easy!
we need no stinking green dots, we have PICs!
Well - that was the plan. Until just after the purchase had been organized, when I realized that there's a fairly infamous comms lock-out (or vendor lock-in) in the current davis firmware, the "green dot issue". No third party comms allowed ("it's for the sake of the childr^Wcompany bottom line"). Lots of people hate davis for that, and I'm quite sure they have lost and are still losing a fair amount of business because of how they orchestrated that bungle (which is a part of why I'm not including any links to their websites on this page).
But all is not lost, people quickly reverse-engineered the setup and Torkel Jodalen has a very nice writeup on a workaround (it's missing just two crucial bits of info that cost me lots of time: they do SPI at 500kHz, and the start and stop timing is really, really tight). The idea is to use a cheap microprocessor of your choice to mimic and authenticate as a green dot data logger, after which the serial protocol is enabled and you can talk to the davis with anything capable of doing RS-232 (at 3.3 V).
Torkel's code is for Atmel and a weird basic dialect; I decided to build something for one of the Microchip PICs from my stash (because I'm familiar with them and not familiar with the Atmel units - yet).
First dead-end: I programmed a big-banged implementation of an SPI protocol slave from scratch, and soldered up a small board with a pic12f683 because I wanted to use the cheapest 8-pin unit I had around that's not too slow. It worked fine in my tests (with a Bus Pirate providing the SPI master) but maxed out at less than 100kHz SPI clock. (let me know if you'd like the SPI slave code; I think it's good code but too slow on an 8MHz pic.)
Second semi-dead-end: I switched to a pic16f88, which has a hardware serial port with all the common modes (async, sync, spi and i2c). That's clearly much faster, tests worked perfectly up into the low MHz SPI clock ranges, I resoldered the little board - except it wouldn't work with the davis.
Third detour: all SPI-related datasheets suck. Every single piece of documentation for SPI that I got my hands on had at least one glaring error, incorrect definitions of mode 0/1/2/3 etc. etc. None of this is helpful if you're trying to debug something that's realtime, over in just 2 milliseconds and tightly timed.
Lots of headscratching later, and just about ready to give up, I eventually dug out my small logic analyzer (an Openbench Logic Sniffer) and had a real peek at the live protocol. the OLS samples quickly but doesn't have much buffer, so you have to get quite fancy with the triggers.
But there it was: my bitbanged protocol could not have worked, ever, with the 500kHz clock the davis uses. A good day more and lots of testing later, and I eventually realized that the startup timing was very tight - much tighter than my very conservative and safe code on the PIC could meet; less than 8 microseconds between chip select and clocking. (The tests with the Bus Pirate always worked because that leaves ample idle time everywhere except the actual data transfers.) So I stripped all safety checks from my code and /just/ got it to work on the 8MHz PIC (anything faster would have required an external crystal). And hey presto, it worked! (all in all this took only three full weekends and a few extra evenings... not especially stellar performance, but it did get done.)
beagles are fairly cheap, mobile data is not
The rest of the work was pretty anti-climactic and very straightforward. The beaglebone black is a great tiny single-board unit (exactly the size of a credit card), and much better than the raspberry pi IMHO - really open, much better board layout, better cpu (hardware floating point), better documentation (I think), and much more flexible as it has 4GB of flash already on board. It ships with and runs plain unmodified debian, which I clearly appreciate a lot.
The beagle does IO at 3.3 V, and has lots of separate UARTs on chip; hooking up one of them to the davis was trivial. This particular beagle is powered by an external 5V transformer, and both PIC and davis console are powered from the beagle's 5 V rail. (davis at least made the console a very energy-efficient design.)
But mobile data in Oz is expensive. Fraser doesn't have much gsm coverage, and we worried we'd have to pay the monopolist's extortionate rates...so I boiled the data uploads down into 22 bytes of UDP data once every 20 seconds. That means a total of less than 250 kilobytes for a full 24 hours. Even a bloodsucker like Tel$tra couldn't suck us dry given that little data expenditure, and all the relevant readings are still fully transferred (if some at at slightly reduced precision).
In the end it turned out that we could use an aldimobile prepaid sim (they resell "parts of Tel$tra's network"), which means it'll cost just between $15 and $20 a year to run this station. Very nice, my bent for frugality is satisfied.
The beagle got a few further helpful goodies of my design: you can SMS it a message for a reset and reboot, and another one makes it start up an OpenVPN for remote administration.
As I found that Wheezy's kernel on the beagle
occasional hangs during soft-reboot, I tied the IO pin nearest the reset
line to said line, and made the board hard reset itself as the
last step of the reboot procedure (let me know if you want the few
relevant lines for /etc/init.d/reboot).
I made a small doghouse for the beagle and attached it to the back of the davis console, and then we tested everything for a few weeks (with the sensor pod in my back yard).
The other side of the system, the server code, was boringly easy; just a bit of perl that receives the UDP uploads, verifies the authentication and sanitizes the inputs, then parks the data in an SQLite database. If desired it can be prodded by cron to upload to weather underground and/or the wow/bom website, and there's a gnuplot-based chart maker and a simple cgi script for display. nothing fancy, but robust.
fido, fraser's (watch)dog, gets a beagle...
The past weekend Keith, John and I installed the setup on Fraser Island, at Happy Valley. Saturday afternoon first saw us dig a pretty deep hole to safely seat a 5 m pole for the sensor pod. This went much faster than we had expected.
And hey presto, the beagle and console and upload part worked immediately after plugging in the power, no fiddling or any magic required. My main remaining worry, that the metal-walled shed would block the wireless transmissions from the sensor pod outside to the davis console inside, turned out to be unfounded; the comms isn't totally perfect and we will lose a packet every now and then, but that's fine.
The station will also not report perfect wind directions because of the location, but again that's fine - the most important component for John is the rain data, and that's unobstructed.
...and dingos!
As everything had worked out without a hitch, we had some spare time and Keith and John introduced me to more of the beautiful spots on Fraser.
On Sunday at the Lake Wabby walking track we saw two dingo pups that inspected everything in the vicinity - and while I'm not a fan of dogs in general, these two looked pretty good and healthy. (yes, I know they're not dogs...I'm still not a great fan of any canides)
less blah, more bling!
As usual all my code is open-source under the GPL and freely available on github at https://github.com/az143/davis_weather. Feel free to use/reuse/abuse or improve :-)
The fruits of all that labor are publicly available online:
you can see the current readings, 2-hour and 24-hour graphs on the fido website.
the data is uploaded to the BOM/WoW site every few minutes, and can be viewed there,fido stopped uploading to BOM/WoWand it's also sent to the weather underground, which produces nicer-looking graphs.weather underground is shutting down their api.
