Tuesday, December 13, 2011

An APRS-equipped high-altitude balloon, using the callsign K6RPT-11 (track on aprs.fi), launched from San Jose, California, has almost crossed the Atlantic Ocean, and is now passing Azores, and approaching North Africa. With a little change in direction it could as well go to Spain or Portugal! It was already a great success when it managed to travel to the east coast of the US.

There is a catch - it's transmitting on the US frequency of 144.390 MHz instead of the usual European frequency of 144.800 MHz. That's will help reception, since 144.390 is very quiet around here, but we need some igates in Morocco, Spain, southern France, Italy, Greece, Turkey, and so on, to temporarily switch frequencies - and do it now! Keep an eye on the track - it could be coming your way!

Please help spread the word, right now, tonight, amongst igate operators around that area.

There might be a very fun recovery operation ahead for some hams down there.

Stephen H. Smith wrote:
"The K6RPT-1 APRS-equipped high-altitude balloon left the U.S. mainland, headed out over the Atlantic about 0330 UTC Tuesday after a coast-to-coast crossing of the US from the San Jose, California launch site.

Radio contact was lost about 400 miles off the New Jersey coast. At that point it was still transmitting and reporting normal battery voltage, holding altitude around 107,000 ft, and headed toward the Straights of Gibraltar at about 150 MPH (240 KM/h)."

"It's now around sunset at it's current location, so it looks as though the balloon's envelope survived the day's UV exposure -- better and better chance now it WILL make landfall over there. ."

Monday, November 21, 2011

As seen in the statistic graphs, aprs.fi was only receiving about 60% of the APRS packets from the APRS-IS between 2011-11-20 23:40:40 UTC and 2011-11-21 08:25:57 UTC. During this time it was connected to a core server which apparently is not getting a full APRS-IS packet feed for some reason. As a visible result the packets of about 40% of APRS stations did not get to aprs.fi.

The core server operator has been notified and aprs.fi is connected to another server again. Sorry for the trouble.

Wednesday, November 2, 2011

I know some of you use aprs.fi while doing volunteer search-and-rescue operations, ARES or other public service work.

If you wish to keep doing so as before, with the convenience of Google Maps, I need you to tell me and everyone else about your use. In writing. Please.

Put up a blog post or write about it on your volunteer organisation's web page. Send me an email pointing to the document. Write about what you are doing, where, when, and how aprs.fi is useful in what you do. Practical examples of real events are probably most useful.

The charging is based on the amount of map loads. aprs.fi currently opens up the real-time map almost 42,000 times per day (when nothing special happens). According to the new Google Maps terms, up to 25,000 map loads per day is free, the rest will cost $4 per 1000 map loads. That amounts up to $68 per day, $2040 per month for me. No, I don't make that much from the advertisements. And there are some other costs involved in running such a site too (computer hardware, hosting, domain names, programming beer, to name a few).

"We will then start billing excess usage to your credit card when we begin enforcing the usage limits in early 2012." (the blog post)

The only real options seem to switching away from Google Maps to something else, or getting aprs.fi to qualify as being "in the public interest":

"Non-profits and applications deemed in the public interest (as determined by Google at its discretion) are not subject to these usage limits. For example, a disaster relief map is not subject to the usage limits even if it has been developed and/or is hosted by a commercial entity." (FAQ)

A lot of you will probably suggest switching to OSM. This is one option, but it has a few drawbacks:

Address/place search isn't as good

No Street View, worse Satellite view, no Terrain view

Less coverage in the countryside

Takes a lot of work from me to switch from Google's API to OSM's API (aprs.fi uses the Google Maps API for a lot of things, like drawing lines & circles and placing car symbols on the map and presenting menus and pop-up balloons)

So, I'll have to try to get in the "public interest" category. It might help if you could document your real usage in "disaster relief" and other public service work in writing. You can find my email address from the blogger profile (on the right side, scroll down to "about me", view my complete profile, contact, email).

Monday, October 10, 2011

The embedded map displays a link back to aprs.fi in the low right corner again, as it used to do until July 2010, when aprs.fi switched to Google Maps API v3 and got Street View support. It should be much less annoying than the old ugly box, though. The new link is smaller and slightly transparent (on modern browsers).

I also upgraded to Google Maps API from version 3.4 to version 3.6, which brought in several enhancements and bug fixes. The API has it's own change log. Noticeable changes include fade transitions for map tiles when loading and changing the map type, Street View pegman previews, some speedups, and several bug fixes for iOS and Android.

I have also recently made some database access optimisations making many map displays and some raw packet searches load much faster.

Since 28th of August most, if not all, APRS symbols now have hover-on tooltips. Keep the mouse cursor on top of a station symbol and a description text should magically appear!

And, as you can see from the attached photo, I have some new toys for 70 cm. Built in GPS, text messaging, trunking support, rugged construction, submersible, digital and analog voice, and a few other nice features. And surprisingly inexpensive on ebay, when compared to new amateur HTs with APRS.

Monday, August 22, 2011

On Saturday and Sunday I did a little coding, and installed the new version before leaving for work this morning.

When an user tries to save a filter list twice with the same name, and error message is now displayed. It turns out that a lot of users press the "Save as new list" button after making any changes to the existing list, and end up with 20 lists (or more!). That button is only for creating a new list. Any changes to existing lists are saved immediately without pressing any additional buttons.

The Preferences view got a new hint text pointing users towards the new Filter tool button. A lot of users are complaining about the "missing filters".

When an user account is deleted, data related to that account is now more thoroughly deleted. That includes web stations, alerts, favourites, and so on.

Alert configuration changes should now actually do all the necessary database changes to enable new alarms and disable old ones.

The real-time map should now more reliably draw a symbol on the last point of a track. An old bug sometimes left the last point without a symbol.

When a client computer wakes up in the morning from a good night's sleep mode, the data reload should now be a quick one, instead of a very slow one.

Session cookie processing was also improved, and memcached was upgraded.

Again, it doesn't make sense to take a photo of software, and a blog needs to have some photos, so kittens it is again. This is FIN*Kukkatarhan Bellium, a chocolate-spotted Ocicat, posing in Mikkeli a few weeks back.

Friday, July 29, 2011

I did a quick upgrade last night. A few bugs were found immediately after the upgrade and needed to be fixed but no serious harm was done.

The upgrade includes revamped session and preferences storage. The user-visible change is that user preferences are now stored in the server backend and attached to your user account if you're logging in when saving the preferences. They will be used on other computers, browsers and mobile devices as long as you're logged in with the same account.

The preferences can still be saved if you're not logged in, in which case they're attached to a browser cookie.

A less visible change is that the settings and session cookies set in the browser are now much shorter - just random keys, instead of containing the whole settings or session data. This makes the HTTP requests smaller, which improves network performance marginally. On a slow mobile connection it might actually make a difference.

I don't have a related photo to attach, so kittens it is again! Buxus and Brassia, aged 6 weeks.

Tuesday, July 26, 2011

I've just finished installing a rather large upgrade on aprs.fi. The upgrade comes with a couple of noticeable new features which everyone should get familiar with, a bunch of smaller style fixes and some tiny bug fixes. The web service was down for a while due to the database schema changes and engine upgrades. Only very little APRS data was lost - APRS-IS traffic was buffered during the database maintenance and processed later when the database was up again.

Here's a list of the major changes:

Complex but easy filters:

The old filters (APRS, AIS, Items+objects, WX) in the Preferences have been replaced by a whole new Filter tool button on the right side of the tool bar. It's the one which tries to look like a funnel (I'll have to work on the symbol image to make it more obvious).

Let's do a little exercise to get you started!

Deselect the All stations filter, so that no stations are displayed. Then enable the Station type: Weather filter and you'll only see stations transmitting weather data.

Next, continue by enabling Station class: APRS stations. At this point you'll see both APRS stations and weather stations, which doesn't make much sense since all the weather stations are also APRS stations. But it starts to make sense again if you click the red downward-pointing arrow for the weather stations, which throws them to the bottom of the list, in the red "AND NOT" department, effectively removing them from the display.

Now, click on the plus (+) button to add your own fancy filters to the list. For example, it's possible to create a filter which only shows APRS stations using Kenwood radios which are also acting as Digipeaters, but do not have a callsign starting with "OH2". That filter can then be used in the green "display any of these" side of filters (think of logical OR if you're into programming) or the red "but not these" side (think of AND NOT).

Once you're finished designing your favourite filter list, use the Save button to save it to a new named list. The default filter will reset to it's factory settings every time the map opens (so that you'll see something if you get lost with the filters). The filters are saved together with your user account, so you'll need to log in to save them. The upside of this is that you can easily use your preconfigured filters on many computers and even mobile devices!

Personally, I've set up several commonly used filter sets in my named lists, such as "Infra" for network infrastructure: digipeaters, igates, and "Moving" for stations which have their speed larger than 2.

New graphing engine:

Telemetry, weather and station graphs are displayed by a new third-party graphing engine instead of a homebrew one. The new engine reduces load on my servers (and adds some on the web browser's side, but that's OK since there are thousands of those against my 2), generates prettier line graphs, and will enable some really fancy interactive features in the future!

Station activity alerts:

This feature is mainly for digipeater and igate operators. It lets you know by email if your station stops working, or when it starts working again. Very handy especially if you're like me and maintain digipeaters far away from your home and would not otherwise notice when they go "kaput".

Click on the Favourites (star) tool button, then select My stations and bookmarks. Add your digipeaters in the Stations I follow list, and then click Settings for each of them and tune the timings.

Please note that due to APRS-IS duplicate filtering aprs.fi often does not get to know when your igate or digipeater hears a packet and repeats it. It's a good idea to set the "has not heard station" alert for a very long timer such as 24 or 48 hours, and then go down or up from there depending on the amount of false alarms!

An additional duplicate packet filter algorithm:

In addition to the old filtering methods, out-of-order and duplicate packets are now filtered using APRS timestamps embedded in the packets. If a packet comes in that has an older timestamp than the previous packet, it is promptly discarded. This should be a very effective duplicate filter when timestamps are transmitted by the tracker.

The raw packets display can now also display the decoded timestamp when using the Decoded mode.

Facelifts:

Since most people come to aprs.fi to look up a station's position, the callsign search box has been moved up on the right-side panel, and the time range selector has been moved down.

The real-time map displays a nice "loading" image until station data has been first received from the servers.

Buttons now have a prettier and consistent style everywhere. Some other layout and style tunings have been applied, too.

Bugfixes:

The real-time map's content reloading logic has been improved. It should now be more responsive when panning and zooming around. There's still a small bug in there - sometimes stations are not reloaded on the map when the filters are changed at precisely the right time. Oh well.

The sortable tables were not working in many views. Now they are. Click on table column headers to sort, ascending or descending.

PHG overlays are now correctly hidden when zooming out to area activity view.

The duplicate stations API bug was fixed.

As always, some smaller ones were fixed too. I also did some profiling and optimisation of old code, hoping to keep the performance of the code on similar level as before, even though new features were added in the critical code path.

Wednesday, July 6, 2011

aprs.fi suffered a partial network outage last night (July 5 21.37 UTC - July 6 5.03 UTC). A gigabit Internet uplink was lost for the duration of maintenance by an ISP and the alternative path didn't quite provide all of the Internet's routes over BGP. Investigation continues.

The outage caused trouble for many to view the site, and some APRS data collection problems. The stats page gives a hint of the scale of the problem: http://aprs.fi/stats/daily

About 50% of the viewers disappeared. It would seem like regular APRS-IS data was collected just fine, but some CWOP data was lost. There are a lot of alternative routes that the data collector can take to get to the APRS-IS, and that seemed to work.

Sunday, June 5, 2011

On 8 June, 2011, Google, Facebook, Yahoo!, Akamai and Limelight Networks will be amongst some of the major organisations that will offer their content over IPv6 for a 24-hour “test flight”. The goal of the Test Flight Day is to motivate organizations across the industry – Internet service providers, hardware makers, operating system vendors and web companies – to prepare their services for IPv6 to ensure a successful transition as IPv4 addresses run out. (source: World IPv6 day)

I'm happy to announce that aprs.fi is also taking part in this experiment, where IPv6 support is suddenly enabled for a large amount of high-profile web sites. Please note that IPv4 support will not be disabled during the experiment. I'm only adding the IPv6 AAAA DNS records for the main aprs.fi site, instead of having it as a separate entry on ipv6.aprs.fi.

If you find that aprs.fi stops working for you on Wednesday (give or take a day to allow for timezone differences), you'll also soon find a large number of other sites not working. Your network operator's network does not support web sites which provide service to both IPv4 and IPv6 users. According to statistics made by Wikimedia on the Wikipedia site, this will hit about 0.3% of users. If you happen to belong to this group, it's important for your network operator to realize that this will eventually become a permanent situation. Your network needs a little fixing.

Saturday, June 4, 2011

Yesterday afternoon aprs.fi made a new record: there were well over 3000 users viewing the real-time map at the same time. Almost all of them were displaying the embedded map on the Copenhagen SuborbitalsLaunch Campaign June 2011 page. The guys built a rocket, capable of taking 1 man to the edge of the space, and did their first successful test flight yesterday. Awesome.

I was happy to see that aprs.fi did well with the larger amount of visitors. They were all looking at the same station, and the caching worked well - the database really didn't get any additional queries due to the amount of viewers. I was able to find some spots where additional optimization would be useful and could take down the CPU usage of the web service considerably. Here are some graphs from one of the two servers:

Saturday, May 7, 2011

I'm coming over to the US to visit Dayton Hamvention two weeks from now. See you there! If all goes well with the manufacturing, you should be able to recognize me from a nice aprs.fi branded hat & shirt.

For the clothing I had to create a new logo, which can already be seen on this blog and the Facebook page! I'm more of a musician and a programmer than a graphics artist, which is probably visible. :)

I've also been working with the new alerts and map filtering functions. They're quite large improvements and still need a lot of work on the user interface side, although the core functionality starts to take shape already. Stay tuned.

Saturday, April 2, 2011

A memory table containing all of the last position reports for the past 30 hours or so, used for quickly loading the real-time map view, filled up last night on one of the two front-end web servers at around 2010-04-01 23:35 UTC (2010-04-02 2:35 local time). I woke up, noticed the SMS alert and fixed the problem at 2010-04-02 6:50 UTC (9:50 local time).

While the table was full, replication was stopped, and about 50% of real-time map loads (without a station tracked, just an area view) resulted in no stations displayed. Also, 50% of page loads on the site last night returned a bit old data (the stopped frontend continued to display the 23:35 UTC state!).

No data was lost during the incident, APRS-IS data collection continued normally.

Wednesday, March 16, 2011

[UPDATE: Do read the comments below the post too, for workaround suggestions.]

If you happen to have a Kantronics KPC-3+ TNC, please do not use it for your APRS igate. It appears to have a software bug which causes delays of over 10 minutes when receiving packets from the radio and then forwarding them to the computer, which then forwards them to the Internet (or possibly retransmits the old packets back to the radio channel, if digipeating).

This is a very bad thing to do, as the greatly delayed packets cause network overload and make moving vehicles jump back and forth between their current positions and the past positions. Looks very funny when aprs.fi tries to draw a track line between the received positions.

For years there have been anecdotal stories and suggestions about a possible problem. Yesterday Alan (radionerd1) has uploaded three videos to Youtube demonstrating the problem. This serves as a nice technical proof that the problem is real, and demonstrably a problem of the KPC-3+. There have been hints that the bug could be in UI-View32 (when using it with the KPC-3+), but Alan demonstrated the problem without involving UI-View.

Alan ran APRSIS32 on a computer, and connected it to two APRS receivers. One used a KPC-3+ and one used AGW packet engine (sound card packet decoder). At first, the two ports received the same packets at the same time. After about a week the KPC-3+ started to misbehave - the received packets were given to the computer only after they had been held as hostage for over 10 minutes. Some people have reported that it can go in this delaying mode within hours or days - it might be due to bad luck, or due to the amount of traffic received. The KPC-3+ did put out a KISS packet to the computer every time a packet was heard from the radio, but it was an old one. When the TNC was reset, it started performing well again.

My guess, as a programmer, would be that the KPC-3+ looses count of the packet it should be transmitting on the serial port. It receives a packet on the radio port, puts it in the received packets buffer, and then prints the wrong packet on the serial port. It might be the oldest packet in the buffer, or thereabouts. The amount of perceived delay would depend on the amount of traffic received in your area.

So, I repeat: If you have a KPC-3+ on your igate in KISS mode, please switch it to something else as soon as possible (KPC-3, OpenTracker, TinyTrak, TNC2 clones, whatever). If you wish to continue using it later, please contact Kantronics at service@kantronics.com and ask them to fix the software bug.

It has been said that the problem only exists in KISS mode. So if you're using the KPC-3+ as a stand-alone digipeater, it should be fine (in this respect). If you're using it as a digipeater in KISS mode, with the digipeating happening on the computer (digipeating enabled in UI-View32 or APRSIS32), the effects are seriously bad, as you're transmitting old packets on the radio channel.

If you happen to be Kantronics, please fix the bug. It's causing a great deal of havoc to the APRS network, and it's messing the contents of the aprs.fi database. Thank you!

Saturday, March 5, 2011

I won't be running an aprs.fi booth or anything, but if you're selling APRS stuff and/or using aprs.fi to demonstrate it in your booth, I'll be happy to stop by and answer questions. Drop me an email – my address can be found in the blogger profile.

Friday, February 18, 2011

It's now possible to upload your position to aprs.fi from Google Latitude. Latitude, in turn, supports uploading position from a large number of mobile devices (Android, iPhone, Blackberry, Nokia/Symbian, Windows Mobile) using Google's Maps and Latitude applications.

If you don't see your position immediately on aprs.fi, please wait for a few minutes, and check if Google Latitude has your position on it's own web site. If it isn't shown there, the problem lies with your mobile application's configuration or lack of accurate position information on the phone. aprs.fi currently rejects positions with an accuracy that is worse than 4000 meters.

The web stations page in the user guide is a new thing, too. In addition to Latitude support, it documents the manual updating (point-and-click) and Geolocation API (web browser with GPS support on a mobile device) methods for position updating.

I've run it on Linux and Mac OS X only, but it should run on any operating system with a Perl interpreter, including Windows, maybe. But I can't help you if it doesn't run on Windows - please find someone with Windows programming experience if you run into trouble.

The POCSAG message encoding happens within the POCSAG::Encode Perl module. On the computer. Not within the Arduino modem. The Arduino modem software is not useful on it's own, the perl code is required to send POCSAG messages.

The Arduino modem is needed because (1) my computers don't have serial ports any more, and (2) bit-banging synchronous 512/1200/2400 bit/s FSK using handshaking pins of a serial/parallel port, like the Baycoms did, is not possible using an USB serial adapter cable (the timing is too strict). The Arduino is a convenient and relatively low-cost platform with a good bunch of TTL level I/O pins and an USB port, and could be easily programmmed to drive the transmitter. If you wish, you can write a kernel driver to drive the FSK bits generated by the encoder, baycom-style, on a serial port (if you have one) and skip the Arduino part.

If you feel the Arduino Duemilanove/Uno boards are too expensive, you can look at the other boards too. Just get one with enough SRAM. If you're brave, you can also put the CPU and a few necessary components on a breadboard or a custom printed board of your own. Instructions here and here and here.

The POCSAG::Encode module is published on the CPAN, and the Arduino modem on the wiki.

Yes, the POCSAG encoder can be connected to any VHF/UHF radio which has a suitable FSK input. Many radios (including some that do have a 9600 bit/s data input) aren't up to the task. I've tried the Motorola GM340 and the Teknomen POCSAG radio only. The GM340 needed some configuration (using the programming software) to make the data input work the way it should for this application. See the comments on the previous post for wiring tips.

Yes, the POCSAG::Encode perl module should be useful when trying to transmit POCSAG with other kinds of modems.

Monday, January 3, 2011

Today I implemented ambiguous position support in aprs.fi. Many APRS transmitters using MIC-E or uncompressed packets can be configured to intentionally transmit less precise positions. This may seem a bit backward at first, but there are perfectly good reasons to do so. Some people might want to transmit a rough location of their car without revealing the exact parking spot where their expensive radio gear spend their night in. Some might like some aspects of APRS but wish to adjust the level of privacy by hiding their precise location.

Ambiguity is configured by setting the number of digits which will be truncated from the end of the position. Plaintext APRS positions are transmitted in degrees and decimal minutes (DD° MM.mm'), with two decimals of minutes. When ambiguity is set to 1, it'll be truncated to DD° MM.m', 2 will transmit DD° mm', 4 will transmit DD° only, resulting in a resolution of 1 degree.

Stations transmitting ambiguous positions are now shown with a purple dashed border around the callsign label. The precision of the position can be visualized by hovering the mouse pointer on top of the station symbol – a purple rectangle will pop up, showing the area of ambiguity. The station symbol is drawn at a slightly randomized position within that area. Also, the info page now says "Position ambiguous: Precision reduced at transmitter by 3 digits, position resolution approximately 18.5 km."

There is one known big bug in this code: the ambiguity is not yet stored together with each position in the position history database. Only the ambiguity of the most recently received position is stored. If you turn ambiguity on or off, the effects on aprs.fi might be slightly weird when looking up old tracks. I'll fix that later when I next change the position history table schema.

I've been told some Kenwood mobile rig owners have accidentally enabled ambiguous positions in their transmitters without realizing what the setting actually does. This change should make the effect visible in a more meaningful way. On the TM-D710, the setting lives in menu 606 (BEACON INFORMATION), POSITION AMBIGUITY.

Off-topic news: Kukka (FIFE GIC Moosegrove Orqideas Doiradas), our lovely chocolate spotted ocicat, is expecting kittens in late January. The sire is CFA CH Windhaven Salt and Pepper, an immigrant from the USA (family tree). She's gained quite some weight, as can be seen in this photo. Kittens for sale later this spring, details at the cattery site!