Secondary Navigation

Report on 6M ISCAT-A vs. -B comparisons

VE1SKY and I were delighted to welcome WSJT9.3 and ISCAT-A to the suite. (I would also point out that Joe, K1JT kindly added elevation to the 9.3 Echo software

Message 1 of 5
, Oct 8, 2012

VE1SKY and I were delighted to welcome WSJT9.3 and
ISCAT-A to the suite. (I would also point out that Joe, K1JT kindly added
elevation to the 9.3 Echo software at my request--very handy for combining
multiple days' EME echo runs for spreadsheet analysis of lobes under real
conditions.) Joe indicates that ""ISCAT now has two sub-modes.
ISCAT-B is the original ISCAT mode with total bandwidth 1809 Hz. ISCAT-A runs at
half the rate, uses half the bandwidth, and (for average decodes on steady
signals) is about 1 dB more sensitive. It has been found very effective for
making aircraft scatter QSOs at 10 GHz. ISCAT-B probably remains the best mode
for 6 meters, but experimentation is
encouraged!"

Happy to oblige, we decided to conduct some
on-the-air comparisons with some other stations ranging from several hundred
miles away to over 1000 to begin to get a feel for how the two compare
under what conditions. When both of us were available, we alternated 30sec
intervals and sent ISCAT-A solidly for minutes ending 1,2,3, and 4, and ISCAT-B
for minutes 6,7,8,9. Minutes ending 0 and 5 were for both ends to switch between
-A and -B. A similar approach was used for others to send to me later. Stations
participating with us included N9OLT (significant participation on two different
days at a distance of 1070 miles), W3NF, NY2NY, W9NHE, and KD9NH. Receiving
stations were to set sync to -20, tolerance to 400, and keep hands off the
decoding process so the records were automatic. VE1SKY and I exchanged mutual
reports but because we were so close these were not included in the study,
although they provided substance to a question about S/N reports discussed
below. The original design was that in addition to concentrating exchanges into
relatively short periods with rapid alternation between the two modes,
I would put the results on Excel, sort by signal strength and see if there was a
difference in decoding results by signal strength.

The first thing to say is I am very impressed with
the capabilities of both -A and -B. Thanks, Joe--another triumph! Many of these
test sequences were above 1000 miles, and despite the fact that the times were
not of high M/S activity, a high percentage of sequences on each decoded with
relatively little garbage or garbled results. Watching the incoming signals from
others, I noted that there were a number of decodes without visible or
audible pings, and some decodes where there was no visible pattern on the screen
or a very subdued visible ISCAT pattern that often went on for
quite a number of seconds. Airplane scatter would also appear to be a
possibility here, as the EU-NA flightpaths cross our horizon above the Bay
of Fundy and Gulf of Maine. I would be interested in others' impressions of what
propagation modes are likely involved.

I applied some predetermined standards to which
all.txt lines to include in the report (resulting in the selection of a total of
87 ISCAT-As and 100 ISCAT-Bs, partly due to when stations started or stopped
transmitting or recording). Perfect decodes were valued at 1.0, single errors
were valued at 0.5, and everything else was zero. Anyone wishing to receive the
Excel summary or the individual all.txt files should email me and it is very
likely that someone more advanced technically can derive additional or more
accurate results and interpretations.

Based on weighted decodes divided by numbers of
sequences, -A scored 59.2% and -B scored 71.5%. This may be what K1JT would have
predicted , but I believe he has also mentioned a potential 1dB sensitivity
difference in favor of -A, so I was surprised.

However, early on and consistently, it appeared
that the ISCAT-A read-out was reporting S/N levels averaging several dBs
below those of -B. This led me to question whether -A was fully driving the
transmitted signal, but in fact my output meter was showing exactly the
same on both. Subsequently, VE1SKY and I pointed antennas at each other and
conducted a specific test of S-meter differences and the software's S/N
reports and found no discernible shift in S-meters between -A and -B but a
consistent 3 to 4 dB (mostly 4dB) difference in the ISCAT-reported S/N reports.
This led me to wonder if the -A decoding software was performing correctly, as I
had assumed -A would report a more favorable comparison to noise level than the
broader -B digital passband. I wrote K1JT about this and he promptly
replied:

"Hi Jim,

"Something may well be amiss with the reported
S/N values in ISCAT-A relative to ISCAT-B. The "A" mode was cobbled together
rather quickly when the Aussies were doing airplane scatter at 10 GHz, and I
don't remember doing any exhaustive tests of the measured parameters. You'd
better do your comparative analysis of decodes-vs-SNR treating SNR as consistent
within each submode, but possibly not meaningful between modes.

"The
important thing, after all, is whether it decoded or not. Reported signal
parameters are of secondary importance.

"-- Joe, K1JT"

I still couldn't resist looking at the decode
scores (1, 0.5 and 0) for the two modes individually at a reported -20 S/N.
ISCAT-A scored 9/30 or 30% and ISCAT-B scored 9.5/37 or 25.7%. I'm still not
sure how to interpret this given the differences in reported S/N values, and as
Joe commented, these comparisons between modes may not be valid.

I am looking forward to using ISCAT in next
summer's Transatlantic Es openings and may again ask other 6M users to
participate in a study. Another potential use is in Transatlantic aurora
periods--both VE1SKY and I have been copied in Finland on 6M ISCAT doing a bank
shot off the aurora. In the meantime, I encourage others to conduct similar (or
probably more technically sophisticated) A/B tests of their own and share their
results with the community. If they care to and find it feasible, they would be
welcome to combine the data we have collected (probably their own filtering of
the raw data) with further collections..

73,

Jim, VE1JF

Bruce Brackin

Jim, Roger, et.al. If you compute the simple 95% confidence intervals for the proportion of A and B decodes, there s no statistical or practical difference.

Message 2 of 5
, Oct 8, 2012

Jim, Roger, et.al. If you compute
the simple 95% confidence intervals for the proportion of A and
B decodes, there's no statistical or practical difference.
Given the small sample size (30 and 37), you'd expect fairly
wide error bars. Probably the luck (or unlucky) of the draw.

Keep up the good work.

Bruce N5SIX

"I still couldn't resist looking at the decode scores (1, 0.5
and 0) for the two modes individually at a reported -20 S/N.
ISCAT-A scored 9/30 or 30% and ISCAT-B scored 9.5/37 or 25.7%.
I'm still not sure how to interpret this given the differences
in reported S/N values, and as Joe commented, these comparisons
between modes may not be valid."

Barry Garratt

Bruce et al, Actually I wouldn t expect wide errors at all Bruce even with the small sample size. Remember the testing was all being done basically with

Message 3 of 5
, Oct 8, 2012

Bruce et al,

Actually I wouldn’t expect wide errors at all Bruce even with the small sample size. Remember the testing was all being done basically with meteors and the mode isn’t designed for that. Hence both modes performed equally poorly. Conversely FSK441 works very well for MS because that’s what it was designed for. If the tests were run against FSK441 I would expect to see a wide margin between the modes but not between the two Iscat modes.

Jim, Roger, et.al. If you compute the simple 95% confidence intervals for the proportion of A and B decodes, there's no statistical or practical difference. Given the small sample size (30 and 37), you'd expect fairly wide error bars. Probably the luck (or unlucky) of the draw.

Keep up the good work.

Bruce N5SIX

"I still couldn't resist looking at the decode scores (1, 0.5 and 0) for the two modes individually at a reported -20 S/N. ISCAT-A scored 9/30 or 30% and ISCAT-B scored 9.5/37 or 25.7%. I'm still not sure how to interpret this given the differences in reported S/N values, and as Joe commented, these comparisons between modes may not be valid."

Leigh Rainbird

G day, I downloaded the new version over the weekend and did some testing with ISCAT B and JT65a on the 6m band. I should have spent some more time on it and

Message 4 of 5
, Oct 8, 2012

G'day,

I downloaded the new version over the weekend and
did some testing with ISCAT B and JT65a on the 6m band. I should have spent some
more time on it and tried ISCAT A. I may do that still.

But just briefly, I found that ISCAT B worked OK
for meteors but not so good for direct, constant type signals like tropo.

The meteors had to be quite long duration to obtain
a full decode with no errors, short pings weren't good enough. The tropo signals
needed to become quite strong before any real info could be obtained from the
message being sent.

These tests were with stations 770 and 400 km
ranges.

On the shorter 400 km paths, I tried for maybe an
hour with VK3SMC and got nothing decoded despite being able to hear the signal
by ear at times. This is mainly on tropo. We decided to try JT65a in comparison,
and from the very first transmissions we had decoded right away and completed in
about 5 mins.

So from this test Ive currently concluded that on
6m paths via E or F layer, JT65a wins hands down over ISCAT.

I will find another station and test ISCAT A and
see how it goes for us.

Actually I wouldnt expect wide errors
at all Bruce even with the small sample size. Remember the testing was all
being done basically with meteors and the mode isnt designed for that. Hence
both modes performed equally poorly. Conversely FSK441 works very well for MS
because thats what it was designed for. If the tests were run against FSK441
I would expect to see a wide margin between the modes but not between the two
Iscat modes.

Jim,
Roger, et.al. If you compute the simple 95% confidence intervals for the
proportion of A and B decodes, there's no statistical or practical
difference. Given the small sample size (30 and 37), you'd expect fairly
wide error bars. Probably the luck (or unlucky) of the draw.

Keep
up the good work.

Bruce N5SIX

"I still couldn't resist
looking at the decode scores (1, 0.5 and 0) for the two modes
individually at a reported -20 S/N. ISCAT-A scored 9/30 or 30% and ISCAT-B
scored 9.5/37 or 25.7%. I'm still not sure how to interpret this given the
differences in reported S/N values, and as Joe commented, these comparisons
between modes may not be valid."

Jim and Hannelore Fisher

Some additional thoughts on yesterday s report, prompted by several emails and a conversation with VE1SKY: o My personal focus is not on evaluating FSK441

Message 5 of 5
, Oct 9, 2012

Some additional thoughts on yesterday's report,
prompted by several emails and a conversation with VE1SKY:

o My personal focus is not on
evaluating FSK441 vs. ISCAT. I assume that if M/S pings are all that are going
on FSK441 will outperform ISCAT for the distances feasible for M/S. (My
impression is that more than one propagation mode was weakly active during these
tests.) My 6M goal is DXCC (will complete my 10BDXCC since I just wrapped up my
9th band), and all DXCC entities within M/S reach for me are already worked or
are easily worked by other modes. Therefore, having already used JT65A for EME
and terrestrial DX and ISCAT-B to work EU, I am seeking to get a feel for how -A
and -B perform under different circumstances and be ready for further
studies and hopefully a bunch of new countries in EU, AF and maybe AS
during the next transatlantic Es season.

o VE1SKY and I have both been
heard via aurora in EU on ISCAT (-B at the time) and I am hopeful -A will be at
least as good (perhaps in part due to greater tolerance for Doppler as well as
the additional 1dB sensitivity?) and will result in some additional DX
possibilities when the moon is down and Es and F2 are quiet. Using some of
the near-real-time aurora images on Internet, I expect us to be able to refine
our aiming techniques and understanding and use ON4KST 50-70 to alert potential
QSO partners in EU when it may be useful to look for us. As a fellow HF
contester used to say, you aren't loud if you aren't on.

o Others may wish to test FSK441
vs. ISCAT-A and/or B, but I am particularly hopeful that European stations will
continue their gradual acceptance of ISCAT as an improvement over JT6M. Perhaps
somebody who considers this still an unresolved issue would like to design and
conduct a compressed test rapidly alternating JT6M with ISCAT-A and/or -B during
transatlantic openings. I take K1JT's word that ISCAT is better than JT6M and am
mainly interested in deciding when to use -A or -B or JT65A.

I am hoping some additional members of our 6M
digital community will either replicate or hopefully improve our test. For one
thing, I am sure some others with deeper technical understanding can make better
use of the complete raw data collected including all.txt data columns other than
S/N. I still feel that the compressed alternations and various stations' all.txt
records including segment-by-segment S/N figures provided me with a better feel
than a series of individual QSOs spread over time.

73,

Jim VE1JF

Your message has been successfully submitted and would be delivered to recipients shortly.