Before releasing the astropulse application Eric had to add a couple fields to the result tables in the science database that are now necessary. These are large fields, and it's taking informix forever to update the table. The job was started 24 hours ago and is still chugging along. I guess it doesn't help that the assimilator queue is still rather large (though it is draining). So the release is delayed until this job finishes.

The radar blanking stuff I was whining about the other day has nothing to do with the astropulse release, in case there was some confusion about that. Josh and I are working on two completely separate and different forms of radar mitigation. Mine is to better clean up data before any splitting/analysis, Josh's is to deal with radar that squeaked through the first pass and made it all the way to the client. The good news is that I made significant progress on mine today.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Before releasing the astropulse application Eric had to add a couple fields to the result tables in the science database that are now necessary. These are large fields, and it's taking informix forever to update the table. The job was started 24 hours ago and is still chugging along. I guess it doesn't help that the assimilator queue is still rather large (though it is draining). So the release is delayed until this job finishes.

The radar blanking stuff I was whining about the other day has nothing to do with the astropulse release, in case there was some confusion about that. Josh and I are working on two completely separate and different forms of radar mitigation. Mine is to better clean up data before any splitting/analysis, Josh's is to deal with radar that squeaked through the first pass and made it all the way to the client. The good news is that I made significant progress on mine today.

- Matt

Do you use any data substitution on server-side data cleanup (I mean, do you insert something in data array instead of removed radar pulses ? )
If yes, is it possible to mark these "blanked" areas in task header so client could know what areas are "true" ones and what contain some substituted stub data? Some list of indexes of blanked regions for example...

Do you use any data substitution on server-side data cleanup (I mean, do you insert something in data array instead of removed radar pulses ? )
If yes, is it possible to mark these "blanked" areas in task header so client could know what areas are "true" ones and what contain some substituted stub data? Some list of indexes of blanked regions for example...

Do you use any data substitution on server-side data cleanup (I mean, do you insert something in data array instead of removed radar pulses ? )...

What angle are you going for?

My scraps of understanding from previous posts are that there is a radar detector on the telescope at Arecibo. The output from that is recorded on a spare data recorder channel in parallel to the 14 streams of radio data. Matt's code then substitutes psuedo-random random noise into the data stream to replace the radar mess whenever the radar has been detected. The psuedo-random noise then goes innocently through the rest of the data processing as though Arecibo hasn't heard anything other than the usual background noise.

The radar source is a nearby military radar. The radar is kindly blanked for the arc sweeping Arecibo. However, that doesn't stop reflections that bounce off aircraft (or other local structures?) splattering Arecibo...

Perhaps:

Also set aside an air exclusion zone around Arecibo?

Change the radar frequency to something much higher than the Arecibo receivers? (Or will the radar ERP saturate anything more sensitive than an AC power rectifier diode?!)

An interesting question is whether Matt has had to add any cleverness to avoid a signal spike being detected by the FFT for the transition to/from the psuedo-random noise section...? I would expect the FFT to "see" the transition in the noise "texture".

Happy crunchin',
Martin
____________
See new freedom: Mageia4Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Can anyone comment on the mathematical correctness of adding pseudorandom noise?

It seems to me that doing so would add pseudo-energy to the broadband power spectrum. This has the effect of reducing S/N a bit. For example, if 50% of the time slice had radar noise and it was replaced with random noise, then 50% of the power spectrum, across frequencies, would be artificial. That just doesn't sound right.

Wouldn't just replacing the time with a constant, or if processed after the fact replace the radar blank period with the signal average for the time slice, then this would only add noise at the DC level and then would be ignored.

I'm sure Matt and others have thought about this, so I'm just looking to understand the analysis a bit.

Thanks for the postings here. It's nice to know there is a human or two somewhere in the mix watching over, updating and maintaining things, and it's also nice to have someone crack the door so we can peek in.

Change the radar frequency to something much higher than the Arecibo receivers? (Or will the radar ERP saturate anything more sensitive than an AC power rectifier diode?!)

If it is a ground based radar it likely will saturate even the power diode.

Heck, when I worked on OTH-B ... when we radiated on a frequency no one in the world could use that slot ... the good news is that it was low frequency so most people could not care less ... but it was funny working with the Radar Control guys ... we would suggest a frequency band and they would find a clear spot and then "park" the radar there with short scans before changing the whole system over ... they kinda played nice by reserving the space before stepping on anyone that had had an idea of coming up on that spot ...