Data Diary: Flagging

I’ve been done with the first part of the processing for a while, and now it’s time for the final step to become Level 1 data – flagging. Basically, this is the step where I’m “cleaned up” for future work. This step is somewhat time consuming, which is why I’ve had to wait to get here – the people doing the flagging have lots of other work to help keep them busy and haven’t been able to do the flagging as fast as I’m being observed at the telescope.

The main reason for having to go through this step in the reduction process is that people use radio waves for communication. This means that I am full of human-generated radio frequency interference (RFI) in addition to astronomical signals. RFI is not of interest for science and it can cause problems in later reduction steps, so it needs to be ‘flagged’ away. In order to do this, an ALFALFA team member sits down and looks over me and places boxes to mark all the spots that are bad (contain RFI) and should be ignored in future data reduction steps. For each ‘drift’ I contain, there are seven beams with two polarizations, for fourteen total images that need to be examined by an astronomer. Depending on how long a block of time was assigned the night I was observed, I can consist of anywhere from 20-50 drifts, so this becomes a lot of images to be examined. Below is a screenshot of an example image from this process. This is one of the fourteen images for a single drift. (Click for a larger image.)

View of data flagging

This is a ‘good’ picture of the flagging process – I’m being very well behaved in this data set. (There are plenty of examples where I’m not so nice that I’m sure will be shared in the future.) The horizontal axis is channel number, or frequency. The left end is the lowest frequency observed and the right end the highest. The vertical axis is time, from 0 to 599 seconds, encompassing the length of one drift. (For the ALFALFA drift-scan survey mode, time corresponds to a position in right ascension and the known beam position for the night gives a declination.) The two boxes on the edges (numbers 0 and 1) are the result of the bandpass edges having a lower response (see the first picture in this post), which generates poor data quality, and not of human-generated RFI. In this picture, there’s actually very little RFI present. You can see a skinny box to the left labelled with the number 5. This box is where the FAA radar for the San Juan airport shows up. It is barely present here, which means that the airport has decided to use their radar at a lower frequency, outside of the range observed for ALFALFA. The other prominent feature is the bright signal to the right. This isn’t flagged with a box because it’s not RFI – it’s our Galaxy! This is a real signal, so I don’t want people to remove it from me. While this is a good picture of me, it’s also a little boring; you can see bright galaxies at this step, but there aren’t any in this picture.

Now that I’ve been flagged, I’m Level 1 data. Next up, I start processing to become Level 2 data. These steps will be about creating ‘grids’ – this is the format I’m put into for source extraction, the end goal.