Summary of MDdeeptest_pixrej_20140403

For a while it has been bothersome to find junk in the MD stacks through just simple visual inspection as would not be expected in N=60 (refstack) and definitely not for N>>60 (deepstack), and they can also come and go depending on the inputs used. This seems to be at least configs needing tuning, at worst code bugs.

A related issue also to be addressed is the ratty edge of the MD stacks due to poorer edge chips and smaller number of input pixels. Possible alternatives for a solution are

hard cut by radius field by field and filter -- difficult to implement

combine MD+3PI to help improve by including better 3PI pixels -- non-trivial setup and unknown config behavior for both types of data sets (MD and 3PI)

only include pixel if N input > some minimum, similar to the SAFE (N=2) option but with a larger N minimum -- edge may vary greatly between filters

often edge is dominated by SUSPECT mask pixels, set those to bad and reject -- potential to impact entire skycell (may be good since SUSPECT may actually be bad)

Full Field Test Samples

All stacks should be available via the Postage Stamp Server (PSS), if large or full sets are desired then they can be found as usual on the datastore under ps1-deeptest with the data_group name the same as the label. Warp examples should be obtainable via the PSS and will not be pushed to the distribution server.

MD04 PV2 warps -- the full test runs below used all available PV2 through the end of the survey (MD.PV2)

warp label: MD04.pv2.20130903 -- all filters

not all examples below use the full set of warps, many use only the same warps that went into MD04.cut2009deeptest.20131103 in order to provide a similar stack for comparison.

3PI exposures overlapping with MD04 -- span Apr. 2009 to Feb. 17 2014, however any exposures before June 2010 will be excluded just like the MD04 exposures (i.e., cut2009 condition described at MD04.pv2_stacktest.20130903#DeepStacks)

warp label: MD04.test3pistack.20140217 -- g-band only

(need to add total and used numbers)

MD04+3PI overlap test with just bug fixes to compare the edge quality -- will include additional PV2 warps through end of survey for MD04 and similarly 3PI though Feb. 2014

label: MD04_3PI.cut2009teststack.20140404 -- g-band only

MD04+3PI overlap test with bug fixes and config modifications to compare the edge quality -- will include same available warps as prior test

label: MD04_3PI.cut2009teststack.20140411 -- again g-band only

THRESHOLD.MASK 0.01

COMBINE.REJ 2.5

will take work to evaluate impact of config modifications between the 3PI and MD here.. -- likely will have to consider this option after much more testing

MD04 with bug fixes, config modifications, N>20 inputs per pixel to clean up the field edge -- can compare to MD04.cut2009deeptest.20131103 since PV2 inputs limited by date here through end of Sept. 2013

MD09 example as a shifted center field case -- MD09.PV1_deepstack_tests and manual re-test showed a large amount of junk, so also set SUSPECT input pixels as BAD pixels, edge properties unknown so keep pixels N>10 inputs (NMINPIX)

label: MD09.cut2009n10suspectbaddeeptest.20140427 -- g-band only, full PV2 set after 6/2010 like cut2009

THRESHOLD.MASK 0.3

COMBINE.REJ 3.5

NMINPIX 10

MASK.VAL MASK.VALUE,CONV.BAD,GHOST,SUSPECT

MD09 example to try exclude extra junk -- NMINPIX now 30, adding a MASK.BLANKBORDER to mask the edge of the skycell since it goes through essentially no rejection process and often a pile of junk. Is set 20 pixels since the border is set in code to be KERNEL.SIZE in the convolution and overlap setup. No pixels are lost since the MD skycells have an overlap of 320 pixels, now they will overlap by only a mere 300 pixels.

label: MD09.cut2009n30th01suspectbadborder.deeptest.20140502 -- g-band only, full PV2 set after 6/2010 like cut2009

THRESHOLD.MASK 0.1

COMBINE.REJ 3.0

NMINPIX 30

MASK.VAL MASK.VALUE,CONV.BAD,GHOST,SUSPECT

MASK.BLANKBORDER 20

MD04 example with changes made in the MD09 above and with imposed smaller target PSF -- limit the target PSF for convolution to check if smaller THRESHOLD.MASK has any problems