This is a reformatted-for-wiki version of ~/BobKPNO/src/README.2004,
which details my efforts on getting the SII calibration into shape in
June 2004. Some technical details of formatting issues and gotchas are
described in EmacsWikiFormat
More recent work is detailed in OrionSiiTere.
Contents:
Things still to do for teresa:
Still haven't done points (1-4) above. However, have made progress on a lot of other fronts.
A. Velocity calibration
NB. We still need to do a reality check with the horizontal slits - probably best to allow slit-averaged upeak to be a low-order polynomial in RA, finding the best-fit coefficients to agree with the horizontal slit.
Smoothing slows things down a bit, but only the first time it is run for a particular slit since I write a fits file of the smoothed spectrum, which will be reused the next time round. If the parameters change, you need to remove these files to force them to be regenerated (same as with the background subtraction). Note that I had big problems with array aliasing between module variable fitsimage and the local argument to the smoothing routine since the routine calls fitsread, which tries to reuse fitsimage - solved by not passing outer fitsimage directly, but copying it to another array first.
Some programs I have run along the way:
# generates the 1d spectra of the whole slits (sans high-velocity parts)
for i in $(seq -w 1 109) ; do
printf '%s\n' KPNOsiil jw4-tere.csv $i \
y '0 0' 2 1 0 y \
3 '-50 -20' '10 40' 0.05 3 '-150 150' \
siil-sp1d-$i n n siil-sp1d-$i.png/png \
| src/wspecplot1d
done
# same but without using the ufiddles: this can then be used by
# sii_shift.f90 in order to calculate them
for i in $(seq -w 1 109) ; do
printf '%s\n' KPNOsiil jw4-tere.csv $i \
n '0 0' 2 1 0 y \
3 '-50 -20' '10 40' 0.05 3 '-150 150' \
siil-sp1d-nf-$i n n /null \
| src/wspecplot1d
done
i. The rectification problem. Is it worth looking for the best polynomial (maybe straight line) that would bring the 6716 and 6731 vpeaks into alignment, and then applying it to the 6716 images (some sort of interpolation shift on each row).
OI skyline is now sorted! Still a few problems something odd with the flux normalizations in wisomom_raw. The 70 micron slit comes out faint.
Need to fix normalization of smoothing. Done. It was actually a problem with the order things were done in wisomom_raw, resulting in fluxfiddle being applied twice to smoothed images. The order of application has now been made the same in all programs:
Have now generated the oi isovelocity images in 4km/s intervals using:
for i in $(seq -12 4 52); do
printf '%s\n' SPMoi jw4-sky.csv $i $((i + 4)) 1 | src/wisomom_raw
done
for i in $(seq -12 4 52); do
printf '%s\n' \
oi_$(printf '%+3.3i%+3.3i' $i $((i + 4))).wisomom-sum-raw \
1 01 |./src/wisomom_smooth
done
And made RGB images using oi-makergb.sh - they look really good!
Looking forward to doing the same for siii and sii.
for i in $(seq -24 4 52); do printf '%s\n' KPNOsiil jw4-new.csv $i $((i + 4)) 1 | src/wisomom_raw; done for i in $(seq -24 4 52); do printf '%s\n' siil_$(printf '%+3.3i%+3.3i' $i $((i + 4))).wisomom-sum-raw 1 01 |./src/wisomom_smooth ; done
Big problem with wisomom_raw and sii. Some of the slits (70 micron) come out way too faint. Fixed.
More progress made. Teresa has further tweaked the siil files: jw4-tere.gnumeric. Main change is to eliminate some of the KPNO slits, which turned out to be duplicates of one another. This is a big help. Am now running the same with her new version of her siis files. There may be some remaining bad slits that can be further pruned.
Did the stuff with the f547m filter to give the scattered continuum. Still need to produce good images of this for printing.
Tere's siis file was bad (jw4-tere). Have added ufiddles from my jw4-new and made fluxfiddle same as for siil. The extra correction to the KPNO siis slits is now done via night_info (factor 0.88 for night = {1,2})
Ran this with limits -100+100 and compared with siil. Some slits are funny:
#79 ::
A lot of problems might be solved by averaging the y-pixels.
Yes, it solves some
#52 has a bad reflection: was going to remove it, but others have bad reflections too - will sort it by defining bad row ranges.
printf '%s\n' KPNOsiil jw4-tere.csv -100 100 0.25 | src/wisomom_raw
printf '%s\n' siil_-100+100.wisomom-sum-raw 0 600 | src/isopgm
printf '%s\n' KPNOsiis jw4-tere.csv -100 100 0.25 | src/wisomom_raw
printf '%s\n' siis_-100+100.wisomom-sum-raw 0 400 | src/isopgm
printf '%s\n' siis_-100+100.wisomom-sum-raw \
siil_-100+100.wisomom-sum-raw \
ratio_-100+100.wisomom-sum-raw \
| src/isoratio
printf '%s\n' ratio_-100+100.wisomom-sum-raw 0.3 1.2 | src/isopgm
xv -raw -drift -4 0 -expand 10:5 \
{siil,siis,ratio}_-100+100.wisomom-sum-raw.pgm &
printf '%s\n' siil_-100+100.wisomom-sum-raw 1 01 | src/wisomom_smooth
printf '%s\n' siis_-100+100.wisomom-sum-raw 1 01 | src/wisomom_smooth
printf '%s\n' siis_-100+100.wisomom-sum-01 \
siil_-100+100.wisomom-sum-01 \
ratio_-100+100.wisomom-sum-01 \
| src/isoratio
printf '%s\n' siil_-100+100.wisomom-sum-01 0 600 | src/isopgm
printf '%s\n' siis_-100+100.wisomom-sum-01 0 400 | src/isopgm
printf '%s\n' ratio_-100+100.wisomom-sum-01 0.3 1.2 | src/isopgm
xv -raw -drift -4 0 -expand 4 {siil,siis,ratio}_-100+100.wisomom-sum-01.pgm &
printf '%s\n' KPNOsiil jw4-tere.csv -10 40 0.25 | src/wisomom_raw
printf '%s\n' siil_-010+040.wisomom-sum-raw 0 600 | src/isopgm
printf '%s\n' KPNOsiis jw4-tere.csv -10 40 0.25 | src/wisomom_raw
printf '%s\n' siis_-010+040.wisomom-sum-raw 0 400 | src/isopgm
printf '%s\n' siis_-010+040.wisomom-sum-raw \
siil_-010+040.wisomom-sum-raw \
ratio_-010+040.wisomom-sum-raw \
| src/isoratio
printf '%s\n' ratio_-010+040.wisomom-sum-raw 0.4 1.1 | src/isopgm
xv -raw -drift -4 0 -expand 10:5 \
{siil,siis,ratio}_-010+040.wisomom-sum-raw.pgm &
printf '%s\n' siil_-010+040.wisomom-sum-raw 1 01 | src/wisomom_smooth
printf '%s\n' siis_-010+040.wisomom-sum-raw 1 01 | src/wisomom_smooth
printf '%s\n' siis_-010+040.wisomom-sum-01 \
siil_-010+040.wisomom-sum-01 \
ratio_-010+040.wisomom-sum-01 \
| src/isoratio
printf '%s\n' siil_-010+040.wisomom-sum-01 0 600 | src/isopgm
printf '%s\n' siis_-010+040.wisomom-sum-01 0 400 | src/isopgm
printf '%s\n' ratio_-010+040.wisomom-sum-01 1.1 0.4 | src/isopgm
xv -raw -drift -4 0 -expand 4 {siil,siis,ratio}_-010+040.wisomom-sum-01.pgm &
isopgm called with range {10,30}, so each 1/100 unit is 0.2 km/s
Corrected in jw4-tere 19 Jun 2004
isopgm called with range 10 40, so sigma = 10.0+0.3*I, where I is the pixel value measured in xv. siil is slightly narrower so differences show up more there.
Decided to remove all 4.
small differences between
group 1: 20,22,24,25,26 - SPM 70 (night 5)
group 2: 17,18,19,21,23,27 - SPM 150 (night 3)
those in group 1 are too narrow, so we are obviously not convolved enough still
relevant image pixels are 16, 22, corresponding to fwhm of 14.8, 16.6.
Subtracting in quadrature gives 7.52, which divided by 2.355 gives sigma = 3.2. We already had 4.8 as the smoothing sigma: this suggests it should be 5.75. That went too far the other way, especially for siis. Change to 5.4?
There is little evidence of problems between KPNO and SPM - leave those alone
for i in $(seq -24 4 56); do printf '%s\n' KPNOsiil jw4-tere.csv $i $((i + 4)) 0.25 | src/wisomom_raw; done for i in $(seq -24 4 56); do printf '%s\n' siil_$(printf '%+3.3i%+3.3i' $i $((i + 4))).wisomom-sum-raw 1 01 |./src/wisomom_smooth ; done for i in $(seq -24 4 56); do printf '%s\n' KPNOsiis jw4-tere.csv $i $((i + 4)) 0.25 | src/wisomom_raw; done for i in $(seq -24 4 56); do printf '%s\n' siis_$(printf '%+3.3i%+3.3i' $i $((i + 4))).wisomom-sum-raw 1 01 |./src/wisomom_smooth ; done for i in $(seq -24 4 56); do vv=$(printf '%+3.3i%+3.3i' $i $((i + 4))) printf '%s\n' siis_$vv.wisomom-sum-01 siil_$vv.wisomom-sum-01 ratio_$vv.wisomom-sum-01 | src/isoratio printf '%s\n' ratio_$vv.wisomom-sum-01 1.1 0.4 | src/isopgm convert ratio_$vv.wisomom-sum-01.pgm ratio_$vv.wisomom-sum-01.png rm ratio_$vv.wisomom-sum-01.pgm done id=wisomom-sum-raw for i in $(seq -24 4 56); do vv=$(printf '%+3.3i%+3.3i' $i $((i + 4))) printf '%s\n' siis_$vv.$id siil_$vv.$id ratio_$vv.$id | src/isoratio printf '%s\n' ratio_$vv.$id 1.3 0.5 | src/isopgm convert ratio_$vv.$id.pgm ratio_$vv.$id.png rm ratio_$vv.$id.pgm done
How to display all the density images:
xv -raw -drift -4 0 -expand 8:4 $(for i in $(seq -24 4 56); do printf 'ratio_%+3.3i%+3.3i.wisomom-sum-raw.png ' $i $((i + 4)); done)&
There are various problems with these: many could be due to the fact that our velocity limits in wisomom_raw are too sharp, which will give jittering. Yes: this makes things a lot better.
Remaining problems are the gradient in velocity and the spurious siis spm component.
Will fix these with a program that applies a shear to the PV images. This will be done to the fits files themselves to make it independent of the rest of the processing chain.
Program written: pvshear.f90. Tried it out with an exaggerated shear of 10 km/s (over full length of slit) and it seems to work fine. If given a positive shear it shifts the top end of the slit to the blue and the bottom end to the red.
Returning to the peak difference maps I made earlier:
printf '%s\n' siis_-010+040.wisomom-peak-raw \
siil_-010+040.wisomom-peak-raw \
diff_-010+040.wisomom-peak-raw \
| src/isodiff
printf '%s\n' diff_-010+040.wisomom-peak-raw -5 5 | src/isopgm
Looking at the jw6 spectra we get pixel values at the quartiles of:
JW6 JW3 JW3-W top 65-85 2 60 1.1 50 0.1 midtop 56 0.7 56 0.7 40 -0.9 middle 51 0.2 47 -0.2 30 -1.9 midbot 49 0.0 39 -1. 22 -2.7 bot 25-35 -2 30 -1.9 10 -3.9
| . | JW6 | . | JW3 | . | JW3-W | . |
|---|---|---|---|---|---|---|
| . | pix | v | pix | v | pix | v |
| top | 65-85 | 2 | 60 | 1.1 | 50 | 0.1 |
| midtop | 56 | 0.7 | 56 | 0.7 | 40 | -0.9 |
| middle | 51 | 0.2 | 47 | -0.2 | 30 | -1.9 |
| midbot | 49 | 0.0 | 39 | -1. | 22 | -2.7 |
| bot | 25-35 | -2 | 30 | -1.9 | 10 | -3.9 |
velocity is 0.1*(pix-49), giving the second column above, so full range would suggest shear of 4 but half range, only shear of 1.5 - in other words the error seems to be non-linear, getting worse at the ends. The sense of the error is that siis is redder than siil at the top of the slits, so we would have to apply a negative shear to siil or positive shear to siis. Looking at the individual vpeak image, I think it would be best to apply the shear to siis.
The jw3 slits imply a shear of about 3 for the eastern bunch and about 4 for the western bunch (see above). The western bunch are also need about a 2 km/s shift between siis and siil. This is also obvious in the ratio maps. Will fix that next.
# JW6 bunch: shear of 2 km/s
for i in $(seq 1 10) 12 13 14; do
printf '%s\n' KPNOsiis jw4-tere.csv $i 2.0 | src/pvshear
done
# 2002 bunch: shear of 3 km/s
for i in $(seq 55 70) 74; do
printf '%s\n' KPNOsiis jw4-tere.csv $i 3.0 | src/pvshear
done
# JW3 bunch: shear of 4 km/s
for i in $(seq 81 84) 87 $(seq 89 93); do
printf '%s\n' KPNOsiis jw4-tere.csv $i 4.0 | src/pvshear
done
This worked well, except for jw3, which had gone too far - try 3 km/s instead:
# JW3 bunch: shear of 3 km/s
rm KPNOsiis/jw3*-smooth.fits
for i in $(seq 81 84) 87 $(seq 89 93); do
printf '%s\n' KPNOsiis jw4-tere.csv $i 3.0 | src/pvshear
done
That is now much better. Remaining issue is the shift between siis and siil for some slits. Some jw3 slits have siis-siil = -2 km/s and it looks like it is the siis that should be moved to the red. ##81,82,83 have now been changed.
Zooming in on finer velocity differences...
printf '%s\n' diff_-010+040.wisomom-peak-raw -2 2 | src/isopgm
we see things are much better.
Still problems with some slits: #84 looks like it shouldn't have been sheared. Fixed that.
Now looking at general problems that affect both siis and siil.
Helped a little bit.
Ran a new version of the thin velocity slices. We still have problems with the ratios.
Made new version of the mean velocity images and diference between siil and siis:
printf '%s\n' \
siis_-010+040.wisomom-mean-raw \
siil_-010+040.wisomom-mean-raw \
diff_-010+040.wisomom-mean-raw | src/isodiff
printf '%s\n' siis_-010+040.wisomom-mean-raw 15 30 | src/isopgm
printf '%s\n' siil_-010+040.wisomom-mean-raw 15 30 | src/isopgm
printf '%s\n' diff_-010+040.wisomom-mean-raw -1 1 | src/isopgm
xv -raw -drift -4 0 -expand 8:4 \
{siis,siil,diff}_-010+040.wisomom-mean-raw.pgm \
siil_-010+040.wisomom-sum-raw.pgm&
The problems evident in the difference map are the same as those visible in the ratio maps.
Start with cases where difference shows something: Each pixel unit = 0.02 km/s
Done that - now for things that affect both siis and siil: on the individual mean images, each unit is 0.15 km/s
show general darkness of 12-18 units = 1.8->2.7
Try adding 2km/s to all of them - didn't work too well: instead made the change gradually at each end.
This looks very much like a high-ionization line, such as siii or oiii.
I've tried taking scaled versions of the emission in the far wing (56->100) and subtracting it from the 3 affected frames (40,44,48):
printf '%s\n' siis_+040+044.wisomom-sum-raw \ siis_+056+100.wisomom-sum-raw 0.4 siis_+040+044.correct | src/isodiff printf '%s\n' siis_+044+048.wisomom-sum-raw \ siis_+056+100.wisomom-sum-raw 0.43 siis_+044+048.correct | src/isodiff printf '%s\n' siis_+048+052.wisomom-sum-raw \ siis_+056+100.wisomom-sum-raw 0.5 siis_+048+052.correct | src/isodiff printf '%s\n' siis_+040+044.correct 0 20 | src/isopgm printf '%s\n' siis_+044+048.correct 0 10 | src/isopgm xprintf '%s\n' siis_+048+052.correct 0 6 | src/isopgm id=wisomom-sum-raw for i in $(seq 40 4 48); do vv=$(printf '%+3.3i%+3.3i' $i $((i + 4))) printf '%s\n' siis_$vv.correct siil_$vv.$id ratio_$vv.$id | src/isoratio printf '%s\n' ratio_$vv.$id 1.3 0.5 | src/isopgm convert ratio_$vv.$id.pgm ratio_$vv.$id.png rm ratio_$vv.$id.pgm done
This doesn't quite work since we end up oversubtracting a bit. There might be a way to get this to work though. On reflection, maybe the red wing is actually a hydrogen line. That would explain why it is so broad.
...or it could be siii
Gave the removal of this contamination as a task for Teresa