How To Break Expensive Stuff

Despite the announcement yesterday that Sheffield is the happiest city in the UK, I still haven’t quite got the better of my positive emotions. It’s hard to pick out the good points from the last couple of weeks, especially after eating the worst dinner since I became a middle-class, well-educated, free-thinking adult (honey on toast, soya mince with barbecue sauce, olives and jelly babies). That said, I have made *some* progress, and I’m still managing to make it into work before 10am every day, so it can’t be too bad.

Since my last post about the problems with ULTRASPEC, it’s been “fixed” and broken again another two times. Being taken apart by the team at ATC in Edinburgh turned out to be quite a positive event for the whole project. The engineers discovered immediately after opening the lid what the original problem was – the tiny cables which operate the heater inside the cryostat had melted because the current running through them was too high. This caused a short circuit between the wires which measure the temperatures, so the nonsensical temperature readings I mentioned before were just that: rubbish.


Sooty, blackened wires which caused the original problem!

This in itself was a relatively easy fix (they simply replaced the wires with stronger, higher grade ones, and re-programmed the heater controller to chill out a bit), but it’s far better that this happened here and now, and not in 6 months time when the whole thing is all set up and ready to go in Thailand. In addition, they found that the chip was actually seated 0.5mm lower than originally thought – meaning all the lens designs would have been calculated incorrectly. Though 0.5mm doesn’t seem like very much, it would totally screw up the image quality of the camera. So they fixed that too, put it all back together, and sent it back down to Sheffield. Overall, quite a productive visit then…

Or so we though. We got it all set up again, with the vacuum pumped and the cryostat cooled, by Friday afternoon. It was at this point that we discovered a new and entirely different problem. We found that the chip was measuring very high dark current (thermally excited electrons within the chip which are *not* caused by light). We also saw a bizarre structure in the images, which looked like another problem with the electronics, as well as the dark current issue.


Strange structure on the left side of the image, suggesting a serious problem with the read-out electronics.

The strange structure appeared to be that pixels in the lower left corner of the image were magically replaced in the top left corner, or that entire columns seemed to be shifted upwards by varying amounts. This somehow managed to make it look like there were counts/charges in the overscan area on the top of the chip (the white section in the top right of the image above). These overscan pixels don’t actually exist – they are imaginary pixels whose count levels should never vary, so finding definite values of charge in them is very worrying. Unfortunately I spent several hours every day of the bank holiday weekend unsuccessfully trying to fix this, with limited support from my supervisor, who was busy running a marathon, in Edinburgh of all places!

After many worried phone calls between my supervisor and the engineers in Edinburgh, we decide to open up the cryostat here and see if we could find any obvious problems. My supervisor was confident that there was a issue with the thermal connection between the cold liquid nitrogen tank and the chip, so this is what we really wanted to check. We took everything into a special clean room, donned real science-y overalls, boots and gloves, and had a look.

Actually, we did find that the thermal contact between the chip and the cold tank was on the dodgy side of ‘precarious’, but there was nothing evidently wrong with the cabling or electronics. We fixed the thermal contact and put it all back together. After another day of vacuum pumping and cooling, we tested the dark current and … found it to be unbelievably low. In addition the weird pattern had totally disappeared. In fact, after proper cooling and time to settle down, the dark current was reduced to zero, hurray! I was convinced this was too good to be true.

Indeed, it was. This time I really have no idea what’s happening, but after trying to test the responsiveness to light, we discovered the chip was still going absolutely bananas. Lots of nasty vertical stripes had appeared, and refused to leave. Even when allowing ample light to fall on to the chip it was only measuring it on a horizontal strip at the bottom, then sending these weird stripes all the way up the image. I don’t have a picture of this unfortunately, but basically the entire thing is still totally screwed. We’ll be opening it up again tomorrow to try to test the cabling, but it doesn’t look good. If we can’t fix it, it’ll be back up to Edinburgh again…Bah!

Not much other news at the moment. I wasted two days fighting with the formatting of my references for writing up. I finished most of my report on the testing so far, though I’m sure it will need a lot more work before it’s up to the right standard. I’ve also been working a bit with data from the robotic pt5m.  This took several days to get anywhere with, but I’ve now managed to modify a Python script from my supervisor(s) which automatically plots a light curve for the data, and measures the magnitudes in each filter too, which is pretty neat! Sadly there’s no exciting science to come out of that yet, but hopefully there will be eventually.

Now it’s time for ice cream. Byee!

When nothing goes right, and the future’s dark as night…

Today has undoubtedly been the worst day so far. Not for any particular reason, I hasten to add. It’s just been rubbish. In fact, this entire week has been pretty awful. Again, nothing in particular. My mood and motivation is at an all time low.

Maybe it’s because I was woken up at 5:45 on the morning of the half marathon because we’d just been burgled, and they’d taken my favourite jacket, amongst many other things. Maybe it’s because my knees weren’t really ready to run 21km in one go, and they’ve been hurting all week. Maybe it’s because I’d forgotten how boring writing reports can be, and how frustrated one can get if one wrestles with LaTeX formatting and references all day. Maybe it’s because I had the noisy, painful kind of hiccups for an hour today. Maybe it’s because I spent 20 minutes wandering around the supermarket, unable to decide what to have for dinner. Maybe it’s because despite perhaps giving off an air of direction and foresight, I still have absolutely no idea who I am, what I’m doing, or where I’m going. Maybe I didn’t spend enough time digging holes in France. Maybe I’m just upset because I can’t squeeze enough ‘maybe’s into this blog post.

Or maybe, just maybe, there’s nothing wrong at all. Maybe I just woke up on the wrong side of the bed four days in a row. Maybe, and most terrifying of all, this is simply what real ‘real’ life is like, and it doesn’t get any better… Surely not?

And I know you can’t have good days without the bad, I know that. So I’m just hoping that after 4 definitively bad days, maybe tomorrow might be better. Maybe?

‘Maybe’ count: 16 [edit]

The shortest, toughest week of the year



Despite only having 4 days, this working week has been a heavy slog. On Tuesday and Wednesday I was finishing the tests with ULTRASPEC. The bulk of the work involved creating a ‘defect mask’, which highlights all the dodgy areas of the CCD and helps observers see where to avoid placing their targets. These defects can be anything from consistently under-performing pixels to specks of dust. Unfortunately the ULTRASPEC CCD is overflowing with dodgy pixels, which meant I had to spend several hours painstakingly scouring the test images, testing the fractional difference in performance around suspected bad pixels, and ironically labelling each one either “Moderate” or “Disastrous”. The finished product is shown above.

This was desperately dull, but not as dull as testing the dark current as a function of temperature. This task had me in the lab until after midnight on Tuesday, and quite late on Wednesday too. The tests involved setting the temperature of the chip (easy enough, except that it takes 15-20 minutes for the temperature to stabilise each time), then taking 45 minutes of long exposures. The plus side of this is that I could get on with other things whilst the exposures were running (e.g. making the pixel mask, or watching cartoons, as the night went on). 

By Thursday morning I was just about halfway through the dark current tests, but I was in for a bit of a rough day. By 10am, for a still unknown reason, the electronics within the detector itself which are used to read the temperatures, went totally bananas. The CCD chip suddenly began reading values close to room temperature, despite being in a thermally cooled environment, supposedly at -100°C. In addition, the thermal conductor between the liquid nitrogen tank and the chip was displaying values around 65K. This is 10 degrees cooler than the boiling point of liquid nitrogen, and simply impossible. After checking the spare cabling and the spare temperature controller, we deduced that something was severely wrong on the inside of the detector. This was bad news, and means that the whole thing has to go back up to Edinburgh to be taken apart and tested by the experts. Balls.

The upshot of this is that I don’t have to come in on weekends and fill the cryostat, which is a relief. But I am disappointed. I was definitely expecting to finish the testing this week, and now I might not get a chance to finish at all. Thankfully, the remaining tests have been done in the past, but it would be nice to repeat them myself. 

The rest of the week has been spent catching up with the recent data collected by pt5m. Unfortunately, it was only after downloading, organising and plotting all the data that I discovered that 90% of it was pretty useless due to heavy cloud cover. So much for La Palma being one of the top three observing sites in the world!

On a positive note, I’ve now booked my travel to Paris in June, which is quite exciting. I hope I can find a couchsurfing host for it though – my travel grant is already feeling rather stretched. Other plus points of this week include seeing Ironman 3 on Wednesday which I thoroughly enjoyed, eating lunch in the sunshine, and discovering a highly motivational and enlightening video on YouTube titled “This Is Water” (google it – it’s definitely worth your time, if you can ignore the consumerist references). 

If this week has been the most mentally straining week so far, this weekend will match it perfectly in terms of physical strain. On Sunday morning I’ll be running the Sheffield Half Marathon, not because I like running (I don’t), nor because I like raising money for charity (I hate asking people to donate). I think I wanted to dedicate something to the memory of a good friend, great mentor, and fantastic lecturer, Dr Tim Richarson, who passed away earlier this year. Tim was a true inspiration for so many students and staff, and I miss him. Perhaps running 21km is a strange way to show it, but he was a brilliantly strange person. If you’d like to donate, explore the charity webpages in the link above, or alternatively, you can help destitute asylum seekers in Leicester by texting “LCOS13 £5” to 70070. 

Looking cool at -196 °C


Collecting liquid nitrogen for ULTRASPEC, complete with hipster safety glasses

It’s been another manically busy week in the lab, and progress has been slow. But I did identify and help to solve several potentially serious glitches with ULTRASPEC, and I now have almost all the juicy content for a small chapter of my thesis. It will need a lot of wordy input, and some more research before I really know what I’m talking about, but it’s far more substantial than I was expecting for one month in to the PhD!

The downside to working on the instrumentation at the moment is that the cryostat (insulated bucket which holds the CCD) has to be kept cold at all times. This means filling it with liquid nitrogen (77K, or -196 °C) twice a day, including weekends. This is what exciting science looks like on TV (see photo above!), but in reality it’s actually very tedious.

The most exciting bit of this week was actually a gig last night at the Library Theatre. This really is a theatre, and not somewhere you’d usually expect to watch a hip-hop brass band (the very entertaining Renegade Brass Band) followed by a 9-piece Israeli funk band (possibly the best live performance I’ve ever seen, from The Apples), all for a measly £8. Absolute bargain! Dancing like a maniac for 3 hours is definitely the best way to end an exhausting week.

Hopefully by the end of next week I’ll have completely finished all the testing, and I’ll be ready to start something new. In the mean time, I’ll just continue with the mind-numbingly repetitive key-strokes until all the checks are done. Whoever said astronomy was exciting has clearly never tried testing every possible read-out mode of a modern CCD detector a hundred times!