Utilizing the 25m Dwingeloo Radio Telescope (DRT) to Study Fast Radio Bursts (FRBs) at ASTRON, the Netherlands 🇳🇱 📡
This github repo is no longer active, for updates about the pipeline see the gitlab repo
"We look to the stars, but all we find is RFI." - Found on the back of a poster at ASTRON
Over summer 2024, I was a researcher at The Netherlands Institute for Radio Astronomy (ASTRON) with their Summer Research Programme. For my project, I was using and operating the 25-m Dwingeloo radio telescope. I am studing bright repeating Fast Radio Bursts (FRBs) to understand the potential connections between repeating and apparently non-repeating FRBs. As FRBs are hard to catch, I will also observe pulsars to both test the methodology and learn the relevant techniques.
This is me inside the control room, observing with the 25-m Dwingeloo Radio Telescope, and this is my confirmed detection of FRB20240619D.
|
|
Pipeline GitLab Repo: Here
Astron Daily Image: Here
The Big Picture of my project is to learn & apply radio astronomy techniques for detecting FRBs. This entails understanding how the radio telescope turns radio signals into 'raw' data (in our case voltages), and then how the 'raw' data is further reduced by a data pipeline. The data pipeline searchs for and cuts out Radio Frequency Interference (RFI), and searchs for possible extragalactic radio signals, at present this is done with the presto package. Part of the Big Idea is to make a series of minor to moderate (or even major) improvements to the search pipeline. If I am very lucky, we will be able to detect a Repeating FRB over summer. However, in the more likely case we will be looking at a bright Pulsar to test as a final 'proof of concept' that the DRT can be used in dedicated campaigns to observe bright repeating FRBs.
- reading material
- ref sheet of usefull cmds
- software requirements and versions
- example_pipeline_h5_output contains a notebook that opens a
.h5file, makes a waterfall plot for practice. Using a.h5file made from the pipeline - frb_example_data_june_2024 contains a notebook making waterfall plots from CHIME
.npyfiles, and two notebooks trying outfitburston CHIME data, a simulation and a possible detection of an FRB - noise_channels contains several notebooks, and a script that looks at past observations taken and calculutes bad frequency channels to mask
- inject_sims_into_fil, contains notebooks and scripts for injecting simulated bursts into
.fildata for injection testing the pipeline - dbcan_clustering, contains a notebook for trying out dbscan for clustering candidates in 2d (DM, Time), contains rough code to edit
check_frb.pyto cluster - calling_fetch, contains a notebook and a few modified scripts from the gitlab repo. This enables fetch to be called from within
check_frb.py, as well as moving the candidate.h5files into good or bad directors and making dir plots for ones in the good dir - ddplan, contains a notebook trying out
DDplan.pyfrom presto on my laptop, and the new function that has been incorporated into the pipeline - presto_with_docker, contains a notebook for running, and executing presto commands - with presto running in a docker container
- recording_baseband_data, contains a notebook for trying to record baseband data, take slices for good candidates, and slices for good candidates in other bands
- Crab analysis, contains notebooks, and work for analysing the pulses from the crab
- FRB analysis, contains a plotting notebook for my FRB detection. It also has python functions for dedispersing and bandwidth correction
- Final Presentation Slides
- If you want any data, just contact me
- Crab giant pulse detected with the Dwingeloo radio telescope on 2024-07-30
Details:
- See ATel #16690, detected by MeerKat and ATel #16732, detected by Westerbork
- Expected DM: 465 (pc/cm^3)
- Expected Ra, Dec: 297.37, -25.21 (deg)
- We have limited observation time, typically its up ~23:00 - 04:00, I have to be physically in the telescope to take data...
- I have made a postive detection! No Baseband data was recorded due to high RFI
- See ATel #16737 , detected by CHIME
- Expected DM: 351 (pc/cm^3)
- Expected Ra, Dec: 354.58, 32.38
- See ATel#16670 by Vishwangi Shah (McGill University) on behalf of the CHIME/FRB Collaboration
- Expected DM: 176 (pc/cm^3)
- Expected RA, Dec: 289.91, 86.06 (deg)
- Observing Cadence: Typically ~07:00-10:00, 11:00-15:00 (UTC) Monday to Friday (I have to be physically in the telescope
- Total of around 200 hours on target. Stopped observing around the 16th of July.
- Have the camras mastodon bot post "good" candidates from fetch
- Send as an email to myself, and Tammo, others as well
- Add a poster outside of the Camras Telescope to include its FRB research (With a QR code to an engaging video!)
- Try 3d DBscan clustering, taking into account the brightness of the detection (right now its time, and DM)
- Probably not a substaintial improvement
- Try out Transientx over PRESTO for the pipeline for speed
- Have Desktop application to click to start program on Mercurius for ease of use
- Auto stop when storage has less then 20Gb left
- update FRB.cat file to be in
.keyformat - Write a comprehensive observation log, including durations, headers, and clustering info
- When there is a 'hyper-active' FRB, have a camras team try to take many hours of data - I will commute from Amsterdam for this.
- Maybe make a post asking for extra volunteers
- Inject Simulations into
.filfiles to test how many FRBs the pipeline recovers- Concern that bright FRB's might be marked as RFI, see unexpectedly low Crab pulses in the bottom right of this plot
Tasks:
Completed:
-
Understand how FRB signals from space turn into dynamic spectra. See flowchart.
-
Learn how to operate the 25-m Dwingeloo Radio Telescope.
-
Learn how to use Presto for single pulse searches and RFI removal.
-
Learn how the current pipeline works (
check_frb.py). See flowchart. -
Learn how Fetch works and implement it into the pipeline.
- Fetch is installed and working on Uranus!
-
Learn how TransientX works.
-
Understand what Burst Parameters can be observed & measured directly, and which ones can be inferred.
- List out model components from FITBURST, polarization, fluence, etc.
-
Understand how to use fitburst.
-
Compare pipeline outputs when using the
--ignorechanflag incheck_frb.py -
Try using
fitburston the CHIME data I have, simulated data, and my possible detection of FRB20240209A. -
Try a clustering algorithm for reducing the total number of candidates (e.g. DBSCAN).
-
Implement dbscan clustering into
check_frb.py- Reduces number of candidates by ~ the number of DM trials
-
Modify the scrits to record observations on Uranus (instead of Mercurius)
- Paul, and Tammo did this, plugging in some cabels and running a data stream from mercurius to uranus
- can record L and P bands directly onto Uranus
-
Modify
check_frb.pyto runfetch, and move the files into the good and bad dirs, make diagnostic.pngs -
Add back log file for candidates
- Added a
#header
- Added a
-
Added a log file for pipeline crashes
-
Added observing time, and number of good candidates to
frb_dashboard.py -
Work on making the pipeline run in real time - [x] Modify storage location of
.filfiles from/dateto/date/>bandname<for runnning the pipeline in real time - [x] Modifyfrb_dashboard.pyfor the new file organization - [x] Make agooddir to hold all the good.h5and.pngfiles located at/date/good -
Figure out what causes the pipeline to crash, and adjust the number of CPU cores as needed - [x]
dashboard.pywas using all the CPUS... fixed - [x] Limit GPU to one call offetch- Good CPU limit seems around 12 per band?, No noticable increase after this - Still crashes, L1 and L2 recording seems unstable- Double Check
ifstatments for--ignorechanoption incheck_frb.py - Look into making simulated, injecting simulations into
.filfiles for testing.-
fitbursthas a coolsimulate_burst.pyscript that can simulate dedispersed or dispersed dynamic spectrums -
will is a simulator that can be used to inject (and extract!) simulated pulses into
.filfiles!- Struggling on controlling the amplitude (SNR) of the injected signal, talked to Dirk who offered advice but haven't worked on this in a while
-
- Double Check
-
record raw voltages
- Understand how to record raw voltages aka baseband data
- Wrote function to slice a small chunk of data into a new file for a candidate
- Write a function to convert arrival times for different frequency bands based on the DM
- Write function to loop through candidates, and then delete the big baseband files
- Implement into pipeline
- Added storage on ram disk to
frb_dashboard.py - Test
- Read how the
single_pulse_searchworks inpresto, and how it determines the SNR - How it determines SNR - what time it uses for the time of the candidate (it is topocentric, top of band) - How many, and what size time boxcars in the search
-
Analysis of the Crab Pulsar (see Crab analysis:
- Make plots of SNR, Fluence,
$E_{\nu}$ for the crab over many hours of observations - Compare to 2019, 2024 Crab paper, and Nature Paper
- Addtionaly make some stastical plots of the Crab in the different Bands
To Do:
-
[ ]Implement TransientX into the pipeline. - Waiting for it to be installed on Uranus.
-
Write documenation for pipelines, new flowcharts
-
Understand what Paul and Tammo did to have the data record on Uranus
-
write an introduction section on
FRBS, the DRT, and our observational parameters (bandwidth, devices, data points per second etc) -
Fill in black boxes in the flowcharts.
- Look into how
prestoactually removes RFI and finds pulses.- Look into how candidates are extracted from the
.filfile.
- Look into how candidates are extracted from the
- Look into how
-
Make a 'hip' mastodon bot to display the pngs from the good candidates
- maybe make the pngs nan out detected RFI
-
Make a script to make
.h5files with full resolution for a good fetch candidate. -
Injection testing the pipeline.
-
Test the pipeline on Crab or Pulsar and compare the number of recovered vs. missed bursts.
- This would be a good test for our clustering methodology as well.
-
Combine data with other telescopes to measure fringes/localization.
-
Maybe make a docker container version of the pipeline.
-
Read more FRB papers.
-
Make nicely formatted documentation for the pipeline.
-
Understand what happens when intra-channel Dispersion is dominate.
-
Understand how our FRB search and CHIME's differ, IE with many antennas .
-
Make 'hip' popular science videos on research
-
Understand Red vs White Noise.
- Make plots of SNR, Fluence,