by alter-ego » Fri Apr 16, 2021 12:24 am
Chris Peterson wrote: ↑Thu Apr 15, 2021 3:36 pm
neufer wrote: ↑Thu Apr 15, 2021 2:32 pm
RocketRon wrote: ↑Thu Apr 15, 2021 8:05 am
Some of these 'images' have been produced by a vast amount of data processing. It would be interesting to see what sort of safeguards have been put in place for the processing not to be self fulfilling ?
Confirmation bias is always a risk but you do the best you can.
Confirmation bias may need to be guarded against to some degree in building the models that simulate black hole appearance. But in this case, there seems to be little opportunity for bias in processing and displaying the actual data. AFAIK the processing involved is deterministic and has no free parameters (or highly constrained parameters).
Reduction and display is an involved process. The paths to biased processing are numerous. As you said, independent image reduction was done independently from BH models, but image corroboration with model(s) was an important in the process. In addition, data from the 8 EHT sites covering multiple days had to be processed. From these papers, the image generation was far from deterministic.
First M87 Event Horizon Telescope Results. I. The Shadow of the Supermassive Black Hole wrote:
5. Images and Features
We reconstructed images from the calibrated EHT visibilities, which provide results that are independent of models (Paper IV).
However, there are two major challenges in reconstructing images from EHT data. First, EHT baselines sample a limited range of spatial frequencies, corresponding to angular scales between 25 and 160 μas. Because the (u, v) plane is only sparsely sampled (Figure 2), the inverse problem is under-constrained. Second, the measured visibilities lack absolute phase calibration and can have large amplitude calibration uncertainties.
To address these challenges, imaging algorithms incorporate additional assumptions and constraints that are designed to produce images that are physically plausible (e.g., positive and compact) or conservative (e.g., smooth), while remaining consistent with the data. We explored two classes of algorithms for reconstructing images from EHT data. The first class of algorithms is the traditional CLEAN approach used in radio interferometry (e.g., Högbom 1974; Clark 1980). CLEAN is an inverse-modeling approach that deconvolves the interferometer point-spread function from the Fourier-transformed visibilities. When applying CLEAN, it is necessary to iteratively self-calibrate the data between rounds of imaging to solve for time-variable phase and amplitude errors in the data. The second class of algorithms is the so-called regularized maximum likelihood (RML; e.g., Narayan & Nityananda 1986; Wiaux et al. 2009; Thiébaut 2013). RML is a forward-modeling approach that searches for an image that is not only consistent with the observed data but also favors specified image properties (e.g., smoothness or compactness). As with CLEAN, RML methods typically iterate between imaging and self-calibration, although they can also be used to image directly on robust closure quantities immune to station-based calibration errors. RML methods have been extensively developed for the EHT (e.g., Honma et al. 2014; Bouman et al. 2016; Akiyama et al. 2017; Chael et al. 2018b; see also Paper IV).
Every imaging algorithm has a variety of free parameters that can significantly affect the final image. We adopted a two-stage imaging approach to control and evaluate biases in the reconstructions from our choices of these parameters. In the first stage, four teams worked independently to reconstruct the first EHT images of M87* using an early engineering data release. The teams worked without interaction to minimize shared bias, yet each produced an image with a similar prominent feature: a ring of diameter ~38–44 μas with enhanced brightness to the south (see Figure 4 in Paper IV).
In the second imaging stage, we developed three imaging pipelines, each using a different software package and associated methodology. Each pipeline surveyed a range of imaging parameters, producing between ~103 and 104 images from different parameter combinations. We determined a "Top-Set" of parameter combinations that both produced images of M87* that were consistent with the observed data and that reconstructed accurate images from synthetic data sets corresponding to four known geometric models (ring, crescent, filled disk, and asymmetric double source). For all pipelines, the Top-Set images showed an asymmetric ring with a diameter of ~40 μas, with differences arising primarily in the effective angular resolutions achieved by different methods.
For each pipeline, we determined the single combination of fiducial imaging parameters out of the Top-Set that performed best across all the synthetic data sets and for each associated imaging methodology (see Figure 11 in Paper IV). Because the angular resolutions of the reconstructed images vary among the pipelines, we blurred each image with a circular Gaussian to a common, conservative angular resolution of 20 μas. The top part of Figure 3 shows an image of M87* on April 11 obtained by averaging the three pipelines' blurred fiducial images. The image is dominated by a ring with an asymmetric azimuthal profile that is oriented at a position angle ~170° east of north. Although the measured position angle increases by ~20° between the first two days and the last two days, the image features are broadly consistent across the different imaging methods and across all four observing days. This is shown in the bottom part of Figure 3, which reports the images on different days (see also Figure 15 in Paper IV). These results are also consistent with those obtained from visibility-domain fitting of geometric and general-relativistic magnetohydrodynamics (GRMHD) models (Paper VI).
[quote="Chris Peterson" post_id=312541 time=1618500981 user_id=117706]
[quote=neufer post_id=312539 time=1618497146 user_id=124483]
[quote=RocketRon post_id=312525 time=1618473921]
Some of these 'images' have been produced by a vast amount of data processing. It would be interesting to see what sort of safeguards have been put in place for the processing not to be self fulfilling ?[/quote]
Confirmation bias is always a risk but you do the best you can.
[/quote]
Confirmation bias may need to be guarded against to some degree in building the models that simulate black hole appearance. But in this case, there seems to be little opportunity for bias in processing and displaying the actual data. AFAIK the processing involved is deterministic and has no free parameters (or highly constrained parameters).
[/quote]
Reduction and display is an involved process. The paths to biased processing are numerous. As you said, independent image reduction was done independently from BH models, but image corroboration with model(s) was an important in the process. In addition, data from the 8 EHT sites covering multiple days had to be processed. From these papers, the image generation was far from deterministic.
[quote="[url=https://iopscience.iop.org/article/10.3847/2041-8213/ab0ec7/meta#apjlab0ec7s4]First M87 Event Horizon Telescope Results. I. The Shadow of the Supermassive Black Hole[/url]"]
[b][size=130]5. Images and Features[/size][/b]
We reconstructed images from the calibrated EHT visibilities, which provide results that are independent of models (Paper IV).[color=#0000FF][size=120][b]However, there are two major challenges in reconstructing images from EHT data. First, EHT baselines sample a limited range of spatial frequencies, corresponding to angular scales between 25 and 160 μas. Because the (u, v) plane is only sparsely sampled (Figure 2), the inverse problem is under-constrained. Second, the measured visibilities lack absolute phase calibration and can have large amplitude calibration uncertainties.[/size][/color][/b]
[attachment=1]Fig 2.jpg[/attachment]
To address these challenges, imaging algorithms incorporate additional assumptions and constraints that are designed to produce images that are physically plausible (e.g., positive and compact) or conservative (e.g., smooth), while remaining consistent with the data. We explored two classes of algorithms for reconstructing images from EHT data. The first class of algorithms is the traditional CLEAN approach used in radio interferometry (e.g., Högbom 1974; Clark 1980). CLEAN is an inverse-modeling approach that deconvolves the interferometer point-spread function from the Fourier-transformed visibilities. When applying CLEAN, it is necessary to iteratively self-calibrate the data between rounds of imaging to solve for time-variable phase and amplitude errors in the data. The second class of algorithms is the so-called regularized maximum likelihood (RML; e.g., Narayan & Nityananda 1986; Wiaux et al. 2009; Thiébaut 2013). RML is a forward-modeling approach that searches for an image that is not only consistent with the observed data but also favors specified image properties (e.g., smoothness or compactness). As with CLEAN, RML methods typically iterate between imaging and self-calibration, although they can also be used to image directly on robust closure quantities immune to station-based calibration errors. RML methods have been extensively developed for the EHT (e.g., Honma et al. 2014; Bouman et al. 2016; Akiyama et al. 2017; Chael et al. 2018b; see also Paper IV).
[color=#0000FF][b][size=120]Every imaging algorithm has a variety of free parameters that can significantly affect the final image.[/size][/b][/color] We adopted a two-stage imaging approach to control and evaluate biases in the reconstructions from our choices of these parameters. In the first stage, four teams worked independently to reconstruct the first EHT images of M87* using an early engineering data release. The teams worked without interaction to minimize shared bias, yet each produced an image with a similar prominent feature: a ring of diameter ~38–44 μas with enhanced brightness to the south (see Figure 4 in Paper IV).
In the second imaging stage, we developed three imaging pipelines, each using a different software package and associated methodology. Each pipeline surveyed a range of imaging parameters, producing between ~103 and 104 images from different parameter combinations. We determined a "Top-Set" of parameter combinations that both produced images of M87* that were consistent with the observed data and that reconstructed accurate images from synthetic data sets corresponding to four known geometric models (ring, crescent, filled disk, and asymmetric double source). For all pipelines, the Top-Set images showed an asymmetric ring with a diameter of ~40 μas, with differences arising primarily in the effective angular resolutions achieved by different methods.
For each pipeline, we determined the single combination of fiducial imaging parameters out of the Top-Set that performed best across all the synthetic data sets and for each associated imaging methodology (see Figure 11 in Paper IV). Because the angular resolutions of the reconstructed images vary among the pipelines, we blurred each image with a circular Gaussian to a common, conservative angular resolution of 20 μas. The top part of Figure 3 shows an image of M87* on April 11 obtained by averaging the three pipelines' blurred fiducial images. The image is dominated by a ring with an asymmetric azimuthal profile that is oriented at a position angle ~170° east of north. Although the measured position angle increases by ~20° between the first two days and the last two days, the image features are broadly consistent across the different imaging methods and across all four observing days. This is shown in the bottom part of Figure 3, which reports the images on different days (see also Figure 15 in Paper IV). These results are also consistent with those obtained from visibility-domain fitting of geometric and general-relativistic magnetohydrodynamics (GRMHD) models (Paper VI).
[attachment=0]Brightness Temperature.jpg[/attachment]
[/quote]