Metrics for crystallographic diffraction- and fit-data: a review of existing ones and the need for new ones

Raw diffraction data and reproducibility

In recent years, there has been a major expansion in digital storage capability for hosting raw diffraction datasets. Naturally, the question has now arisen as to the benefits and costs for the preservation of such raw, i.e., experimental diffraction datasets. We describe the consultations made of the global structural chemistry, i.e., chemical crystallography community from the points of view of the International Union of Crystallography (IUCr) Committee on Data, of which JRH was the Chair until very recently, and the IUCrData Raw Data Letters initiative, for which LKB is the Main Editor. The monitoring by the CCDC of CSD depositions which cite the digital object identifiers of raw diffraction datasets provides interesting statistics by probe (x-ray, neutron, or electron) and by home lab vs central facility. Clearly, a better understanding of the reproducibility of current analysis procedures is at hand. Policies for publication requiring raw data have been updated in IUCr Journals for macromolecular crystallography, namely, that raw data should be made available for a new crystal structure or a new method as well as the wwPDB deposition. For chemical crystallography, such a step requiring raw data archiving has not yet been recommended by the IUCr Commission on Structural Chemistry.

Progress in detection of and correction for low-energy contamination

Contamination with low-energy radiation leads to an increased number of weighted residuals being larger in absolute terms than three standard uncertainties. For a Gaussian distribution, these rare events occur only in 0.27% of all cases, which is a small number for small- to medium-sized data sets. The correct detection of rare events - and an adequate correction procedure - thus relies crucially on correct standard uncertainties, which are often not available [Henn (2019), Crystallogr. Rev. 25, 83-156]. It is therefore advisable to use additional, more robust, metrics to complement the established ones. These metrics are developed here and applied to reference data sets from two different publications about low-energy contamination. Other systematic errors were found in the reference data sets. These errors compromise the correction procedures and may lead to under- or overcompensation. This can be demonstrated clearly with the new metrics. Empirical correction procedures generally may be compromised or bound to fail in the presence of other systematic errors. The following systematic errors, which were found in the reference data sets, need to be corrected for prior to application of the low-energy contamination correction procedure: signals of 2λ contamination, extinction, disorder, twinning, and too-large or too-low standard uncertainties (this list may not be complete). All five reference data sets of one publication show a common resolution-dependent systematic error of unknown origin. How this affects the correction procedure can be stated only after elimination of this error. The methodological improvements are verified with data published by other authors.