Raw diffraction data preservation and reuse: overview, update on practicalities and metadata requirements

The evolution of raw data archiving and the growth of its importance in crystallography

The hardware for data archiving has expanded capacities for digital storage enormously in the past decade or more. The IUCr evaluated the costs and benefits of this within an official working group which advised that raw data archiving would allow ground truth reproducibility in published studies. Consultations of the IUCr's Commissions ensued via a newly constituted standing advisory committee, the Committee on Data. At all stages, the IUCr financed workshops to facilitate community discussions and possible methods of raw data archiving implementation. The recent launch of the IUCrData journal's Raw Data Letters is a milestone in the implementation of raw data archiving beyond the currently published studies: it includes diffraction patterns that have not been fully interpreted, if at all. The IUCr 75th Congress in Melbourne included a workshop on raw data reuse, discussing the successes and ongoing challenges of raw data reuse. This article charts the efforts of the IUCr to facilitate discussions and plans relating to raw data archiving and reuse within the various communities of crystallography, diffraction and scattering.

A MATLAB Toolbox for Findable, Accessible, Interoperable, and Reusable Atom Probe Data Science

Atom probe tomography (APT) data analytics have traditionally been based on manual analytics by researchers. As newer atom probes together with focused ion beam-based specimen preparation have opened APT to many more materials, yielding much more complex mass spectra, building up a systematic understanding of the pathway from raw data to final interpretation has increasingly become important. This demands a system in which the data and treatment can be traced, ideally by any interested party. Such an approach of findable, accessible, interoperable, and reusable (FAIR) data and analysis policies is becoming increasingly important, not just in APT. In this paper, we present a toolbox, written in MATLAB, which allows the user to store the raw and processed data in a standardized FAIR format (hierarchical data format 5) and process the data in a largely scriptable environment to minimize manual user input. This allows for the experiment data to be interchanged without owner explanations and the analysis to be reproduced. We have devised a metadata scheme that is extensible to other experiments in the materials science domain. With this toolbox, collective knowledge can be built up, and a large number of data sets can be analyzed in a fully automated fashion.

Raw diffraction data and reproducibility

In recent years, there has been a major expansion in digital storage capability for hosting raw diffraction datasets. Naturally, the question has now arisen as to the benefits and costs for the preservation of such raw, i.e., experimental diffraction datasets. We describe the consultations made of the global structural chemistry, i.e., chemical crystallography community from the points of view of the International Union of Crystallography (IUCr) Committee on Data, of which JRH was the Chair until very recently, and the IUCrData Raw Data Letters initiative, for which LKB is the Main Editor. The monitoring by the CCDC of CSD depositions which cite the digital object identifiers of raw diffraction datasets provides interesting statistics by probe (x-ray, neutron, or electron) and by home lab vs central facility. Clearly, a better understanding of the reproducibility of current analysis procedures is at hand. Policies for publication requiring raw data have been updated in IUCr Journals for macromolecular crystallography, namely, that raw data should be made available for a new crystal structure or a new method as well as the wwPDB deposition. For chemical crystallography, such a step requiring raw data archiving has not yet been recommended by the IUCr Commission on Structural Chemistry.

Data format standards in analytical chemistry

Research data is an essential part of research and almost every publication in chemistry. The data itself can be valuable for reuse if sustainably deposited, annotated and archived. Thus, it is important to publish data following the FAIR principles, to make it findable, accessible, interoperable and reusable not only for humans but also in machine-readable form. This also improves transparency and reproducibility of research findings and fosters analytical work with scientific data to generate new insights, being only accessible with manifold and diverse datasets. Research data requires complete and informative metadata and use of open data formats to obtain interoperable data. Generic data formats like AnIML and JCAMP-DX have been used for many applications. Special formats for some analytical methods are already accepted, like mzML for mass spectrometry or nmrML and NMReDATA for NMR spectroscopy data. Other methods still lack common standards for data. Only a joint effort of chemists, instrument and software vendors, publishers and infrastructure maintainers can make sure that the analytical data will be of value in the future. In this review, we describe existing data formats in analytical chemistry and introduce guidelines for the development and use of standardized and open data formats.

Raw diffraction data are our ground truth from which all subsequent workflows develop

Defining best practice in science is challenging. International consensus is facilitated by the International Science Council via its members such as the International Union of Crystallography (IUCr). The crystallographic community has many decades of tradition linking articles with the underpinning data, and is admired across all sciences accordingly. Crystallography has always been at the forefront of harnessing new technology in the service of consensus. Technology has provided new vast data-archiving opportunities, allowing the preservation of raw diffraction data, along with article and database depositions of a model's coordinates and associated structure factors. The raw diffraction data, which can now be preserved, are the ground truth from which all subsequent workflows develop. Journal editorial boards provide a practical forum for setting the criteria to decide if a study's files are truly the version of record. Within that, reality involves a variance of reasonable workflows. But what is a reasonable variance? Workflows must be detailed carefully by authors in explaining what they have done. There is a great, and increasing, diversity of macromolecular crystallography analyses, and yet an increased constraint on how much can be written in an article about the workflow used. Raw data provide the ultimate reproducibility evidence. A part of reproducibility and replicability is using an agreed vocabulary; the meaning of words such as precision and accuracy and, more recently, the confidence of a protein structure prediction should feature in approaching `truth'.

Pre- and Post-publication Verification for Reproducible Data Mining in Macromolecular Crystallography

Like an article narrative is deemed by an editor and referees to be worthy of being a version of record on acceptance as a publication, so must the underpinning data also be scrutinized before passing it as a version of record. Indeed without the underpinning data, a study and its conclusions cannot be reproduced at any stage of evaluation, pre- or post-publication. Likewise, an independent study without its own underpinning data also cannot be reproduced let alone be considered a replicate of the first study. The PDB is a modern marvel of achievement providing an organized open access to depositor and user of the data held there opening numerous applications. Methods for modeling protein structures and for determination of structures are still improving their precision, and artifacts of the method exist. So their accuracy is realized if they are reproduced by other methods. It is on such foundations that reproducible data mining is based. Data rates are expanding considerably be they at synchrotrons, the X-ray free electron lasers (XFELs), electron cryomicroscopes (cryoEM), or at the neutron facilities. The work of a person as a referee or user with a narrative and its underpinning data may well be complemented in future by artificial intelligence with machine learning, the former for specific refereeing and the latter for the more general validation, both ideally before publication. Examples are described involving rhenium theranostics, the anti-cancer platins and the SARS-CoV-2 main protease.

Gold Standard for macromolecular crystallography diffraction data

Macromolecular crystallography (MX) is the dominant means of determining the three-dimensional structures of biological macromolecules. Over the last few decades, most MX data have been collected at synchrotron beamlines using a large number of different detectors produced by various manufacturers and taking advantage of various protocols and goniometries. These data came in their own formats: sometimes proprietary, sometimes open. The associated metadata rarely reached the degree of completeness required for data management according to Findability, Accessibility, Interoperability and Reusability (FAIR) principles. Efforts to reuse old data by other investigators or even by the original investigators some time later were often frustrated. In the culmination of an effort dating back more than two decades, a large portion of the research community concerned with high data-rate macromolecular crystallography (HDRMX) has now agreed to an updated specification of data and metadata for diffraction images produced at synchrotron light sources and X-ray free-electron lasers (XFELs). This 'Gold Standard' will facilitate the processing of data sets independent of the facility at which they were collected and enable data archiving according to FAIR principles, with a particular focus on interoperability and reusability. This agreed standard builds on the NeXus/HDF5 NXmx application definition and the International Union of Crystallo-graphy (IUCr) imgCIF/CBF dictionary, and it is compatible with major data-processing programs and pipelines. Just as with the IUCr CBF/imgCIF standard from which it arose and to which it is tied, the NeXus/HDF5 NXmx Gold Standard application definition is intended to be applicable to all detectors used for crystallography, and all hardware and software developers in the field are encouraged to adopt and contribute to the standard.

checkCIF validation ALERTS: what they mean and how to respond

Authors of a paper that includes a new crystal-structure determination are expected to not only report the structural results of inter-est and their inter-pretation, but are also expected to archive in computer-readable CIF format the experimental data on which the crystal-structure analysis is based. Additionally, an IUCr/checkCIF validation report will be required for the review of a submitted paper. Such a validation report, automatically created from the deposited CIF file, lists as ALERTS not only potential errors or unusual findings, but also suggestions for improvement along with inter-esting information on the structure at hand. Major ALERTS for issues are expected to have been acted on already before the submission for publication or discussed in the associated paper and/or commented on in the CIF file. In addition, referees, readers and users of the data should be able to make their own judgment and inter-pretation of the underlying experimental data or perform their own calculations with the archived data. All the above is consistent with the FAIR (findable, accessible, inter-operable, and reusable) initiative [Helliwell (2019 ▸). Struct. Dyn. 6, 05430]. Validation can also be helpful for less experienced authors in pointing to and avoiding of crystal-structure determination and inter-pretation pitfalls. The IUCr web-based checkCIF server provides such a validation report, based on data uploaded in CIF format. Alternatively, a locally installable checkCIF version is available to be used iteratively during the structure-determination process. ALERTS come mostly as short single-line messages. There is also a short explanation of the ALERTS available through the IUCr web server or with the locally installed PLATON/checkCIF version. This paper provides additional background information on the checkCIF procedure and additional details for a number of ALERTS along with options for how to act on them.

The Integrated Resource for Reproducibility in Macromolecular Crystallography: Experiences of the first four years

It has been increasingly recognized that preservation and public accessibility of primary experimental data are cornerstones necessary for the reproducibility of empirical sciences. In the field of molecular crystallography, many journals now recommend that authors of manuscripts presenting a new crystal structure should deposit their primary experimental data (X-ray diffraction images) to one of the dedicated resources created in recent years. Here, we describe our experiences developing the Integrated Resource for Reproducibility in Molecular Crystallography (IRRMC) and describe several examples of a crucial role that diffraction data can play in improving previously determined protein structures. In its first four years, several hundred crystallographers have deposited data from over 5200 diffraction experiments performed at over 60 different synchrotron beamlines or home sources all over the world. In addition to improving the resource and curating submitted data, we have been building a pipeline for extraction or, in some cases, reconstruction of the metadata necessary for seamless automated processing. Preliminary analysis indicates that about 95% of the archived data can be automatically reprocessed. A high rate of reprocessing success shows the feasibility of using the automated metadata extraction and automated processing as a validation step for the deposition of raw diffraction images. The IRRMC is guided by the Findable, Accessible, Interoperable, and Reusable data management principles.

FACT and FAIR with Big Data allows objectivity in science: The view of crystallography

A publication is an important narrative of the work done and interpretations made by researchers securing a scientific discovery. As The Royal Society neatly states though, "Nullius in verba" ("Take nobody's word for it"), whereby the role of the underpinning data is paramount. Therefore, the objectivity that preserving that data within the article provides is due to readers being able to check the calculation decisions of the authors. But how to achieve full data archiving? This is the raw data archiving challenge, in size and need for correct metadata. Processed diffraction data and final derived molecular coordinates archiving in crystallography have achieved an exemplary state of the art relative to most fields. One can credit IUCr with developing exemplary peer review procedures, of narrative, underpinning structure factors and coordinate data and validation report, through its checkcif development and submission system introduced for Acta Cryst. C and subsequently developed for its other chemistry journals. The crystallographic databases likewise have achieved amazing success and sustainability these last 50 years or so. The wider science data scene is celebrating the FAIR data accord, namely, that data be Findable, Accessible, Interoperable, and Reusable [Wilkinson et al., "Comment: The FAIR guiding principles for scientific data management and stewardship," Sci. Data 3, 160018 (2016)]. Some social scientists also emphasize more than FAIR being needed, the data should be "FACT," which is an acronym meaning Fair, Accurate, Confidential, and Transparent [van der Aalst et al., "Responsible data science," Bus Inf. Syst. Eng. 59(5), 311-313 (2017)], this being the issue of ensuring reproducibility not just reusability. (Confidentiality of data not likely being relevant to our data obviously.) Acta Cryst. B, C, E, and IUCrData are the closest I know to being both FACT and FAIR where I repeat for due emphasis: the narrative, the automatic "general" validation checks, and the underpinning data are checked thoroughly by subject specialists (i.e., the specialist referees). IUCr Journals are also the best that I know of for encouraging and then expediting the citation of the DOI for a raw diffraction dataset in a publication; examples can be found in IUCrJ, Acta Cryst D, and Acta Cryst F. The wish for a checkcif for raw diffraction data has been championed by the IUCr Diffraction Data Deposition Working Group and its successor, the IUCr Committee on Data.

Findable Accessible Interoperable Re-usable (FAIR) diffraction data are coming to protein crystallography

The policy of IUCr Journals on diffraction data is defined.

Findable Accessible Interoperable Re-usable (FAIR) diffraction data are coming to protein crystallography

The policy of IUCr Journals on diffraction data is defined.

Findable Accessible Interoperable Re-usable (FAIR) diffraction data are coming to protein crystallography

The policy of IUCr Journals on diffraction data is defined.

Findable Accessible Interoperable Re-usable (FAIR) diffraction data are coming to protein crystallography

The policy of IUCr Journals on diffraction data is defined.

Sharing powder diffraction raw data: challenges and benefits

Scientific data are as important as scientific publications. If this statement holds true, why are we not routinely sharing scientific data? The tools are now out there, for instance Zenodo and related repositories. It could be a lack of motivation of researchers derived from an apparent lack of short-term reward. Here the author will try to show the importance of sharing ready-to-analyse raw powder diffraction data with immediate benefits for authors and for the wider community. Moreover, it is speculated that sharing curated scientific data may have more important medium-term benefits, including credibility and not least reproducibility. Raw data sharing is coming.

Synchrotron Big Data Science

The rapid development of synchrotrons has massively increased the speed at which experiments can be performed, while new techniques have increased the amount of raw data collected during each experiment. While this has created enormous new opportunities, it has also created tremendous challenges for national facilities and users. With the huge increase in data volume, the manual analysis of data is no longer possible. As a result, only a fraction of the data collected during the time- and money-expensive synchrotron beam-time is analyzed and used to deliver new science. Additionally, the lack of an appropriate data analysis environment limits the realization of experiments that generate a large amount of data in a very short period of time. The current lack of automated data analysis pipelines prevents the fine-tuning of beam-time experiments, further reducing their potential usage. These effects, collectively known as the "data deluge," affect synchrotrons in several different ways including fast data collection, available local storage, data management systems, and curation of the data. This review highlights the Big Data strategies adopted nowadays at synchrotrons, documenting this novel and promising hybridization between science and technology, which promise a dramatic increase in the number of scientific discoveries.

Principles and methods used to grow and optimize crystals of protein-metallodrug adducts, to determine metal binding sites and to assign metal ligands

The characterization of the interactions between biological macromolecules (proteins and nucleic acids) and metal-based drugs is a fundamental prerequisite for understanding their mechanisms of action. X-ray crystallography enables the structural analysis of such complexes with atomic level detail. However, this approach requires the preparation of highly diffracting single crystals, the measurement of diffraction patterns and the structural analysis and interpretation of macromolecule-metal interactions from electron density maps. In this review, we describe principles and methods used to grow and optimize crystals of protein-metallodrug adducts, to determine metal binding sites and to assign and validate metal ligands. Examples from the literature and experience in our own laboratory are provided and key challenges are described, notably crystallization and molecular model refinement against the X-ray diffraction data.

Against Method: Table 1-Cui Bono?

The almost universally required "Table 1," summarizing data-collection and data-processing statistics, has in its present form outlived its usefulness in almost all publications of biomolecular crystal structure reports. Information contained in "Table 1" is insufficient to evaluate or repeat the experiment; is redundant with information extractable from deposited diffraction data; and includes data items whose meaning is under increased scrutiny in the crystallographic community. Direct and consistent extraction and analysis of data quality metrics from preferably unmerged intensity data with graphical presentation of reciprocal space features, including impact on map and model features, should replace "Table 1."

Bringing diffuse X-ray scattering into focus

X-ray crystallography is experiencing a renaissance as a method for probing the protein conformational ensemble. The inherent limitations of Bragg analysis, however, which only reveals the mean structure, have given way to a surge in interest in diffuse scattering, which is caused by structure variations. Diffuse scattering is present in all macromolecular crystallography experiments. Recent studies are shedding light on the origins of diffuse scattering in protein crystallography, and provide clues for leveraging diffuse scattering to model protein motions with atomic detail.

Triethanolaminate iron perchlorate revisited: change of space group, chemical composition and oxidation states in [Fe7(tea)3(tea-H)3](ClO4)2 (tea-H3 is triethanolamine)

The X-ray crystal structure of tris[N-(2-hydroxyethyl)-2,2'-iminodiethanolato]tris(2,2',2''-nitrilotriethanolato)tetrairon(II)triiron(III) bis(perchlorate), [Fe₇(C₆H₁₂NO₃)₃(C₆H₁₃NO₃)₃](ClO₄)₂ or [Fe₇(tea)₃(tea-H)₃](ClO₄)₂ (tea-H₃ is triethanolamine), is known from the literature [Liu et al. (2008). Z. Anorg. Allg. Chem. 634, 778-783] as a heptanuclear coordination cluster. The space group was given as I2₁3 and is reinvestigated in the present study. We find a new space-group symmetry of Pa-3 and could detect O-H hydrogens, which were missing in the original publication. Consequences on the Fe oxidation states are investigated with the bond-valence method, resulting in a mixed-valence core of four Fe^II and three Fe^III centres. Symmetry relationships between the two space groups and the average supergroup Ia-3 are discussed in detail.

The science is in the data

Understanding published research results should be through one's own eyes and include the opportunity to work with raw diffraction data to check the various decisions made in the analyses by the original authors. Today, preserving raw diffraction data is technically and organizationally viable at a growing number of data archives, both centralized and distributed, which are empowered to register data sets and obtain a preservation descriptor, typically a 'digital object identifier'. This introduces an important role of preserving raw data, namely understanding where we fail in or could improve our analyses. Individual science area case studies in crystallography are provided.

Healing X-ray scattering images

X-ray scattering images contain numerous gaps and defects arising from detector limitations and experimental configuration. We present a method to heal X-ray scattering images, filling gaps in the data and removing defects in a physically meaningful manner. Unlike generic inpainting methods, this method is closely tuned to the expected structure of reciprocal-space data. In particular, we exploit statistical tests and symmetry analysis to identify the structure of an image; we then copy, average and interpolate measured data into gaps in a way that respects the identified structure and symmetry. Importantly, the underlying analysis methods provide useful characterization of structures present in the image, including the identification of diffuse versus sharp features, anisotropy and symmetry. The presented method leverages known characteristics of reciprocal space, enabling physically reasonable reconstruction even with large image gaps. The method will correspondingly fail for images that violate these underlying assumptions. The method assumes point symmetry and is thus applicable to small-angle X-ray scattering (SAXS) data, but only to a subset of wide-angle data. Our method succeeds in filling gaps and healing defects in experimental images, including extending data beyond the original detector borders.

Data archiving and availability in an era of open science

The importance of preserving and making available the original experimental data underlying biological structural models is discussed, both for crystallography, where the raw data images pose particular challenges, and for other structure determination techniques.

Sharing Big Data

Macromolecular Big Data provide numerous challenges and a number of initiatives that are starting to overcome these issues are discussed.