The BioImage Database Project: organizing multidimensional biological images in an object-relational database

Operational Data-Driven Intelligent Modelling and Visualization System for Real-World, On-Road Vehicle Emissions—A Case Study in Hangzhou City, China

On-road vehicle emissions play a crucial role in affecting air quality and human exposure, particularly in megacities. In the absence of comprehensive traffic monitoring networks with the general lack of intelligent transportation systems (ITSs) and big-data-driven, high-performance-computing (HPC) platforms, it remains challenging to constrain on-road vehicle emissions and capture their hotspots. Here, we established an intelligent modelling and visualization system driven by ITS traffic data for real-world, on-road vehicle emissions. Based on the HPC platform (named “City Brain”) and an agile Web Geographic Information System (WebGISs), this system can map real-time (hourly), hyperfine (10~1000 m) vehicle emissions (e.g., PM2.5, NOx, CO, and HC) and associated traffic states (e.g., vehicle-specific categories and traffic fluxes) over the Xiaoshan District in Hangzhou. Our results show sharp variations in on-road vehicle emissions on small scales, which even fluctuated up to 31.2 times within adjacent road links. Frequent and widespread emission hotspots were also exposed. Over custom spatiotemporal scopes, we virtually investigated and visualized the impacts of traffic control policies on the traffic states and on-road vehicle emissions. Such results have important implications for how traffic control policies should be optimized. Integrating this system with chemical transport models and air quality measurements would bridge the technical gap between air pollutant emissions, concentrations, and human exposure.

Open source bioimage informatics for cell biology

Significant technical advances in imaging, molecular biology and genomics have fueled a revolution in cell biology, in that the molecular and structural processes of the cell are now visualized and measured routinely. Driving much of this recent development has been the advent of computational tools for the acquisition, visualization, analysis and dissemination of these datasets. These tools collectively make up a new subfield of computational biology called bioimage informatics, which is facilitated by open source approaches. We discuss why open source tools for image informatics in cell biology are needed, some of the key general attributes of what make an open source imaging application successful, and point to opportunities for further operability that should greatly accelerate future cell biology discovery.

Minimum information specification for in situ hybridization and immunohistochemistry experiments (MISFISHIE)

One purpose of the biomedical literature is to report results in sufficient detail that the methods of data collection and analysis can be independently replicated and verified. Here we present reporting guidelines for gene expression localization experiments: the minimum information specification for in situ hybridization and immunohistochemistry experiments (MISFISHIE). MISFISHIE is modeled after the Minimum Information About a Microarray Experiment (MIAME) specification for microarray experiments. Both guidelines define what information should be reported without dictating a format for encoding that information. MISFISHIE describes six types of information to be provided for each experiment: experimental design, biomaterials and treatments, reporters, staining, imaging data and image characterizations. This specification has benefited the consortium within which it was developed and is expected to benefit the wider research community. We welcome feedback from the scientific community to help improve our proposal.

A relational database for trial-based behavioral experiments

We describe a relational database (RDB) structure suitable for trial-based experiments such as human psychophysics and neural recording studies in trained animals. An RDB is a collection of tables, each composed of columns. Some of the tables contain columns that reference specific columns of other tables. This referencing system links the tables to each other and makes it possible to extract any subset of the data with trivial commands. An equally important advantage of an RDB is that it imposes a consistent data format on applications that generate and analyze data. The result is a centralization and standardization of data storage that facilitates the pooling, cross-checking and re-analysis of data from various experiments. We present a robust RDB structure originally designed for neurophysiological data; however, it is abstract enough to accommodate data from a variety of trial-based experimental designs. Moreover, we demonstrated the advantages of this RDB structure and indicated its implementation in other laboratories.

A review of content-based image retrieval systems in medical applications-clinical benefits and future directions

Content-based visual information retrieval (CBVIR) or content-based image retrieval (CBIR) has been one on the most vivid research areas in the field of computer vision over the last 10 years. The availability of large and steadily growing amounts of visual and multimedia data, and the development of the Internet underline the need to create thematic access methods that offer more than simple text-based queries or requests based on matching exact database fields. Many programs and tools have been developed to formulate and execute queries based on the visual or audio content and to help browsing large multimedia repositories. Still, no general breakthrough has been achieved with respect to large varied databases with documents of differing sorts and with varying characteristics. Answers to many questions with respect to speed, semantic descriptors or objective image interpretations are still unanswered. In the medical field, images, and especially digital images, are produced in ever-increasing quantities and used for diagnostics and therapy. The Radiology Department of the University Hospital of Geneva alone produced more than 12,000 images a day in 2002. The cardiology is currently the second largest producer of digital images, especially with videos of cardiac catheterization ( approximately 1800 exams per year containing almost 2000 images each). The total amount of cardiologic image data produced in the Geneva University Hospital was around 1 TB in 2002. Endoscopic videos can equally produce enormous amounts of data. With digital imaging and communications in medicine (DICOM), a standard for image communication has been set and patient information can be stored with the actual image(s), although still a few problems prevail with respect to the standardization. In several articles, content-based access to medical images for supporting clinical decision-making has been proposed that would ease the management of clinical data and scenarios for the integration of content-based access methods into picture archiving and communication systems (PACS) have been created. This article gives an overview of available literature in the field of content-based access to medical image data and on the technologies used in the field. Section 1 gives an introduction into generic content-based image retrieval and the technologies used. Section 2 explains the propositions for the use of image retrieval in medical practice and the various approaches. Example systems and application areas are described. Section 3 describes the techniques used in the implemented systems, their datasets and evaluations. Section 4 identifies possible clinical benefits of image retrieval systems in clinical practice as well as in research and education. New research directions are being defined that can prove to be useful. This article also identifies explanations to some of the outlined problems in the field as it looks like many propositions for systems are made from the medical domain and research prototypes are developed in computer science departments using medical datasets. Still, there are very few systems that seem to be used in clinical practice. It needs to be stated as well that the goal is not, in general, to replace text-based retrieval methods as they exist at the moment but to complement them with visual search tools.

Three-Dimensional Electron Microscopy at Molecular Resolution

Emerging methods in cryo-electron microscopy allow determination of the three-dimensional architectures of objects ranging in size from small proteins to large eukaryotic cells, spanning a size range of more than 12 orders of magnitude. Advances in determining structures by "single particle" microscopy and by "electron tomography" provide exciting opportunities to describe the structures of subcellular assemblies that are either too large or too heterogeneous to be investigated by conventional crystallographic methods. Here, we review selected aspects of progress in structure determination by cryo-electron microscopy at molecular resolution, with a particular emphasis on topics at the interface of single particle and tomographic approaches. The rapid pace of development in this field suggests that comprehensive descriptions of the structures of whole cells and organelles in terms of the spatial arrangements of their molecular components may soon become routine.

Object oriented database and electronic notebook for transmission electron microscopy

As high-resolution biological transmission electron microscopy (TEM) has increased in popularity over recent years, the volume of data and number of projects underway has risen dramatically. A robust tool for effective data management is essential to efficiently process large data sets and extract maximum information from the available data. We present the Electron Microscopy Electronic Notebook (EMEN), a portable, object-oriented, web-based tool for TEM data archival and project management. EMEN has several unique features. First, the database is logically organized and annotated so multiple collaborators at different geographical locations can easily access and interpret the data without assistance. Second, the database was designed to provide flexibility to the user, so it can be used much as a lab notebook would be, while maintaining a structure suitable for data mining and direct interaction with data-processing software. Finally, as an object-oriented database, the database structure is dynamic and can be easily extended to incorporate information not defined in the original database specification.

FEMME database: topologic and geometric information of macromolecules

FEMME (Feature Extraction in a Multi-resolution Macromolecular Environment: http://www.biocomp.cnb.uam.es/FEMME/) database version 1.0 is a new bioinformatics data resource that collects topologic and geometric information obtained from macromolecular structures solved by three-dimensional electron microscopy (3D-EM). Although the FEMME database is focused on medium resolution data, the methodology employed (based on the so-called alpha-shape theory) is applicable to atomic resolution data as well. The alpha-shape representation allows the automatic extraction of structural features from 3D-EM volumes and their subsequent characterisation. FEMME is being populated with 3D-EM data stored in the electron microscopy database EMD-DB (http://www.ebi.ac.uk/msd/). However, and since the number of entries in EMD-DB is still relatively small, FEMME is also being populated in this initial phase with structural data from PDB and PQS databases (http://www.rcsb.org/pdb/ and pqs.ebi.ac.uk/, respectively) whose resolution has been lowered accordingly. Each FEMME entry contains macromolecular geometry and topology information with a detailed description of its structural features. Moreover, FEMME data have facilitated the study and development of a method to retrieve macromolecular structures by their structural content based on the combined use of spin images and neural networks with encouraging results. Therefore, the FEMME database constitutes a powerful tool that provides a uniform and automatic way of analysing volumes coming from 3D-EM that will hopefully help the scientific community to perform wide structural comparisons.

A novel three-dimensional variant of the watershed transform for segmentation of electron density maps

Electron density maps at moderate resolution are often difficult to interpret due to the lack of recognizable features. This is especially true for electron tomograms that suffer in addition to the resolution limitation from low signal-to-noise ratios. Reliable segmentation of such maps into smaller, manageable units can greatly facilitate interpretation. Here, we present a segmentation approach targeting three-dimensional electron density maps derived by electron microscopy. The approach consists of a novel three-dimensional variant of the immersion-based watershed algorithm. We tested the algorithm on calculated data and applied it to a wide variety of electron density maps ranging from reconstructions of single macromolecules to tomograms of subcellular structures. The results indicate that the algorithm is reliable, efficient, accurate, and applicable to a wide variety of biological problems.

A cell-centered database for electron tomographic data

Electron tomography is providing a wealth of 3D structural data on biological components ranging from molecules to cells. We are developing a web-accessible database tailored to high-resolution cellular level structural and protein localization data derived from electron tomography. The Cell Centered Database or CCDB is built on an object-relational framework using Oracle 8i and is housed on a server at the San Diego Supercomputer Center at the University of California, San Diego. Data can be deposited and accessed via a web interface. Each volume reconstruction is stored with a full set of descriptors along with tilt images and any derived products such as segmented objects and animations. Tomographic data are supplemented by high-resolution light microscopic data in order to provide correlated data on higher-order cellular and tissue structure. Every object segmented from a reconstruction is included as a distinct entity in the database along with measurements such as volume, surface area, diameter, and length and amount of protein labeling, allowing the querying of image-specific attributes. Data sets obtained in response to a CCDB query are retrieved via the Storage Resource Broker, a data management system for transparent access to local and distributed data collections. The CCDB is designed to provide a resource for structural biologists and to make tomographic data sets available to the scientific community at large.

Multi-resolution contour-based fitting of macromolecular structures

A novel contour-based matching criterion is presented for the quantitative docking of high-resolution structures of components into low-resolution maps of macromolecular complexes. The proposed Laplacian filter is combined with a six-dimensional search using fast Fourier transforms to rapidly scan the rigid-body degrees of freedom of a probe molecule relative to a fixed target density map. A comparison of the docking performance with the standard cross-correlation criterion demonstrates that contour matching with the Laplacian filter significantly extends the viable resolution range of correlation-based fitting to resolutions as low as 30 A. The gain in docking precision at medium to low resolution (15-30 A) is critical for image reconstructions from electron microscopy (EM). The new algorithm enables for the first time the reliable docking of smaller molecular components into EM densities of large biomolecular assemblies at such low resolutions. As an example of the practical effectiveness of contour-based fitting, a new pseudo-atomic model of a microtubule was constructed from a 20 A resolution EM map and from atomic structures of alpha and beta tubulin subunits.

VANQUIS, a system for the interactive semantic content analysis and spatio-temporal query by content of videos

Using the video metadata descriptors and data model defined in the accompanying paper (Shotton, D. M. et al. (2002) A metadata classification schema for semantic content analysis of videos. J. Microsc. 205, 33-42), we discuss how analysis of the content of scientific videos, and subsequent query by content of the resulting semantic metadata, can be enhanced by the use of an object-relational database. We illustrate this by describing VANQUIS, a Web-based prototype video analysis and query interface system for the interactive spatio-temporal analysis and subsequent query by content of videos. Using VANQUIS to generate standard SQL (structured query language) statements that address complex data types stored in an object-relational database, relationships between characters and events contained within and between videos can be identified, and the appropriate video segments containing these characters and events can be retrieved for viewing. We give examples of analysis and query implementation by using VANQUIS to analyse a biological microscopy video, and discuss the wider potential of this methodology for the analysis and query by content of videos containing more general subject matter.

A metadata classification schema for semantic content analysis of videos

Simple ancillary metadata, such as those encompassed by the 15 elements of the Dublin Core, may be sufficient and entirely appropriate for basic coarse-granularity cross-domain resource discovery. However, they are insufficient and inappropriate for content description of complex data types such as videos, which require more detailed relational models. We propose a metadata classification schema for the characterization of items and events in videos that permits subsequent query by content. Following MPEG-7 nomenclature, metadata intrinsic to the information content of the video are defined as either structural or semantic, where structural metadata are numerical feature primitives produced by analysing the colour, shape, texture, structure and motion within the video frames, whereas semantic metadata describe the locations and timings of individual items and particular actions or events in the video, and are thus of higher information value. In this paper, the semantic metadata required to describe the visual information content of videos are defined and classified into four distinct classes: Media Entities; Content Items; Events; and Supplementary Items, and three types of property tables are defined: Identity Tables; Spatio-Temporal Position Tables; and Event Tables, in which these metadata may be stored in a relational database.

The life sciences Global Image Database (GID)

Although a vast amount of life sciences data is generated in the form of images, most scientists still store images on extremely diverse and often incompatible storage media, without any type of metadata structure, and thus with no standard facility with which to conduct searches or analyses. Here we present a solution to unlock the value of scientific images. The Global Image Database (GID) is a web-based (http://www.gwer.ch/qv/gid/gid.ht m ) structured central repository for scientific annotated images. The GID was designed to manage images from a wide spectrum of imaging domains ranging from microscopy to automated screening. The annotations in the GID define the source experiment of the images by describing who the authors of the experiment are, when the images were created, the biological origin of the experimental sample and how the sample was processed for visualization. A collection of experimental imaging protocols provides details of the sample preparation, and labeling, or visualization procedures. In addition, the entries in the GID reference these imaging protocols with the probe sequences or antibody names used in labeling experiments. The GID annotations are searchable by field or globally. The query results are first shown as image thumbnail previews, enabling quick browsing prior to original-sized annotated image retrieval. The development of the GID continues, aiming at facilitating the management and exchange of image data in the scientific community, and at creating new query tools for mining image data.

Determining the structure of biological macromolecules by transmission electron microscopy, single particle analysis and 3D reconstruction

Single particle analysis and 3D reconstruction of molecules imaged by transmission electron microscopy have provided a wealth of medium to low resolution structures of biological molecules and macromolecular complexes, such as the ribosome, viruses, molecular chaperones and photosystem II. In this review, the principles of these techniques are introduced in a non-mathematical way, and single particle analysis is compared to other methods used for structural studies. In particular, the recent X-ray structures of the ribosome and of ribosomal subunits allow a critical comparison of single particle analysis and X-ray crystallography. This has emphasised the rapidity with which single particle analysis can produce medium resolution structures of complexes that are difficult to crystallise. Once crystals are available, X-ray crystallography can produce structures at a much higher resolution. The great similarities now seen between the structures obtained by the two techniques reinforce confidence in the use of single particle analysis and 3D reconstruction, and show that for electron cryo-microscopy structure distortion during sample preparation and imaging has not been a significant problem. The ability to analyse conformational flexibility and the ease with which time-resolved studies can be performed are significant advantages for single particle analysis. Future improvements in single particle analysis and electron microscopy should increase the attainable resolution. Combining single particle analysis of macromolecular complexes and electron tomography of subcellular structures with high-resolution X-ray structures may enable us to realise the ultimate dream of structural biology-a complete description of the macromolecular complexes of the cell in their different functional states.

Recent developments in cryo-electron microscopy reconstruction of single particles

Cryo-electron microscopy and single-particle 3D image reconstruction techniques have been used to examine a broad spectrum of samples ranging from 500 kDa protein complexes to large subcellular organelles. The attainable resolution has improved rapidly over the past few years. Structures of both symmetric and asymmetric assemblies at approximately 7.5 A have been reported. Together with X-ray crystallography, three-dimensional cryo-electron microscopy reconstruction has provided important insights into the function of many biological systems in their native biochemical contexts.

Cryo-electron microscopy reveals the functional organization of an enveloped virus, Semliki Forest virus

Semliki Forest virus serves as a paradigm for membrane fusion and assembly. Our icosahedral reconstruction combined 5276 particle images from 48 cryo-electron micrographs and determined the virion structure to 9 A resolution. The improved resolution of this map reveals an N-terminal arm linking capsid subunits and defines the spike-capsid interaction sites. It illustrates the paired helical nature of the transmembrane segments and the elongated structures connecting them to the spike projecting domains. A 10 A diameter density in the fusion protein lines the cavity at the center of the spike. These clearly visible features combine with the variation in order between the layers to provide a framework for understanding the structural changes during the life cycle of an enveloped virus.

Visualizing 3D data obtained from microscopy on the Internet

The Internet is a powerful communication medium increasingly exploited by business and science alike, especially in structural biology and bioinformatics. The traditional presentation of static two-dimensional images of real-world objects on the limited medium of paper can now be shown interactively in three dimensions. Many facets of this new capability have already been developed, particularly in the form of VRML (virtual reality modeling language), but there is a need to extend this capability for visualizing scientific data. Here we introduce a real-time isosurfacing node for VRML, based on the marching cube approach, allowing interactive isosurfacing. A second node does three-dimensional (3D) texture-based volume-rendering for a variety of representations. The use of computers in the microscopic and structural biosciences is extensive, and many scientific file formats exist. To overcome the problem of accessing such data from VRML and other tools, we implemented extensions to SGI's IFL (image format library). IFL is a file format abstraction layer defining communication between a program and a data file. These technologies are developed in support of the BioImage project, aiming to establish a database prototype for multidimensional microscopic data with the ability to view the data within a 3D interactive environment.

A framework for querying a database for structural information on 3D images of macromolecules: A web-based query-by-content prototype on the BioImage macromolecular server

Nowadays we are experiencing a remarkable growth in the number of databases that have become accessible over the Web. However, in a certain number of cases, for example, in the case of BioImage, this information is not of a textual nature, thus posing new challenges in the design of tools to handle these data. In this work, we concentrate on the development of new mechanisms aimed at "querying" these databases of complex data sets by their intrinsic content, rather than by their textual annotations only. We concentrate our efforts on a subset of BioImage containing 3D images (volumes) of biological macromolecules, implementing a first prototype of a "query-by-content" system. In the context of databases of complex data types the term query-by-content makes reference to those data modeling techniques in which user-defined functions aim at "understanding" (to some extent) the informational content of the data sets. In these systems the matching criteria introduced by the user are related to intrinsic features concerning the 3D images themselves, hence, complementing traditional queries by textual key words only. Efficient computational algorithms are required in order to "extract" structural information of the 3D images prior to storing them in the database. Also, easy-to-use interfaces should be implemented in order to obtain feedback from the expert. Our query-by-content prototype is used to construct a concrete query, making use of basic structural features, which are then evaluated over a set of three-dimensional images of biological macromolecules. This experimental implementation can be accessed via the Web at the BioImage server in Madrid, at http://www.bioimage.org/qbc/index.html.

Design and realization of an on-line database for multidimensional microscopic images of biological specimens

The BioImage database is a new scientific database for multidimensional microscopic images of biological specimens, which is available through the World Wide Web (WWW). The development of this database has followed an iterative approach, in which requirements and functionality have been revised and extended. The complexity and innovative use of the data meant that technical and biological expertise has been crucial in the initial design of the data model. A controlled vocabulary was introduced to ensure data consistency. Pointers are used to reference information stored in other databases. The data model was built using InfoModeler as a database design tool. The database management system is the Informix Dynamic Server with Universal Data Option. This object-relational system allows the handling of complex data using features such as collection types, inheritance, and user-defined data types. Informix datablades are used to provide additional functionality: the Web Integration Option enables WWW access to the database; the Video Foundation Blade provides functionality for video handling.

Video on the Internet: An introduction to the digital encoding, compression, and transmission of moving image data

In this paper, we seek to provide an introduction to the fast-moving field of digital video on the Internet, from the viewpoint of the biological microscopist who might wish to store or access videos, for instance in image databases such as the BioImage Database (http://www.bioimage.org). We describe and evaluate the principal methods used for encoding and compressing moving image data for digital storage and transmission over the Internet, which involve compromises between compression efficiency and retention of image fidelity, and describe the existing alternate software technologies for downloading or streaming compressed digitized videos using a Web browser. We report the results of experiments on video microscopy recordings and three-dimensional confocal animations of biological specimens to evaluate the compression efficiencies of the principal video compression-decompression algorithms (codecs) and to document the artefacts associated with each of them. Because MPEG-1 gives very high compression while yet retaining reasonable image quality, these studies lead us to recommend that video databases should store both a high-resolution original version of each video, ideally either uncompressed or losslessly compressed, and a separate edited and highly compressed MPEG-1 preview version that can be rapidly downloaded for interactive viewing by the database user.