Automatic orientation cues for intuitive immersive interrogation of 3D echocardiographic images in virtual reality using deep learning

Funding Acknowledgements

Type of funding sources: Other. Main funding source(s): NIHR i4i funded 3D Heart Project Wellcome / EPSRC Centre for Medical Engineering (WT 203148/Z/16/Z)

onbehalf

3D Heart Project

Background/Introduction:

In echocardiography (echo), image orientation is determined by the position and direction of the transducer during examination, unlike cardiovascular imaging modalities such as CT or MRI. As a result, when echo images are first shown their display orientation has no external anatomical landmarks, thus the user has to identify anatomical landmarks in the regions of interest to understand the orientation. 

Purpose

To display an anatomical model of a standard heart, automatically aligned to an acquired patient’s 3D echo image - assisting image interpretation by quickly orienting the viewer. 

Methods

 

47 echo datasets from 13 pediatric patients with hypoplastic left heart syndrome (HLHS) were annotated by manually indicating the cardiac axes in both ES and ED volumes. We chose a view akin to the standard four chamber view in healthy hearts as the reference view, showing the AV valves, the right atrium, the left atrium and the hypoplastic ventricle. We then trained a deep convolutional neural network (CNN) to predict the rotation required for re-orientation to the reference view. Three data strategies were explored: 1) using 3D images to estimate orientation, 2) using three orthogonal slices only (2.5D approach) and 3) using the central slice only (2D approach). Three different algorithms were investigated: 1) an orientation classifier, 2) an orientation regressor with mean absolute angle error, and 3) an orientation regressor with geodesic loss. The data was split into training, validation and test sets with a 8:1:1 ratio. The training data was augmented by applying random rotations in the range [−10◦, +10◦] and updating labels accordingly. The model with smallest validation error was applied in tandem with the VR visualisation of the echo volumes. 

Results

Experimental results suggest that a 2.5D CNN classifying discrete integer angles performs best in re-orienting volumetric images to the reference view, with a mean absolute angle error on the test set of 9.0 deg (test set error ranges from 10.8 to 25.9 deg. for other algorithms). An HLHS volumetric data (left) is automatically aligned with the cardiac model (right) using our trained network when loaded in VR as shown in Figure 1. The volume and the model are both cropped at the referencing plane.

Conclusion

A deep learning network to align 3D echo images to a reference view was successfully trained and then integrated  into VR to reorient echo volumes to match a standard anatomical view. This work demonstrates the potential of combining artificial intelligence and VR in medical imaging, although further user study is expected to evaluate its clinical impact.

Caption for Abstract Picture

 The VR user interface informs the user of the 3D echo image orientation, automatically aligning it with an anatomical model, here showing the four chamber apical view.

Abstract Figure. Deep learning model integrated into VR

PD-L1 Expression in Sarcomas: An Immunohistochemical Study

Introduction/Objective

Immunotherapy is increasingly used for the treatment of metastatic melanoma and carcinomas. PD-I (programmed death 1) and its associated ligand (PD-L1) inhibits the activation of T lymphocytes. This inhibition can be impacted by a number of drugs. Response to these drugs is predicted by assessment of PD-L1 expression. PD-L1 expression varies between 19% and 92% in melanomas and carcinomas. PD-L1 expression is less well documented for sarcomas.

Methods

Fifty-one sarcomas of various histopathologic types were immunohistochemically stained (IHC) for PD-L1 using the antibody clone SP263 (Ventana, Tuscan, AZ). Membrane staining of tumor cells was quantitated as a percentage of total tumor cells. Sarcomas were judged as non-expressors (less than 1%) low-expressors (1 to 50%) and high expressors (greater than 50%). The percentage of each type of sarcoma judged as an expressor was determined.

Results

The percentage of each type of sarcoma expressing PD-L1 is reported and 20% of sarcomas expressed PD- L1. The percentage of sarcomas expressing PD-L1 varied significantly between types but the majority of sarcomas were non-expressors.

Conclusion

PD-L1 IHC expression is valuable in predicting response to immune-modulating drugs. Such therapies may be useful for treatment of metastatic sarcomas. Expression of PD-L1 in carcinomas and melanomas is variable ranging from 19% to 92%. In our study, a minority (20%) of sarcomas expressed PD-L1. Other studies have shown similar results with between 1.4 and 59% (average 24%) of sarcomas expressing PD-L1. Expression appears to be type specific. These finding suggest that PD-L1 based therapy may be less useful in sarcomas than in other malignancies.

[Endovascular aneurysm repair (EVAR) : Complication management]

A variety of complications may be experienced during or after endovascular repair (EVAR) of abdominal aortic aneurysms (AAA). Acute procedure-related complications include injury to the access vessels, improper endograft placement, ischaemic and systemic complications as well as the postimplantation syndrome. Femoral pseudoaneurysms, endograft migration, kinking or occlusion, endoleaks and endograft infection are complications that have been observed at follow-up. Meticulous patient selection and preprocedural work-up can reduce the incidence of these complications. The majority of the complications, acute as well those at follow-up, can be dealt with endoluminally. It is of essence, however, that a wide variety of catheters, guidewires, stents, stent-grafts and embolics are readily available in the repertoire of the interventionalist. In a small number of patients with endograft occlusion or infection or with bleeding complications/pseudoaneurysms of the access vessels, open surgical intervention may become necessary. To obtain optimal results, close interdisciplinary co-operation, especially between the interventionist, the vascular surgeon and the anaesthetist is mandatory.

Sperm whale long-range echolocation sounds revealed by ANTARES, a deep-sea neutrino telescope

Despite dedicated research has been carried out to adequately map the distribution of the sperm whale in the Mediterranean Sea, unlike other regions of the world, the species population status is still presently uncertain. The analysis of two years of continuous acoustic data provided by the ANTARES neutrino telescope revealed the year-round presence of sperm whales in the Ligurian Sea, probably associated with the availability of cephalopods in the region. The presence of the Ligurian Sea sperm whales was demonstrated through the real-time analysis of audio data streamed from a cabled-to-shore deep-sea observatory that allowed the hourly tracking of their long-range echolocation behaviour on the Internet. Interestingly, the same acoustic analysis indicated that the occurrence of surface shipping noise would apparently not condition the foraging behaviour of the sperm whale in the area, since shipping noise was almost always present when sperm whales were acoustically detected. The continuous presence of the sperm whale in the region confirms the ecological value of the Ligurian sea and the importance of ANTARES to help monitoring its ecosystems.

Classical swine fever (CSF) in wild boar: the role of the transplacental infection in the perpetuation of CSF

Thirty-four pregnant wild sows and their unborn progeny derived from an endemically infected population in the district of Nordvorpommern (Mecklenburg-Western Pomerania) were investigated for classical swine fever virus (CSFV) and antibodies. During the last 2.5 years of the epidemic, 20 out of 34 pregnant wild sows investigated were serologically positive. No CSFV or viral RNA was detected in organs derived from these animals and their progeny. This indicates that young wild boars persistently infected by transplacental virus transmission do not play a crucial role in the perpetuation of CSFV in wild boar. Other factors seem to be more important for the establishment of CSF as well as for virus perpetuation in the population.

Structural basis for pterin antagonism in nitric-oxide synthase. Development of novel 4-oxo-pteridine antagonists of (6R)-5,6,7,8-tetrahydrobiopterin

Pathological nitric oxide (NO) generation in sepsis, inflammation, and stroke may be therapeutically controlled by inhibiting NO synthases (NOS). Here we targeted the (6R)-5,6,7,8-tetrahydro-l-biopterin (H(4)Bip)-binding site of NOS, which, upon cofactor binding, maximally increases enzyme activity and NO production from substrate l-arginine. The first generation of H(4)Bip-based NOS inhibitors employed a 4-amino pharmacophore of H(4)Bip analogous to antifolates such as methotrexate. We developed a novel series of 4-oxo-pteridine derivatives that were screened for inhibition against neuronal NOS (NOS-I) and a structure-activity relationship was determined. To understand the structural basis for pterin antagonism, selected derivatives were docked into the NOS pterin binding cavity. Using a reduced 4-oxo-pteridine scaffold, derivatives with certain modifications such as electron-rich aromatic phenyl or benzoyl groups at the 5- and 6-positions, were discovered to markedly inhibit NOS-I, possibly due to hydrophobic and electrostatic interactions with Phe(462) and Ser(104), respectively, within the pterin binding pocket. One of the most effective 4-oxo compounds and, for comparisons an active 4-amino derivative, were then co-crystallized with the endothelial NOS (NOS-III) oxygenase domain and this structure solved to confirm the hypothetical binding modes. Collectively, these findings suggest (i) that, unlike the antifolate principle, the 4-amino substituent is not essential for developing pterin-based NOS inhibitors and (ii), provide a steric and electrostatic basis for their rational design.

Protein-protein interactions between keratin polypeptides expressed in the yeast two-hybrid system

Keratin filaments are obligatory heteropolymers of type I and type II keratin polypeptides. Specific type I/type II pairs are coexpressed in vivo. In contrast, all type I/type II pairs assemble into filaments in vitro, but the different pairs have different stabilities as demonstrated by treatment with increasing concentrations of urea. We have used the yeast two-hybrid system to analyse type I/type II interactions in a cellular context. We measured interactions between two different keratin pairs and we confirm the findings that K6+K17 form very stable heterodimers whereas K8+K18 interactions were weaker. The deletion of head domains did not reduce the strength of type I/type II interactions. Rather, the affinities were increased and the differences between the two pairs were retained in headless mutants. These findings argue against a major role of the head domains in directing heterodimer interactions and in defining heterodimer stabilities.

Postinjury scopolamine administration in experimental traumatic brain injury

A single bolus dose of scopolamine (1.0 mg/kg) or saline (equal volume) was injected (i.p.) at 15, 30 or 60 min after fluid percussion traumatic brain injury in the rat. Scopolamine administered at 15 min postinjury significantly reduced beam walking deficits and body weight loss assessed for 5 days after injury. Scopolamine treatment at 30 or 60 min postinjury had no effect on behavioral outcome assessed for 5 days after injury. Plasma concentrations of scopolamine were measured with a radioreceptor assay. The plasma half-life for scopolamine was 21.6 min in injured rats and 17.3 min in normal rats (P less than 0.05). These results, along with evidence from previous studies, suggest that a brief period of excessive neuronal excitation can produce relatively long-lasting behavioral deficits. The temporal effectiveness of receptor antagonist intervention in this process appears to be brief.

Cholesterol-transfer protein located in the intestinal brush-border membrane. Partial purification and characterization

Cholesterol absorption by small intestinal brush border membrane vesicles from taurocholate mixed micelles is a second-order reaction. From a comparison of reaction rates and order before and after proteinase K treatment of brush-border membrane vesicles, it is concluded that cholesterol absorption is protein-mediated. It is shown that the desorption of cholesterol from taurocholate mixed micelles is by a factor of about 10(4) faster than that from egg phosphatidylcholine bilayers. When brush border membrane vesicles are stored at room temperature, intrinsic proteinases are activated and proteins are liberated from the brush border membrane. These proteins collected in the supernatant catalyze cholesterol and phosphatidylcholine exchange between two populations of small unilamellar phospholipid vesicles. One of the active proteins present in the supernatant is purified by a two-step procedure involving gel filtration on Sephadex G-75 SF and affinity chromatography on a Nucleosil-phosphatidylcholine column. The protein thus obtained is pure by polyacrylamide gel electrophoresis in sodium dodecyl sulfate. It has an apparent molecular weight of slightly less than 14,000 as determined by sodium dodecyl sulfate polyacrylamide gel electrophoresis and a value of 11,500 determined by gel filtration on Sephadex G-75 SF.