1
|
Rosati R, Romeo L, Vargas VM, Gutierrez PA, Frontoni E, Hervas-Martinez C. Learning Ordinal-Hierarchical Constraints for Deep Learning Classifiers. IEEE Trans Neural Netw Learn Syst 2024; PP:1-14. [PMID: 38347692 DOI: 10.1109/tnnls.2024.3360641] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/16/2024]
Abstract
Real-world classification problems may disclose different hierarchical levels where the categories are displayed in an ordinal structure. However, no specific deep learning (DL) models simultaneously learn hierarchical and ordinal constraints while improving generalization performance. To fill this gap, we propose the introduction of two novel ordinal-hierarchical DL methodologies, namely, the hierarchical cumulative link model (HCLM) and hierarchical-ordinal binary decomposition (HOBD), which are able to model the ordinal structure within different hierarchical levels of the labels. In particular, we decompose the hierarchical-ordinal problem into local and global graph paths that may encode an ordinal constraint for each hierarchical level. Thus, we frame this problem as simultaneously minimizing global and local losses. Furthermore, the ordinal constraints are set by two approaches ordinal binary decomposition (OBD) and cumulative link model (CLM) within each global and local function. The effectiveness of the proposed approach is measured on four real-use case datasets concerning industrial, biomedical, computer vision, and financial domains. The extracted results demonstrate a statistically significant improvement to state-of-the-art nominal, ordinal, and hierarchical approaches.
Collapse
|
2
|
Vargas VM, Gutierrez PA, Barbero-Gomez J, Hervas-Martinez C. Activation Functions for Convolutional Neural Networks: Proposals and Experimental Study. IEEE Trans Neural Netw Learn Syst 2023; 34:1478-1488. [PMID: 34428161 DOI: 10.1109/tnnls.2021.3105444] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Activation functions lie at the core of every neural network model from shallow to deep convolutional neural networks. Their properties and characteristics shape the output range of each layer and, thus, their capabilities. Modern approaches rely mostly on a single function choice for the whole network, usually ReLU or other similar alternatives. In this work, we propose two new activation functions and analyze their properties and compare them with 17 different function proposals from recent literature on six distinct problems with different characteristics. The objective is to shed some light on their comparative performance. The results show that the proposed functions achieved better performance than the most commonly used ones.
Collapse
|
3
|
Casado-Adam A, Rodriguez-Ortiz L, Rufian-Peña S, Muñoz-Casares C, Caro-Cuenca T, Ortega-Salas R, Fernandez-Peralbo MA, Luque-de-Castro MD, Sanchez-Hidalgo JM, Hervas-Martinez C, Romero-Ruiz A, Briceño J, Arjona-Sánchez Á. The Role of Intraperitoneal Intraoperative Chemotherapy with Paclitaxel in the Surgical Treatment of Peritoneal Carcinomatosis from Ovarian Cancer—Hyperthermia versus Normothermia: A Randomized Controlled Trial. J Clin Med 2022; 11:jcm11195785. [PMID: 36233653 PMCID: PMC9570602 DOI: 10.3390/jcm11195785] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2022] [Revised: 09/20/2022] [Accepted: 09/28/2022] [Indexed: 11/29/2022] Open
Abstract
Background: The treatment of ovarian carcinomatosis with cytoreductive surgery and HIPEC is still controversial. The effect and pharmacokinetics of the chemotherapeutics used (especially taxanes) are currently under consideration. Methods: A phase II, simple blind and randomized controlled trial (NTC02739698) was performed. The trial included 32 patients with primary or recurrent ovarian carcinomatosis undergoing cytoreductive surgery (CRS) and intraoperative intraperitoneal chemotherapy with paclitaxel (PTX): 16 in hyperthermic (42–43 °C) and 16 in normothermic (37 °C) conditions. Tissue, serum and plasma samples were taken in every patient before and after intraperitoneal chemotherapy to measure the concentration of PTX. To analyze the immunohistochemical profile of p53, p27, p21, ki67, PCNA and caspase-3 and the pathological response, a scale of intensity and percentage of expression and a grouped Miller and Payne system were used, respectively. Perioperative characteristics and morbi-mortality were also analyzed. Results: The main characteristics of patients, surgical morbidity, hemotoxicity and nephrotoxicity were similar in both groups. The concentration of paclitaxel in the tissue was higher than that observed in plasma and serum, although no statistically significant differences were found between the two groups. No statistically significant association regarding pathological response and apoptosis (caspase-3) between both groups was proved. There were no statistically significant differences between the normothermic and the hyperthermic group for pathological response and apoptosis. Conclusions: The use of intraperitoneal PTX has proven adequate pharmacokinetics with reduction of cell cycle and proliferation markers globally without finding statistically significant differences between its administration under hyperthermia versus normothermia conditions.
Collapse
Affiliation(s)
- Angela Casado-Adam
- Oncologic and Pancreatic Surgery Unit, University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
| | - Lidia Rodriguez-Ortiz
- Oncologic and Pancreatic Surgery Unit, University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
| | - Sebastian Rufian-Peña
- Oncologic and Pancreatic Surgery Unit, University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
| | - Cristobal Muñoz-Casares
- Oncologic and Pancreatic Surgery Unit, University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
| | - Teresa Caro-Cuenca
- Department of Pathology, Reina Sofía University Hospital, 14004 Córdoba, Spain
- Maimonides Biomedical Research Institute of Córdoba (IMIBIC), 14004 Córdoba, Spain
| | - Rosa Ortega-Salas
- Department of Pathology, Reina Sofía University Hospital, 14004 Córdoba, Spain
- Maimonides Biomedical Research Institute of Córdoba (IMIBIC), 14004 Córdoba, Spain
| | | | - Maria Dolores Luque-de-Castro
- Department of Analytical Chemistry, Campus of Rabanales, University of Córdoba, Annex Marie Curie Building, 14071 Córdoba, Spain
| | - Juan M. Sanchez-Hidalgo
- Oncologic and Pancreatic Surgery Unit, University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
| | - Cesar Hervas-Martinez
- Department of Computer Science and Numerical Analysis, University of Córdoba, 14071 Córdoba, Spain
| | - Antonio Romero-Ruiz
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- Department of Biochemistry and Molecular Biology, University of Córdoba, 14004 Córdoba, Spain
| | - Javier Briceño
- Oncologic and Pancreatic Surgery Unit, University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
| | - Álvaro Arjona-Sánchez
- Oncologic and Pancreatic Surgery Unit, University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- CIBERehd, Maimonides Biomedical Research Institute of Cordoba (IMIBIC), University Hospital Reina Sofia, Avda Menéndez Pidal s/n, 14004 Córdoba, Spain
- Correspondence:
| |
Collapse
|
4
|
Guijo-Rubio D, Duran-Rosal AM, Gutierrez PA, Troncoso A, Hervas-Martinez C. Time-Series Clustering Based on the Characterization of Segment Typologies. IEEE Trans Cybern 2021; 51:5409-5422. [PMID: 31945011 DOI: 10.1109/tcyb.2019.2962584] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Time-series clustering is the process of grouping time series with respect to their similarity or characteristics. Previous approaches usually combine a specific distance measure for time series and a standard clustering method. However, these approaches do not take the similarity of the different subsequences of each time series into account, which can be used to better compare the time-series objects of the dataset. In this article, we propose a novel technique of time-series clustering consisting of two clustering stages. In a first step, a least-squares polynomial segmentation procedure is applied to each time series, which is based on a growing window technique that returns different-length segments. Then, all of the segments are projected into the same dimensional space, based on the coefficients of the model that approximates the segment and a set of statistical features. After mapping, a first hierarchical clustering phase is applied to all mapped segments, returning groups of segments for each time series. These clusters are used to represent all time series in the same dimensional space, after defining another specific mapping process. In a second and final clustering stage, all the time-series objects are grouped. We consider internal clustering quality to automatically adjust the main parameter of the algorithm, which is an error threshold for the segmentation. The results obtained on 84 datasets from the UCR Time Series Classification Archive have been compared against three state-of-the-art methods, showing that the performance of this methodology is very promising, especially on larger datasets.
Collapse
|
5
|
Perez-Ortiz M, Gutierrez PA, Tino P, Hervas-Martinez C. Oversampling the Minority Class in the Feature Space. IEEE Trans Neural Netw Learn Syst 2016; 27:1947-1961. [PMID: 26316222 DOI: 10.1109/tnnls.2015.2461436] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
The imbalanced nature of some real-world data is one of the current challenges for machine learning researchers. One common approach oversamples the minority class through convex combination of its patterns. We explore the general idea of synthetic oversampling in the feature space induced by a kernel function (as opposed to input space). If the kernel function matches the underlying problem, the classes will be linearly separable and synthetically generated patterns will lie on the minority class region. Since the feature space is not directly accessible, we use the empirical feature space (EFS) (a Euclidean space isomorphic to the feature space) for oversampling purposes. The proposed method is framed in the context of support vector machines, where the imbalanced data sets can pose a serious hindrance. The idea is investigated in three scenarios: 1) oversampling in the full and reduced-rank EFSs; 2) a kernel learning technique maximizing the data class separation to study the influence of the feature space structure (implicitly defined by the kernel function); and 3) a unified framework for preferential oversampling that spans some of the previous approaches in the literature. We support our investigation with extensive experiments over 50 imbalanced data sets.
Collapse
|
6
|
Saez A, Sanchez-Monedero J, Gutierrez PA, Hervas-Martinez C. Machine Learning Methods for Binary and Multiclass Classification of Melanoma Thickness From Dermoscopic Images. IEEE Trans Med Imaging 2016; 35:1036-1045. [PMID: 26672031 DOI: 10.1109/tmi.2015.2506270] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Thickness of the melanoma is the most important factor associated with survival in patients with melanoma. It is most commonly reported as a measurement of depth given in millimeters (mm) and computed by means of pathological examination after a biopsy of the suspected lesion. In order to avoid the use of an invasive method in the estimation of the thickness of melanoma before surgery, we propose a computational image analysis system from dermoscopic images. The proposed feature extraction is based on the clinical findings that correlate certain characteristics present in dermoscopic images and tumor depth. Two supervised classification schemes are proposed: a binary classification in which melanomas are classified into thin or thick, and a three-class scheme (thin, intermediate, and thick). The performance of several nominal classification methods, including a recent interpretable method combining logistic regression with artificial neural networks (Logistic regression using Initial variables and Product Units, LIPU), is compared. For the three-class problem, a set of ordinal classification methods (considering ordering relation between the three classes) is included. For the binary case, LIPU outperforms all the other methods with an accuracy of 77.6%, while, for the second scheme, although LIPU reports the highest overall accuracy, the ordinal classification methods achieve a better balance between the performances of all classes.
Collapse
|
7
|
Garcia-Pedrajas N, Hervas-Martinez C, Munoz-Perez J. COVNET: a cooperative coevolutionary model for evolving artificial neural networks. ACTA ACUST UNITED AC 2012; 14:575-96. [PMID: 18238040 DOI: 10.1109/tnn.2003.810618] [Citation(s) in RCA: 69] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
This paper presents COVNET, a new cooperative coevolutionary model for evolving artificial neural networks. This model is based on the idea of coevolving subnetworks that must cooperate to form a solution for a specific problem, instead of evolving complete networks. The combination of this subnetworks is part of a coevolutionary process. The best combinations of subnetworks must be evolved together with the coevolution of the subnetworks. Several subpopulations of subnetworks coevolve cooperatively and genetically isolated. The individual of every subpopulation are combined to form whole networks. This is a different approach from most current models of evolutionary neural networks which try to develop whole networks. COVNET places as few restrictions as possible over the network structure, allowing the model to reach a wide variety of architectures during the evolution and to be easily extensible to other kind of neural networks. The performance of the model in solving three real problems of classification is compared with a modular network, the adaptive mixture of experts and with the results presented in the bibliography. COVNET has shown better generalization and produced smaller networks than the adaptive mixture of experts and has also achieved results, at least, comparable with the results in the bibliography.
Collapse
|