Publication: Sistema de ficheros paralelo escalable para entornos "cluster"
Loading...
Identifiers
Publication date
2009
Defense date
2009-11-26
Authors
Tutors
Journal Title
Journal ISSN
Volume Title
Publisher
Impact
Export
Abstract
En la actualidad, las aplicaciones utilizadas en los entornos de computación de altas prestaciones (como por ejemplo simulaciones científicas o las dedicadas a la extracción de datos (data-mining), manejan ingentes cantidades de información, por lo que necesitan enormes recursos de cómputo y memoria. Las arquitecturas cluster se han convertido en la solución más común para ejecutar este tipo de aplicationes. Destacan dos tipos: las construidas por la agregación de componentes heteogéneos y las basadas en el uso de recursos homogéneos. Las arquitecturas heterogéneas enfrentan a la problemática de combinar distintas tecnologías hardware y software, ésta reside en la dificultad de integrar diferentes sistemas de almacenamiento. Por otra parte, los grandes clusters lidian con el desequilibrio entre nodos de cómputo y de E/S del sistema de almacenamiento, ya que al ser mayor el número de nodos de cómputo, la E/S se convierte en un cuello de botella para las aplicaciones. Las soluciones actuales para superar las dificultades que muestran estas arquitecturas cluster con respecto la adaptación de los nodos con el fin de eliminar la heterogeneidad en el primer caso, y la utilización de sistemas de ficheros paralelos así como el incremento en la infraestructura del sistema de almacenamiento en el segundo. Estas tendencias tienen enormes costes económicos y temporales en la adaptación y configuración de la infraestructura de E/S. La presente tesis propone solucionar estos problemas, estableciendo los siguientes objetivos: .Proporcionar un acceso homogéneo a los datos y usar tecnologías estándar de E/S para la construcción del sistema de almacenamiento en entornos heterogéneos. .Equilibrar la carga de E/S producida por las aplicaciones y eliminar la sobrecarga de los sistemas de almacenamiento en entornos de gran escala. Para alcanzar estos objetivos se han definido las siguientes soluciones: .Un sistema de ficheros paralelo multiplataforma basado en el uso de tecnologías estándar que persigue: la formación de sistemas de almacenamiento para clusters heterogéneos y proporcionar una plataforma que homogenice el acceso a los datos de las aplicaciones. .Una arquitectura de E/S basada en la ampliación de los esquemas de jerarquía de memoria al entorno de los grandes clusters, que incremente el número de nodos de E/S de los clusters para aumentar el paralelismo y reducir los accesos al sistema de almacenamiento. A lo largo del presente documento se detallan las soluciones propuestas, así como las evaluaciones de las mismas. __________________________________________________
Nowadays, the applications used in environments high performance computing, such as simulations scientific applications dedicated to data extraction (datamining), manage large amounts of information; needing huge computing and memory resources. Cluster architecture is the most common solution for HPC applications. There are two kinds of cluster archltectures: first, based on the aggregation of heterogeneous components and others, built with homogeneous components of large-supercomputers. Heterogeneous cluster architectures have a main problem, because it is built using different hardware and software technologies. There are no parallel file systems to adapt all of these diverse technologies available on these architectures. Moreover, homogeneous large-clusters have an I/O imbalance problem. This is due to the large number of compute nodes available compared to the few number of I/O nodes. This imbalance converts the I/O system on a bottleneck for HPC applications. The most common approach to remove the heterogeneity of the clusters is the adaptation of the nodes integrating technology to allow compatibility with new systems. Moreover, in the case of large clusters, traditional solutions are the use of parallel file systems and include changes in the infrastructure of the storage system, such as increasing the number of I/O nodes. In both cases, the solutions have high economic and time costs in the adaptation and configuration ofthe I/O infrastructure. This thesis proposes a solution for the problems presented above. The goals are the following: .Providing uniform data access using standard I/O technologies with the purpose of constructing storage systems in heterogeneous environments. .Balancing effective I/O load and eliminating the overhead of storage systems in large scale environments. To achieve these objectives we designed the following solutions: .A parallel file system platform based on the use of standard technologies fot the formation ofstorage systems for heterogeneousc lusters, providing further homogenice platform data access to applications. .An I/O architecture based on the extension of the diagrams of the hierarchy of memory to the large clusters environment, increasing the number of I/O nodes of the clusters to improve the parallelism and to reduce the l/O access to the storage. This document details the proposed solutions and shows the evaluations of them.
Nowadays, the applications used in environments high performance computing, such as simulations scientific applications dedicated to data extraction (datamining), manage large amounts of information; needing huge computing and memory resources. Cluster architecture is the most common solution for HPC applications. There are two kinds of cluster archltectures: first, based on the aggregation of heterogeneous components and others, built with homogeneous components of large-supercomputers. Heterogeneous cluster architectures have a main problem, because it is built using different hardware and software technologies. There are no parallel file systems to adapt all of these diverse technologies available on these architectures. Moreover, homogeneous large-clusters have an I/O imbalance problem. This is due to the large number of compute nodes available compared to the few number of I/O nodes. This imbalance converts the I/O system on a bottleneck for HPC applications. The most common approach to remove the heterogeneity of the clusters is the adaptation of the nodes integrating technology to allow compatibility with new systems. Moreover, in the case of large clusters, traditional solutions are the use of parallel file systems and include changes in the infrastructure of the storage system, such as increasing the number of I/O nodes. In both cases, the solutions have high economic and time costs in the adaptation and configuration ofthe I/O infrastructure. This thesis proposes a solution for the problems presented above. The goals are the following: .Providing uniform data access using standard I/O technologies with the purpose of constructing storage systems in heterogeneous environments. .Balancing effective I/O load and eliminating the overhead of storage systems in large scale environments. To achieve these objectives we designed the following solutions: .A parallel file system platform based on the use of standard technologies fot the formation ofstorage systems for heterogeneousc lusters, providing further homogenice platform data access to applications. .An I/O architecture based on the extension of the diagrams of the hierarchy of memory to the large clusters environment, increasing the number of I/O nodes of the clusters to improve the parallelism and to reduce the l/O access to the storage. This document details the proposed solutions and shows the evaluations of them.
Description
Keywords
Proceso en paralelo, Arquitectura de ordenadores, Arquitecturas cluster, Recuperación de la información