Evaluation of orangeFS as a tool to achieve a high-performance storage and massively parallel processing in HPC cluster

Abstract

Nowadays, the requirements of modern software demand a greater computing power; numerous scientific and engineering applications request an increase in data storage capacity, be able to exchange of information at high speeds, as well as a faster data processing and better memory management. The implementation of personal computers interconnected to form a cluster and the use of distributed/parallel file systems are presented as a highly suitable alternative in the solution of complex problems that require these resources as their needs grow. The present work shows the evaluation of OrangeFS as a tool to achieve high performance storage and massive parallel processing. It takes advantage of the capacity of the hard drives included in each node of the cluster through the virtual file system and the network bandwidth, instead of having to add a more expensive type of storage. The tests carried out in a cluster with CentOS show that stripping a large file into small objects and distributed in parallel to the I/O servers provides that upcoming read/write operations runs faster; In addition, the use of the message passing interface in the development and execution of applications allows to increase the parallelism of the data in terms of processing due to the intervention of the multicore processor in each of the clients.