A framework for scientific data indexing, searching and sharing
Apoorva Mohite · 2024
Scientific data continues to grow. Wildfire simulation experiments performed by the WIRC team at SJSU have generated over 138 TB of data so far and it is expected to keep growing. It becomes difficult for researchers to search through that data to find the data of their interest. This data is stored on an HPC cluster that external users do not have access to. The WIRC team also conducts experiments and publishes their research, but the size of data makes it difficult to share these datasets. This project introduces a novel solution to indexing scientific data, searching through the data and sharing it.