News | People | Our Github

SQI - The Scalable Query Interface

 

1. Download

Please click on DOWNLOAD to get the full source, released under the LGPL license.

2. Synopsis

As a team effort by SciDAC Ultravis, we have produced a library for issuing queries on scientific datasets in parallel. The Scalable Query Interface (SQI) is a library for querying scientific data and closely ties in parallel I/O and analysis into the pipeline. SQI is quite similar to the Map-Reduce programming model. Users map data by issuing queries to extract their feature of interest. They can then reduce the data by sorting it on a dimension or variable and performing some type of analysis, most commonly spatial or temporal. SQI uses the BIL API for parallel I/O and currently only has support for the netCDF data format. Support will be added for raw and HDFx formats in the future.

3. Using SQI

A short manual about SQI and the API can be downloaded here.

4. Misc

For advanced implementation details, comments, questions, and bug reporting, please email Wes Kendall at kendall AT eecs dot utk dot edu. SQI makes heavy use of MPI, and it is recommended to go through MPI Tutorials if you are unfamiliar with MPI.

If you plan to use this interface in your work, we would love to hear about it. We also ask that you would reference "Terascale Data Organization for Discovering Multivariate Climatic Trends", Wesley Kendall, Markus Glatter, Jian Huang, Tom Peterka, Rob Latham, and Rob Ross, Proceedings of Supercomputing '09, Portland OR, Nov. 2009, a paper that used SQI for analysis of trends on over a terabyte of climate data.