Skip to main content

PSDI Indexing Service

The Indexing Service provides access to diverse data resources that we harvest, exposing them through unified interfaces such as the OPTIMADE API. It also supports the propagation of indexed data to the PSDI Cross Data Search.

The indexing system is designed to increase the findability and visibility of scientifically valuable data collections, as well as to provide programmatic access via machine-readable interfaces. The latter is especially important for automated workflows, reproducible computational analyses, and large-scale data processing.

Indexed Data Sources​

Please find the full list here.

The data sources exposed publicly via the OPTIMADE API are also listed in a machine-readable format as an OPTIMADE index meta-database.

Available Interfaces​

We currently expose the data only via the OPTIMADE API, a community-developed specification designed to facilitate interoperability between materials databases. However, we also plan to provide additional interfaces in the future.

FAQ​

1. Can you index my data?​

If you expose your data through widely used protocols such as OAI-PMH or APIs like the OPTIMADE API, we can readily index it.

We can also work with custom APIs or database views, particularly where the data is of significant importance to your community.

Another option is to create a community on the PSDI Community Data Collections and deposit your data there. In that case, we will also be able to harvest your data.

Yes. Input from the data owner may also be needed to ensure the data is presented optimally within the PSDI Cross Data Search.

3. How do you fetch the data?​

We set up a dedicated harvester for each data source to automatically retrieve data on a scheduled basis, depending on how frequently it is updated. All our pipelines are orchestrated by Apache Airflow.

4. How do you handle big files, e.g., molecular dynamics trajectories?​

In this case, we harvest the metadata describing these large files, including the URL pointing to their location.

5. How can I get in touch with you?​

Please contact our support team.

Related links:

About this page

If you would like to contribute content to the PSDI Knowledge Base or have feedback you would like to give on this guidance, please contact us.