CryoBench Manual

Datasets, metrics, and performance benchmarks for heterogeneous reconstruction in cryo-EM

This is a portal for detailed documentation on how to carry out the analyses described in the CryoBench manuscript using the tools available at our GitHub repository.

Getting Started

Before running CryoBench, you will first install dependencies, as well as generate reconstruction results using your model(s) of choice:

🛠️Installation Instructions🔭Running Reconstruction Models

Input Datasets

Datasets are available for download at Zenodo:

1. Conf-het (IgG-1D, IgG-RL): https://zenodo.org/records/11629428.

2. Comp-het (Ribosembly, Tomotwin-100): https://zenodo.org/records/12528292.

3. Spike-MD: https://zenodo.org/records/12528784.

Image Formation

See the cryosim repository for scripts to generate synthetic cryo-EM particle images.

Metrics

1. Per-image FSCs

📉Calculating FSC Metrics

Code can be found at the repo folder metrics/fsc

2. Pose errors

🤏Calculating Pose Errors

Code can be found at the repo folder metrics/pose_error

3. UMAP visualization

🗺️Visualizing UMAP Clusterings of Latent Labels

Code can be found at the repo metrics/visualization

News

Dec. 2024 Refactored version 0.2 of CryoBench released, with new per-image FSC analyses; CryoBench presented in a spotlight session at NeurIPS 2024

Aug. 2024 Initial version 0.1 of CryoBench released including per-conformation FSC analyses alongside initial version of manuscript on arXiv

Contact

Please submit any bug reports, feature requests, or general usage feedback as a GitHub issue.

Reference

Jeon, Minkyu, et al. "CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM." NeurIPS 2024 Spotlight. [paper]

@inproceedings{jeon2024cryobench,
 author = {Jeon, Minkyu and Raghu, Rishwanth and Astore, Miro and Woollard, Geoffrey and Feathers, Ryan and Kaz, Alkin and Hanson, Sonya M. and Cossio, Pilar and Zhong, Ellen D.},
 booktitle = {Advances in Neural Information Processing Systems},
 title = {CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM},
 year = {2024}
}