> For the complete documentation index, see [llms.txt](https://ez-lab.gitbook.io/cryobench/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ez-lab.gitbook.io/cryobench/cryobench-manual.md).

# CryoBench Manual

This is a portal for detailed documentation on how to carry out the analyses described in the [CryoBench ](https://arxiv.org/pdf/2408.05526)[manuscript](https://arxiv.org/pdf/2408.05526) using the tools available at our [GitHub repository](https://github.com/ml-struct-bio/CryoBench/tree/main).

## Getting Started

Before running CryoBench, you will first install dependencies, as well as generate reconstruction results using your model(s) of choice:

{% content-ref url="/pages/LAZAPl6Ew7GGLkFRl7Mi" %}
[Installation Instructions](/cryobench/getting-started/installation-instructions.md)
{% endcontent-ref %}

{% content-ref url="/pages/q1hvvE4zger2wOV2thYz" %}
[Running Reconstruction Models](/cryobench/getting-started/running-reconstruction-models.md)
{% endcontent-ref %}

### Input Datasets

Datasets are available for download at Zenodo:

1. **Conf-het (IgG-1D, IgG-RL)**: <https://zenodo.org/records/11629428>.
2. **Comp-het (Ribosembly, Tomotwin-100)**: <https://zenodo.org/records/12528292>.
3. **Spike-MD**: <https://zenodo.org/records/12528784>.

#### Image Formation

See the [cryosim](https://github.com/ml-struct-bio/CryoBench/tree/main/cryosim) repository for scripts to generate synthetic cryo-EM particle images.

## Metrics

#### 1. Per-image FSCs

{% content-ref url="/pages/nPvSJqfa8G7WKeUJ23lG" %}
[Calculating FSC Metrics](/cryobench/cryobench-manual/calculating-fsc-metrics.md)
{% endcontent-ref %}

Code can be found at the repo folder [metrics/fsc](https://github.com/ml-struct-bio/CryoBench/tree/refactor/metrics/fsc)

#### 2. Pose errors

{% content-ref url="/pages/1a8ISQlzPwCuUYktNQ2B" %}
[Calculating Pose Errors](/cryobench/cryobench-manual/calculating-pose-errors.md)
{% endcontent-ref %}

Code can be found at the repo folder [metrics/pose\_error](https://github.com/ml-struct-bio/CryoBench/tree/refactor/metrics/pose_error)

#### 3. UMAP visualization

{% content-ref url="/pages/wSKzZ5rqi74mtxOorDAm" %}
[Visualizing UMAP Clusterings of Latent Labels](/cryobench/cryobench-manual/visualizing-umap-clusterings-of-latent-labels.md)
{% endcontent-ref %}

Code can be found at the repo [metrics/visualization](https://github.com/ml-struct-bio/CryoBench/tree/main/metrics/visualization)

## News

**Dec. 2024** Refactored version 0.2 of CryoBench released, with new per-image FSC analyses; CryoBench presented in a spotlight session at [NeurIPS 2024](https://neurips.cc/virtual/2024/poster/97535)

**Aug. 2024** Initial version 0.1 of CryoBench released including per-conformation FSC analyses alongside initial version of manuscript on arXiv

## Contact

Please submit any bug reports, feature requests, or general usage feedback as a GitHub [issue](https://github.com/ml-struct-bio/CryoBench/issues).

## Reference

Jeon, Minkyu, et al. "CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM." NeurIPS 2024 Spotlight. \[[paper](https://arxiv.org/abs/2408.05526)]

```
@inproceedings{jeon2024cryobench,
 author = {Jeon, Minkyu and Raghu, Rishwanth and Astore, Miro and Woollard, Geoffrey and Feathers, Ryan and Kaz, Alkin and Hanson, Sonya M. and Cossio, Pilar and Zhong, Ellen D.},
 booktitle = {Advances in Neural Information Processing Systems},
 title = {CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM},
 year = {2024}
}
```