# CryoBench Manual

This is a portal for detailed documentation on how to carry out the analyses described in the [CryoBench ](https://arxiv.org/pdf/2408.05526)[manuscript](https://arxiv.org/pdf/2408.05526) using the tools available at our [GitHub repository](https://github.com/ml-struct-bio/CryoBench/tree/main).

## Getting Started

Before running CryoBench, you will first install dependencies, as well as generate reconstruction results using your model(s) of choice:

{% content-ref url="getting-started/installation-instructions" %}
[installation-instructions](https://ez-lab.gitbook.io/cryobench/getting-started/installation-instructions)
{% endcontent-ref %}

{% content-ref url="getting-started/running-reconstruction-models" %}
[running-reconstruction-models](https://ez-lab.gitbook.io/cryobench/getting-started/running-reconstruction-models)
{% endcontent-ref %}

### Input Datasets

Datasets are available for download at Zenodo:

1. **Conf-het (IgG-1D, IgG-RL)**: <https://zenodo.org/records/11629428>.
2. **Comp-het (Ribosembly, Tomotwin-100)**: <https://zenodo.org/records/12528292>.
3. **Spike-MD**: <https://zenodo.org/records/12528784>.

#### Image Formation

See the [cryosim](https://github.com/ml-struct-bio/CryoBench/tree/main/cryosim) repository for scripts to generate synthetic cryo-EM particle images.

## Metrics

#### 1. Per-image FSCs

{% content-ref url="cryobench-manual/calculating-fsc-metrics" %}
[calculating-fsc-metrics](https://ez-lab.gitbook.io/cryobench/cryobench-manual/calculating-fsc-metrics)
{% endcontent-ref %}

Code can be found at the repo folder [metrics/fsc](https://github.com/ml-struct-bio/CryoBench/tree/refactor/metrics/fsc)

#### 2. Pose errors

{% content-ref url="cryobench-manual/calculating-pose-errors" %}
[calculating-pose-errors](https://ez-lab.gitbook.io/cryobench/cryobench-manual/calculating-pose-errors)
{% endcontent-ref %}

Code can be found at the repo folder [metrics/pose\_error](https://github.com/ml-struct-bio/CryoBench/tree/refactor/metrics/pose_error)

#### 3. UMAP visualization

{% content-ref url="cryobench-manual/visualizing-umap-clusterings-of-latent-labels" %}
[visualizing-umap-clusterings-of-latent-labels](https://ez-lab.gitbook.io/cryobench/cryobench-manual/visualizing-umap-clusterings-of-latent-labels)
{% endcontent-ref %}

Code can be found at the repo [metrics/visualization](https://github.com/ml-struct-bio/CryoBench/tree/main/metrics/visualization)

## News

**Dec. 2024** Refactored version 0.2 of CryoBench released, with new per-image FSC analyses; CryoBench presented in a spotlight session at [NeurIPS 2024](https://neurips.cc/virtual/2024/poster/97535)

**Aug. 2024** Initial version 0.1 of CryoBench released including per-conformation FSC analyses alongside initial version of manuscript on arXiv

## Contact

Please submit any bug reports, feature requests, or general usage feedback as a GitHub [issue](https://github.com/ml-struct-bio/CryoBench/issues).

## Reference

Jeon, Minkyu, et al. "CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM." NeurIPS 2024 Spotlight. \[[paper](https://arxiv.org/abs/2408.05526)]

```
@inproceedings{jeon2024cryobench,
 author = {Jeon, Minkyu and Raghu, Rishwanth and Astore, Miro and Woollard, Geoffrey and Feathers, Ryan and Kaz, Alkin and Hanson, Sonya M. and Cossio, Pilar and Zhong, Ellen D.},
 booktitle = {Advances in Neural Information Processing Systems},
 title = {CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM},
 year = {2024}
}
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://ez-lab.gitbook.io/cryobench/cryobench-manual.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.