📉Calculating FSC Metrics

We show here how to use CryoBench tools to generate FSCs between volumes reconstructed using our example methods and the ground truth volumes for our datasets.

Before running these tools, make sure you have installed CryoBench and generated reconstruction results. Commands shown here use the same conda environments as described in the installation instructions!

Calculating per-image FSCs

We have created the cdrgn.py script as an example of how to calculate FSCs between volumes reconstructed by a cryoDRGN model and the ground truth volumes for each particle class in a CryoBench dataset. We generate volumes by selecting a given number of images from the dataset and then using the model decoder to return the volume for the latent space coordinates assigned to each image.

Note that we use 19 for the epoch number even though we ran training for 20 epochs as epoch numbers are 0-indexed:

$ conda activate cryodrgn_bench
(cryodrgn_bench)$ python ~/dev/CryoBench/metrics/fsc/cdrgn.py cryobench_input/IgG-1D/cryodrgn_fixed/ IgG-1D/gt_latents.pkl --gt-dir IgG-1D/vols/128_org/ --epoch 19 -o cryobench_output/IgG-1D/cryodrgn_fixed/ -n 100 --Apix 3.0

This will generate the following outputs in cryobench_output/IgG-1D/cryodrgn_fixed/:

Volumes vol_xxx.mrc generated by the cryoDRGN model for each of the chosen images (100 in this case due to -n 100)
A folder fsc_no_mask/ containing FSC curves calculated between the above reconstructed volumes and ground truth volumes stored in --gt-dir IgG-1D/vols/128_org/. These are stored as .txt files with a column for 1/resolution and a column for FSC values:

pixres fsc
0.0 1.0
0.0078125 1.0
0.015625 0.9962001
0.0234375 0.9762605
0.03125 0.9209269
0.0390625 0.8759735
...

This workflow is the same for cryoDRGN ab-initio outputs!

Masked FSCs and reusing generated volumes

You can apply a mask to the volumes before computing FSCs by passing a --mask argument to the cdrgn.py script:

(cryodrgn_bench)$ python ~/dev/CryoBench/metrics/fsc/cdrgn.py cryobineinch_input/IgG-1D/cryodrgn_fixed/ IgG-1D/gt_latents.pkl --gt-dir IgG-1D/vols/128_org/ --epoch 19 --mask IgG-1D/init_mask/mask.mrc -o cryobench_output/IgG-1D/cryodrgn_fixed_mask/ -n 100 --Apix 3.0

This will generate the same outputs as above — but the new FSCs will be placed in a subfolder named, in this case, fsc_mask/. In general, the _mask suffix will be replaced by the filename of the .mrc mask used.

You can also use cdrgn.py with an -o output folder that already contains generated volumes to recalculate FSCs with a mask, thus putting multiple sets of FSCs in the same output path. The script will skip volume generation in this case and proceed directly to FSC calculation unless the --overwrite flag is used.

Aligning volumes

Reconstructed volumes can be aligned to ground truth volumes using ChimeraX before calculating FSCs:

(cryodrgn_bench)$ python ~/dev/CryoBench/metrics/fsc/cdrgn.py cryobench_input/IgG-1D/cryodrgn_fixed/ IgG-1D/gt_latents.pkl --gt-dir IgG-1D/vols/128_org/ --epoch 19 --mask IgG-1D/init_mask/mask.mrc -o cryobench_output/IgG-1D/cryodrgn_fixed_aligned/ --serial-align -n 100 --Apix 3.0

This will produce the same output as above, but also with a vol.x.txt file for each generated volume containing a log of the alignment process.

Volumes can also be aligned in parallel by submitting jobs to a Slurm compute cluster (if available) using the --parallel-align flag, which is advisable is using a large number of images as aligning all volumes serially can be time-consuming!

Plotting FSCs

To generate visualizations of the FSC curves at cryobench_output/IgG-1D/cryodrgn_fixed/:

(cryodrgn_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cryobench_output/IgG-1D/cryodrgn_fixed/

This will plot the FSCs found in all fsc_* subfolders found in your CryoBench output folder, creating plots such as cryodrgn_fixed/fsc_no_mask.png and cryodrgn_fixed/fsc_no_mask_means.png:

Calculating Per-Conformation FSCs for Other Methods

In our repository we have included scripts for calculating FSC metrics for the other CryoBench example reconstruction methods using the original per-conformation method under metrics/fsc/old/per_conf/; we can provide scripts for calculating FSCs using the newer per-image method upon request!

DRGN-AI

$ conda activate drgnai_bench
(drgnai_bench)$ python ~/dev/CryoBench/metrics/fsc/old/per_conf/drgnai.py cryobench_input/drgnai/IgG-1D/ --epoch 20 --Apix 3.0 --gt-dir IgG-1D/vols/128_org/ --mask IgG-1D/init_mask/mask.mrc -o cBench-output/IgG-1D/drgnai_fixed/ --num-vols 100
(drgnai_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cBench-output/IgG-1D/drgnai_fixed/

OPUS-DSD

$ conda activate opusdsd_bench
(opusdsd_bench)$ python ~/dev/CryoBench/metrics/fsc/old/per_conf/opusdsd.py cryobench_input/opusdsd/IgG-1D/ --epoch 19 -o cryobench_output/IgG-1D/opusdsd/ --Apix 3.0 --gt-dir IgG-1D/vols/128_org/ --mask IgG-1D/init_mask/mask.mrc -o cBench-output/IgG-1D/opusdsd/ --num-vols 100
(opusdsd_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cBench-output/IgG-1D/opusdsd/

CryoSPARC: 3D Ab-Initio Classification

$ conda activate csparc_bench
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/old/per_conf/cryosparc_abinitio.py cryosparc/CryoBench/CS-cryobench/J10/ -o cryobench_output/cryosparc/3d_abinit/ --gt-dir IgG-1D/vols/128_org/ --mask IgG-1D/init_mask/mask.mrc --num-classes 10
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cryobench_output/cryosparc/3d_abinit/

CryoSPARC: 3D Classification

$ conda activate csparc_bench
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/old/per_conf/cryosparc_3dcls.py cryosparc/CryoBench/CS-cryobench/J5 -o cryobench_output/cryosparc/3dcls/ --gt-dir IgG-1D/vols/128_org/ --mask IgG-1D/init_mask/mask.mrc --num-classes 10
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cryobench_output/cryosparc/3dcls/

CryoSPARC: 3DFlex Train

$ conda activate csparc_bench
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/old/per_conf/cryosparc_3dflex.py cryosparc/CryoBench/CS-cryobench/J15 -o cryobench_output/cryosparc/3dflex/ --gt-dir IgG-1D/vols/128_org/ --mask IgG-1D/init_mask/mask.mrc --project-num P647 --job-num J15
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cryobench_output/cryosparc/3dflex/

CryoSPARC: 3D Variability

$ conda activate csparc_bench
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/old/per_conf/cryosparc_3dva.py cryosparc/CryoBench/CS-cryobench/J11 -o cryobench_output/cryosparc/3dva/ --gt-dir IgG-1D/vols/128_org/ --mask IgG-1D/init_mask/mask.mrc --num-vols 10
(csparc_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cryobench_output/cryosparc/3dva/

RECOVAR

$ conda activate recovar_bench
(recovar_bench)$ python ~/dev/CryoBench/metrics/fsc/old/per_conf/re_covar.py cryobench_input/recovar/IgG-1D/ --epoch 19 -o cryobench_output/IgG-1D/recovar/ --Apix 3.0 --gt-dir IgG-1D/vols/128_org/ --mask IgG-1D/init_mask/mask.mrc --num-vols 100
(recovar_bench)$ python ~/dev/CryoBench/metrics/fsc/plot_fsc.py cBench-output/IgG-1D/recovar/

PreviousCryoBench Manual NextCalculating Pose Errors

Last updated 6 months ago