Python-based Hierarchical ENvironment for Integrated Xtallography
Documentation Home

Frequently asked questions for phenix.refine

Interpreting results

Note: questions specific to the GUI can be found in its documentation.


How can I make phenix.refine run faster?

There are currently two options for this:

  • compile from source with OpenMP support, which parallelizes the Fast Fourier Transform (FFT) used in structure factor and gradient calculation
  • specify the 'nproc' parameter (or equivalent control in the GUI), which parallelizes various grid searches by starting multiple Python jobs

The OpenMP parallelization is not particularly efficient, since the FFTs take less than half of the typical runtime; a speedup of 40% is usually the maximum. The process-level parallelization with 'nproc' is most useful when restraint weight optimization is enabled, since these procedures can be run as multiple separate processes. In these circumstances a speedup of 4-5x is possible. However, a run using default parameters will not significantly benefit from setting 'nproc'.

There are several limitations to these options:

  • OpenMP support is incompatible with the Phenix GUI
  • 'nproc' is incompatible with Windows
  • these approaches are incompatible with each other

When should I use simulated annealing?

Simulated annealing (SA) is most useful early in refinement, when the model is far from convergence. Manually built models, or MR solutions involving significant local conformational changes, are common inputs where SA can improve over simple gradient-driven refinement. It is generally less helpful later in refinement, and/or at high resolution.

When should I use rigid-body refinement?

This typically only needs to be performed once after molecular replacement, unless you dock in additional domains later. Continuing to use rigid-body refinement in later runs will not improve your structure, and only adds to the runtime.

What type of experimental data should I use for refinement?

Either amplitudes (F) or intensities (I) may be used in refinement (in any file format), but intensities will be used preferentially. Both anomalous and non-anomalous data are supported; there does not appear to be any particular benefit to model quality using one or the other with the default strategy. However, anomalous data may be used to refine anomalous scattering factors for heavy atoms, and anomalous difference map coefficients will automatically be created in the output MTZ file. For these reasons, anomalous data are recommended, but in general the R-factors will be similar.

Why does phenix.refine not use all data in refinement?

Reflections with abnormal values tend to reduce the performance of the refinement engine. These are identified based on several criteria (see Read 1999 for details) and filtered out at the beginning of each macro-cycle. You can prevent this by setting xray_data.remove_outliers=False.

How many macro-cycles should I run?

We recommend at least five to ensure convergence, but in some cases (especially poorly refined structures) considerably more may be required for optimal results. (The default is three macro-cycles due to speed considerations.)

How should I decide what resolution limit to use for my data?

This is a contentious subject, and not settled, although it is widely agreed that throwing away usable data simply to reduce the R-factors is not an acceptable practice. Traditional criteria for resolution limits include (among others) truncating the data at the resolution where the mean I/sigmaI falls below 2, but this may exclude valuable data. See the documentation on unmerged data (especially the cited references) for more details. In general it may be useful to include additional, weaker data in refinement in the final stages, since the weighting performed by maximum-likelihood refinement prevents these from degrading the model, and may improve it in some cases. (Note that more data will also cause phenix.refine to take longer to run, which often imposes a practical limit on resolution.)


When should I optimize the geometry and/or B-factor restraint weights?

This may be beneficial if the automatic weighting does not pick a good scale for the X-ray and restraint terms; this will often be recognizable by higher-than-expected bond and angle RMSDs. In general it rarely hurts to optimize the weights, and often results in a significantly better refinement, but it is several times slower than ordinary refinement unless you have a highly parallel system. However, we strongly recommend weight optimization in the final round of refinement, where it becomes essential to prevent overfitting.

How can I set the restraint weight manually?

Our usual response is: don't do this manually, use the automatic optimization instead. Although this takes significantly longer to run, in practice most users will spend an equivalent amount of time manually adjusting the weights by trial-and-error. If you are certain you need to have manual control, the parameters fix_wxc (for geometry restraints) and fix_wxu (for B-factor restraints) will set the weights.

When should I use non-crystallographic symmetry (NCS) restraints?

An approximate cutoff for NCS restraints is 2.0 Angstrom - at higher resolution the data alone are usually sufficient, but at lower resolution additional restraints are usually necessary. This is somewhat subjective due to the behavior of the global NCS restraints currently used by default in PHENIX, but will be addressed in future versions.

What is the difference between global and torsion NCS, and which one should I pick?

The global NCS restraints groups as rigid bodies, where all atoms in each group are expected to be related to the others by a single rotation and translation operation. This does not respect local deformations in the related molecules, which are common even at lower resolution. The torsion NCS restraints restraint dihedral angles instead, and allow them to be unrestrained if genuinely different. This will eventually become the default, since it often results in significantly better refinement.

Both NCS restraint types make my structure worse - what should I do?

In most cases this is due to the restraint of B-factors between NCS groups, which may actually have very different levels of disorder. Setting the NCS B-factor weight term to zero usually fixes the problem.

My resolution is X Angstroms; what should RMS(bonds) and RMS(angles) be?

This is somewhat controversial, but absolute upper limits for a well-refined protein structure at high resolution are typically 0.02 for RMS(bonds) and 2.0 for RMS(angles); usually they will be significantly lower. As resolution decreases the acceptable deviation from geometry restraints also decreases, so at 3.5 Angstrom, more appropriate values would be 0.01 and 1.0. We recommend using the POLYGON tool in the validation summary to judge your structure relative to others at similar resolutions.

Why does my output model have very poor geometry (RMS(bonds) and RMS(angles))?

This usually means that the automatic X-ray/geometry weighting did not work properly; this can sometimes happen if the starting model also has poor geometry. Optimizing the weight (optimize_xyz_weight=True, or equivalent GUI control in the "Refinement settings" tab) will usually fix this problem.

I have experimental phases for this structure, but the initial maps were poor. Should I still use phase restraints?

The experimental phases used to restraint refinement describe a bimodal probability distribution for every angle, rather than the single values used to generate a map. In most cases the additional restraints will not hurt refinement, and can often help.

Why is phenix.refine messing up my ligand geometry?

This often happens when the restraints were generated using ReadySet from a PDB file, and the ligand code is not recognizable in the Chemical Components Database. eLBOW will try to guess the molecular topology based on the coordinates alone, but this is imprecise and may not yield the desired result. For best results, restraints for non-standard ligands should be generated in eLBOW using a SMILES string or similar source of topology information.

What can I do to make my low-resolution structure better?

In general, if NCS is present in your structure, you should always use NCS restraints at low resolution; it is worth trying both the Cartesian (global) and torsion restraints to see which works best for your model. This alone usually helps with the geometry and overfitting, although it is rarely sufficient by itself. There are also several different types of restraint specifically designed to help with low-resolution refinement (consult the full phenix.refine manual page for details on each):

  • Reference model: this restraints the torsion angles to a high-resolution model, using a potential that gently releases angles which are genuinely different (allowing both local deformations or domain movements). This is usually the best option, and typically improves both R-free, overfitting, and validation statistics. However, it depends on the availability of a suitable reference model.
  • Secondary structure restraints: these are simply harmonic distance restraints between hydrogen-bonding atoms in protein helices and sheets, and nucleic acid base pairs. Annotation can be either automatic or manual. The primary limitation is currently the need for outlier filtering, which removes spurious bonds but also some genuine ones. It has a small effect on validation statistics, and typically none on R-free, but it does keep secondary structures stable.

I have ions very close to water molecules/protein atoms, and phenix.refine keeps tring to move them apart. How can I prevent this?

Use phenix.ready_set or phenix.metal_coordination to generate custom bond (and optionally, bond angle) restraints, which will be output to a parameter file ending in ".edits". If you are using the PHENIX GUI, there is a toolbar button for ReadySet in the phenix.refine interface, which will automatically load the output files for use in phenix.refine.

I had previously generated custom restraints using ReadySet in the PHENIX GUI, but the atoms have changed. Now phenix.refine crashes because it can't find the atom selections. How do I remove the old custom restraints?

In the Utilities menu, select "Clear custom restraints."


When should I use TLS?

TLS refinement is generally valid at any resolution; at low resolution, it may be best to make each chain a single group, instead of trying to split them into smaller pieces. However, it is best to wait until near the end of refinement to add TLS; until then you should refine with isotropic ADPs only.

Can I use both TLS and anisotropic ADPs?

Yes, but not for the same atoms - since TLS is essentially constrained anisotropic refinement, the two methods are mutually exclusive.

Where is the switch for anisotropic vs. isotropic B-factors/ADPs?

phenix.refine does not have a single global switch for defining ADP parameterization; rather, when the "Individual ADPs" strategy is defined, the program uses several criteria to determine how atoms should be treated:

  • By default, atoms that are anisotropic in the input model (i.e. have ANISOU records) will be kept anisotropic if the resolution is at least 1.7A; isotropic atoms remain isotropic.
  • If the resolution is lower, all atoms will be converted to isotropic unless otherwise specified.
  • The atom selections for isotropic and anisotropic atoms may also be defined explicitly. Typically, the hydrogens will always be isotropic; protein atoms will be anisotropic at high resolution, and sometimes waters as well.

In the GUI, several common parameterizations are pre-defined in the dialog for entering ADP selections. Note that although it is possible to combine all of the different ADP refinement strategies in a single run, the atom selections for individual and grouped refinement may not overlap, nor may the selections for anisotropic ADPs and TLS groups.

When should I refine anisotropic ADPs instead of TLS groups?

There is no precise cutoff where you should turn on anisotropic ADPs, but these are approximate guidelines:

  • At 1.5 Angstrom resolution or better, refine protein/nucleic acid/ligand heavy atoms (C/N/O or heavier) anisotropically, and waters isotropically.
  • At 1.2 Angstrom or better, waters should also be anisotropic.
  • It is almost never appropriate to refine hydrogens anisotropically.

There may be circumstances where anisotropic refinement is permissible at slightly lower resolution, but 1.7 Angstrom is probably a lower limit. Exceptions may sometimes be made for metal ions, since they scatter very strongly. As always, you should use the drop in R-free to judge whether the change in parameterization was appropriate - a decrease of 0.5% (i.e. 0.005) or better indicates success.

When should I refine grouped B-factors/ADPs instead of individual?

It is again difficult to give an exact rule, since it depends on several properties of the crystal including resolution, solvent content, presence of NCS, etc. In general, the higher the data-to-parameter ratio, the more likely individual ADPs are to work well. As an approximate example, consider these two hypothetical structures:

  • a 3.5 Angstrom structure of a single protein chain per asymmetric unit, with 38% solvent content
  • a 4.0 Angstrom structure of 3 NCS-related chains, with 70% solvent content

In this case, the latter structure can probably be refined with individual ADPs, while the former is more marginal. If in doubt, early rounds of refinement may be done with grouped ADPs, switching to individual as the structure nears convergence. In general, it is usually worth trying individual ADPs at some point; ultimately the effect on R-factors (primarily R-free, but also the gap between R-work and R-free) is the most important guideline.

Interpreting results

My resolution is X Angstroms, and my R/R-free are Y and Z. Am I done refining?

A partial answer can be obtained by looking at POLYGON, which plots histograms of statistics for PDB structures solved at similar resolutions, and compares these to the statistics for your output model. As a general rule, R-factors alone should not be used to decide if a structure is "done", but should be examined in combination with the validation report.

My resolution is X Angstroms, the structure is complete and well-validated, the maps look great, bu my R and R-free are still really high. How can I make them lower?

There are several possible explanations for this:

  • Twinning with a relatively small twin fraction (perhaps 10%) may not obviously affect map quality, but can still have a significant impact on R-factors. Run Xtriage to look for evidence of twinning, and possible twin laws for using in phenix.refine. (Note that we currently only support a single twin domain in refinement.)
  • Undiagnosed translational NCS (AKA translational pseudosymmetry) can have a similar effect. In this case, the diffraction images may have been processed incorrectly, thus missing the fainter spots that result from tNCS. The program LABELIT, or more specifically the command labelit.index (distributed with PHENIX, but not yet available in the GUI) can be run on the original images to provide appropriate indexing parameters.

The gap between R-work and R-free is very large - how can I fix this?

Overfitting during refinement is usually helped by adding more restraints, and/or tightening the standard geometry restraints. If the output geometry is already within reasonable limits (typically RMS(bonds) < 0.016 and RMS(angles) < 1.8), ideas to try include adding NCS restraints if NCS is present, secondary structure restraints, or reference model restraints (if a high-resolution structure is available). At lower resolutions (worse than 3.0A), it may also be prudent to try grouped ADP refinement, and if desperate, Ramachandran restraints. TLS refinement can often improve overfitting across a wide range of resolutions. However, depending on the degree of overfitting, it may be necessary to perform extensive manual rebuilding first. (Note that if the large R/R-free gap suddenly appears after refinement of a model that was previously not overfit, this usually indicates incorrect parameterization of the refinement, e.g. using anisotropic ADPs at an inappropriate resolution.)


When should I refine with hydrogens?

This is largely a matter of personal preference. Using explicit riding hydrogen atoms can improve geometry at any resolution; at higher resolutions, approximately 2 Angstrom or better, they will generally improve R-free as well. At atomic resolution (1.5 A or better) they should always be part of the final model. Note that at unless you have true subatomic resolution (0.9 A or better), the hydrogens should always be refined as "riding", meaning that their coordinates are defined by the heavy atoms, not individually refined.

What about water molecules?

Although phenix.ready_set includes an option to add hydrogens to waters, we do not recommend this unless you have exceptionally high resolution and/or neutron data.

Why are my hydrogen atoms added by PHENIX exploding when I run real-space refinement in Coot?

Versions of Coot prior to 0.6.2 used a version of the CCP4 monomer library with hydrogen atoms named according to the PDB format version 2 standard; PHENIX can recognize these, but defaults to PDB v.3. To reconcile the different conventions, you can download the newer version of the monomer library (currently available here) and set the environment variable COOT_REFMAC_LIB_DIR to point to the directory in which you unpack it.

Why can't PHENIX automatically remove hydrogens from the output PDB file?

We strongly discourage removing any atoms used in refinement from the model, as it makes reproducing the published R-factors very difficult and eliminates essential information about how the structure was refined.


How can I model a charged atom?

The charge occupies columns 79-80 at the end of each ATOM or HETATM record, immediately following the element symbol. The format is the number of electrons followed by the charge sign, for example "1-" or "2+". You can edit the PDB file manually to add this, but we recommend using phenix.pdbtools:

phenix.pdbtools model.pdb charge_selection="element Mn" charge=2

This is also available in the GUI under "Model tools". The effect of setting the charge will be to use modified scattering factors for X-ray refinement, which can be helpful if you notice difference density appearing at ion sites. Note that it will have no effect on the geometry, since phenix.refine does not take electrostatics into account.

I can't see density for an arginine sidechain beyond the C-gamma atom. How should I model it?

Opinion in the crystallography differs on the proper approach to disordered sidechains, with significant support for both of the following methods voiced on the PHENIX and CCP4 mailing lists:

  • Delete all atoms not visible in density, but leave the residue name alone. This is arguably the most conservative approach, as it avoids modeling any features not supported by the data, and it is consistent with the treatment of missing loops. The main disadvantage is aesthetic, since it is more difficult to visualize and interpret the biological effects of a structure with missing sidechains. Anecdotal evidence suggests that some non-crystallographers may be confused by this.
  • Pick an appropriate rotamer, and let the B-factors rise to account for disorder. This avoids truncated sidechains that may be mistaken for other residues, and is more realistic when interpreting surface electrostatics. The atomic B-factors and coordinates are actually refined against the data, however weak. It is potentially dangerous because it implies a greater level of confidence in these positions than is justified by the data. Additionally, the ADP restraints will keep the B-factors of nearby atoms similar (within some tolerance), which is normally essential for stable refinement but may artificially lower the B-factors of disordered sidechains.

A third approach, setting the occupancy of missing atoms to zero but leaving them in the model, is strongly disfavored, as the resulting positions and B-factors are entirely theoretical (but not immediately obvious as such).

Running phenix.model_vs_data (or the validation GUI) results in a slightly different R-factor than reported in the PDB header by phenix.refine. Shouldn't these be the same?

phenix.refine and phenix.model_vs_data use the same code to perform the bulk solvent correction and scaling, so they should report approximately the same R-factors given identical inputs. The discrepancy arises when taking a PDB file from refinement and running it back through phenix.model_vs_data. Because of the limited precision of the PDB format (three digits after the decimal point for coordinates, two digits for B-factors and occupancy), the atomic properties recorded in the PDB file will not be exactly the same as their actual refined values. In practice the difference in R-factors is statistically insignificant, however.

When should I perform anomalous group refinement?

This is most useful when you have a large number of strong anomalous scatterers, where map artifacts are common. In such cases a more precise modeling of atomic scattering may improve the R-factors as well. It is generally not necessary for routine cases, but it may be advantageous for identifying weak anomalous scatterers (such as ions from the crystallization condition) when used to calculate an LLG map at the end of refinement.


  • phenix.refine : Afonine, P.V., Grosse-Kunstleve, R.W., Echols, N., Headd, J.J., Moriarty, N.W., Mustyakimov, M., Terwilliger, T.C., Urzhumtsev, A., Zwart, P.H. & Adams, P.D. (2012) Towards automated crystallographic structure refinement with phenix.refine Acta Cryst. D68:352-367.

    Afonine, P.V., Grosse-Kunstleve, R.W. & Adams, P.D. (2005). CCP4 Newsl. 42, contribution 8.

  • Molprobity : Chen, V.B., Arendall III, W.B., Headd, J.J., Keedy, D.A., Immormino, R.M, Kapral, G.J., Murray, L.W., Richardson, J.S. & Richardson, D.C. (2010) MolProbity: all-atom structure validation for macromolecular crystallography Acta Cryst. D66:12-21.

    Davis IW, Leaver-Fay A, Chen VB, Block JN, Kapral GJ, Wang X, Murray LW, Arendall WB 3rd, Snoeyink J, Richardson JS, Richardson DC. MolProbity: all-atom contacts and structure validation for proteins and nucleic acids. Nucleic Acids Res. 2007 35:W375-83.

  • POLYGON : Urzhumtseva L, Afonine PV, Adams PD, Urzhumtsev A. Crystallographic model quality at a glance. Acta Cryst. 2009, D65:297-300.