[phenixbb] automatic weight optimization

[Pavel Afonine]

Hi Michael,

> I have a few questions about the automatic weight optimization in phenix.refine. I have read the manual, but I feel like maybe I'm missing something. I apologize in advance if these are naive questions, but here goes. I ran phenix.refine with a .def file, in which I set the following parameters:
>
>   target_weights {
>      optimize_xyz_weight = True
>      optimize_adp_weight = True
>      wxu_scale = 2.72
>   }
>
>
> My goal was to optimize both xyz and adp weights. In the log file, I see something like this:
>
>
> ===================== xyz refinement (lbfgs minimization) =====================
>
>      R-FACTORS           RMSD     CLASH  RAMA  ROTA CBET WEIGHT    TARGETS
>   work  free  delta  bonds  angl                                  data   restr
> 21.06 20.38  -0.67  0.033   2.0    6.1   0.0   6.6    0   none  0.154  0.1685
> 19.79 21.26   1.47  0.012   1.3    7.6   0.0   5.8    0  2.494  0.131  0.0499
>
> Legend:
>    - first line corresponds to starting state (before refinement)
>    - R-factors reported in percent
>    - delta is Rfree-Rwork in percent
>    - CLASH is all-atom Molprobity clashscore
>    - ROTA is percent of side-chain rotamer outliers
>    - RAMA is percent of Ramachandran plot outliers
>    - CBET is number of Cbeta deviations
>    - WEIGHT is relative weight between X-ray (or neutron) target and restraints
>    - TARGETS: the values of X-ray (or neutron) and restraints target functions
>
> ================================ ADP refinement ===============================
>
>
>                   ----------Individual ADP refinement----------
>
>      R-FACTORS      <Bi-Bj>  <B>   WEIGHT       TARGETS
>   work  free  delta                           data restr
> 19.79 21.26   1.47  1.909  20.062  0.661    0.131
> 19.80 21.28   1.49  1.761  20.011  0.007    0.130
> 19.77 21.41   1.64  1.838  19.884  0.030    0.125
> 20.30 22.63   2.34  2.369  19.168  0.122    0.111
> 20.17 22.82   2.64  2.886  18.929  0.243    0.107
> 20.07 23.04   2.97  3.771  18.737  0.365    0.104
> 20.04 23.14   3.10  4.106  18.751  0.486    0.103
> 20.00 23.19   3.19  4.496  18.794  0.608    0.103
> 19.98 23.23   3.25  4.846  18.851  0.729    0.102
> 19.97 23.27   3.29  5.115  18.940  0.851    0.102
> 19.95 23.30   3.36  5.420  18.988  0.972    0.101
> 19.94 23.33   3.39  5.579  19.008  1.094    0.101
> 19.93 23.29   3.36  5.598  18.917  1.215    0.101
>    max suggested <Bi-Bj> for this run:   10.00
>    max allowed Rfree-Rwork gap:   5.0
>    range of equivalent Rfree:   0.0
> Best ADP weight:    0.007
> Accepted refinement result:
> 19.80 21.28   1.49  1.761  20.011  0.007    0.130
>
>
> The output kind of makes me feel like the adp weight is being optimized, but the xyz weight might not be. There is a list of statistics for varying values of the adp weight, but there is no similar list in the output from the xyz refinement stages.
>
> So my questions are:
>
> Is the xyz weight being optimized?

since starting Rfree (=20.38) is less than Rwork (=21.06), weight 
optimization is skipped for this macro-cycle. Once the gap between Rwork 
and Rfree becomes greater than a certain threshold (I forgot what I put 
as the exact value for this, perhaps 1..2%) then weight optimization 
will start taking place.

Sorry, it's not in the original paper that describes it:

"Improved target weight optimization in phenix.refine"
http://www.phenix-online.org/newsletter/

since we realized we need to do it later after the publication.

Also, if
     optimize_xyz_weight = True
     optimize_adp_weight = True
then any values set for wxu_scale or wxc_scale are ignored.

Also note if you have multiple CPU machine you can use 
nproc=NUMBER_OF_CPUs - that will speed up refinement (with weights 
optimization) significantly!

> Did I miss something in the input that is required to turn on xyz weight optimization?

No, most likely you didn't !

>   How can I check to make sure the xyz weight optimization is running?

If it's not obvious from the output logs then just do what you just did 
- ask us directly or on mailing list!

All the best,
Pavel