[phenixbb] temp files

[Tom Terwilliger]

Hi James,

There is no overall Phenix temp directory specification, but most of the
temp_dir usage is from autosol/autobuild/ligandfit/map_to_model.  Each of
these has the keyword "temp_dir=xxxx" which you should be
able to set to any directory you want (and local is better as you note).
Most programs using a temp_dir also have a keyword clean_up=True as well.

All the best,
Tom T

On Sat, Apr 24, 2021 at 10:51 AM James Holton <jmholton at lbl.gov> wrote:

> Thank you Li-Wei
>
> Definitely not placing blame on one program. Phenix.autobuild is another
> big temp file producer.  So is XDS. Clearly this ligand run was a case
> of a misconfigured, runaway task that never finished. However, the files
> lingered on disk, eating up inodes for 3 years!
>
> The reason I'm asking is I think there are significant performance
> increases to be gained by using fast, local storage for scratch files.
> This is not just in speed but storage and overall system/cluster
> performance.  Very few things are more expensive than an NFS write!
>
> Does anyone know how to change the default temp file location across
> phenix ?  Is this a cctbx thing?
>
> Thanks
>
> -James
>
>
> On 4/23/2021 9:38 PM, Li-Wei Hung wrote:
> > Hi James,
> >
> > I'll leave the global Phenix temp aspect to Billy.
> > For ligand identification specifically, the working directory is where
> > all the files are located.  The program will purge most of the
> > intermediate files upon completion. If the user interrupted the runs
> > or if the program crashed at certain spots, the purge mechanism might
> > not kick in. Even so, it'd take many runs to accumulate 20e6 (2e7?)
> > files. In any case, you've got a point and I'll look into salvaging
> > intermediate files of ligand identification as soon as they are not
> > needed in the process.
> >
> > Thanks,
> >
> > Li-Wei
> >
> > On 4/23/2021 7:03 PM, James Holton wrote:
> >> Hello all,
> >>
> >> Is there a way to configure phenix at install time (or perhaps
> >> post-install) to put temporary files under /tmp ?  I just had to
> >> delete 20e6 temp files over NFS from a single user's phenix ligand
> >> identification run.  The delete took almost a month.
> >>
> >> Apologies if I am neglecting to look somewhere obvious in the
> >> documentation,
> >>
> >> Happy Weekend!
> >>
> >> -James Holton
> >> MAD Scientist
> >>
> >> _______________________________________________
> >> phenixbb mailing list
> >> phenixbb at phenix-online.org
> >> http://phenix-online.org/mailman/listinfo/phenixbb
> >> Unsubscribe: phenixbb-leave at phenix-online.org
> >
>
> _______________________________________________
> phenixbb mailing list
> phenixbb at phenix-online.org
> http://phenix-online.org/mailman/listinfo/phenixbb
> Unsubscribe: phenixbb-leave at phenix-online.org



-- 
Thomas C Terwilliger
Laboratory Fellow, Los Alamos National Laboratory
Senior Scientist, New Mexico Consortium
100 Entrada Dr, Los Alamos, NM 87544
Email: tterwilliger at newmexicoconsortium.org
Tel: 505-431-0010
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phenix-online.org/pipermail/phenixbb/attachments/20210424/1c33bbed/attachment.htm>