- cctbxbb - phenix-online.org

Changing boost versions
by Nicholas Devenish 01 Aug '17

01 Aug '17

Hi All, Graeme's thread reminded me to ask about something I've been struggling with; is there a built-in/easy way to specify a different installation of boost to use? The build seems to construct lib boost_python manually, for some reason, so I don't know how easy it is to disentangle. Thanks, Nick

3 2

Pillow has landed
by markus.gerstel＠diamond.ac.uk 25 Jul '17

25 Jul '17

Hello everyone, Bootstrap now installs Pillow as the new replacement library of the Python Imaging Library (PIL), which has been deprecated/abandonware for a couple of years now. You can update your local installation by running libtbx.python -m pip install Pillow I ran tests locally, and they appeared to run fine. Although this is pretty much a drop-in replacement there may be some import statements which need updating. If you do come across problems you may find this to be of use: https://pillow.readthedocs.io/en/4.2.x/porting.html -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

smtbx.refine not working correctly?
by Graeme.Winter＠diamond.ac.uk 20 Jul '17

20 Jul '17

Not sure who to ask... Tried smtbx.refine from latest phenix & dials builds, both give [gw56@cs03r-sc-serv-16 Refine]$ smtbx.refine lcy.ins lcy.hkl lcy_smtbx.res Traceback (most recent call last): File "/dls_sw/apps/dials/dials-v1-6-4/build/../modules/cctbx_project/smtbx/command_line/refine.py", line 152, in <module> run(args, options) File "/dls_sw/apps/dials/dials-v1-6-4/build/../modules/cctbx_project/smtbx/command_line/refine.py", line 67, in run strictly_shelxl=False) File "/dls_sw/apps/dials/dials-v1-6-4/modules/cctbx_project/smtbx/refinement/__init__.py", line 10, in from_shelx return _(cls, *args, **kwds) File "/dls_sw/apps/dials/dials-v1-6-4/modules/cctbx_project/iotbx/shelx/__init__.py", line 59, in smtbx_refinement_model_from builder.weighting_scheme, AttributeError: 'smtbx_builder' object has no attribute 'weighting_scheme' [gw56@cs03r-sc-serv-16 Refine]$ module load phenix/1.12 Setting up phenix environment and PATH... [gw56@cs03r-sc-serv-16 Refine]$ smtbx.refine lcy.ins lcy.hkl lcy_smtbx.res Traceback (most recent call last): File "/dls_sw/apps/phenix/1.12/phenix-1.12-2829/build/../modules/cctbx_project/smtbx/command_line/refine.py", line 152, in <module> run(args, options) File "/dls_sw/apps/phenix/1.12/phenix-1.12-2829/build/../modules/cctbx_project/smtbx/command_line/refine.py", line 67, in run strictly_shelxl=False) File "/dls_sw/apps/phenix/1.12/phenix-1.12-2829/modules/cctbx_project/smtbx/refinement/__init__.py", line 10, in from_shelx return _(cls, *args, **kwds) File "/dls_sw/apps/phenix/1.12/phenix-1.12-2829/modules/cctbx_project/iotbx/shelx/__init__.py", line 59, in smtbx_refinement_model_from builder.weighting_scheme, AttributeError: 'smtbx_builder' object has no attribute 'weighting_scheme' who's best to ask about this? Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 3

New error appeared recently, do not know who it belongs to...
by Graeme.Winter＠diamond.ac.uk 17 Jul '17

17 Jul '17

Exhibit A: libtbx.python "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/tst.py" rstbx.simage.explore_completeness d_min=10 rstbx.simage.explore_completeness d_min=10 intensity_symmetry=P4 use_symmetry=True multiprocessing=True rstbx.simage.solver d_min=5 rstbx.simage.solver d_min=5 lattice_symmetry=R32:R intensity_symmetry=R3:R rstbx.simage.solver d_min=5 lattice_symmetry=R32:R intensity_symmetry=P1 rstbx.simage.solver d_min=5 lattice_symmetry=P422 intensity_symmetry=P4 index_and_integrate=True multiprocessing=True Traceback (most recent call last): File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/tst.py", line 273, in <module> run(args=sys.argv[1:]) File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/tst.py", line 269, in run exercise_solver() File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/tst.py", line 255, in exercise_solver "index_and_integrate=True", "multiprocessing=True"]) File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/tst.py", line 213, in run command=cmd, stdout_splitlines=False).raise_if_errors().stdout_buffer File "/Users/graeme/svn/cctbx/modules/cctbx_project/libtbx/easy_run.py", line 39, in raise_if_errors raise Error(msg) RuntimeError: child process stderr output: command: 'rstbx.simage.solver d_min=5 lattice_symmetry=P422 intensity_symmetry=P4 index_and_integrate=True multiprocessing=True' Traceback (most recent call last): File "/Users/graeme/svn/cctbx/build/../modules/cctbx_project/rstbx/command_line/simage.solver.py", line 5, in <module> run(args=sys.argv[1:]) File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/solver.py", line 1161, in run return run_fresh(args) File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/solver.py", line 1145, in run_fresh process(work_params, i_calc) File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/solver.py", line 1088, in process process_core(work_params, i_calc.p1_anom, reindexing_assistant, image_mdls) File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/solver.py", line 985, in process_core work_params, reindexing_assistant, image_mdls, usables) File "/Users/graeme/svn/cctbx/modules/cctbx_project/rstbx/simage/solver.py", line 883, in build_image_cluster raise RuntimeError("Insufficient connectivity between images.") RuntimeError: Insufficient connectivity between images. usr+sys time: 0.78 seconds, ticks: 777467, micro-seconds/tick: 1.003 wall clock time: 6.99 seconds OS X & RHEL6 Any volunteers? Cheers Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 5

Re: [cctbxbb] duplicate spam
by Aaron Brewster 10 Jul '17

10 Jul '17

More commit spam incoming. This is from merging a bunch of commits from an LCLS beamtime into the main branch. Sorry for the flood. -Aaron On Tue, Jun 27, 2017 at 12:30 AM, Markus Gerstel < markus.gerstel(a)diamond.ac.uk> wrote: > Due to rewriting git history there are now a number of duplicate mails > coming through here. > > Apologies. > > > > -Markus > > > > -- > > This e-mail and any attachments may contain confidential, copyright and or > privileged material, and are for the use of the intended addressee only. If > you are not the intended addressee or an authorised recipient of the > addressee please notify us of receipt by returning the e-mail and do not > use, copy, retain, distribute or disclose the information in or attached to > the e-mail. > Any opinions expressed within this e-mail are those of the individual and > not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for any > damage which you may sustain as a result of software viruses which may be > transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in England > and Wales with its registered office at Diamond House, Harwell Science and > Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom > >

2 2

Building cctbx & friends with intel compilers
by Graeme.Winter＠diamond.ac.uk 29 Jun '17

29 Jun '17

Greetings all, Do we have a recipe for compiling cctbx etc with icc? To pick a random example, this version: icc (ICC) 17.0.4 20170411 Copyright (C) 1985-2017 Intel Corporation. All rights reserved. Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

incorrect url for downloading fasta files
by R.D. Oeffner 28 Jun '17

28 Jun '17

hi, It would appear that the URL in cctbx_projectiotbxpdbfetch.py for fetching fasta files has been changed by RCSB. So fetch.py therefore downloads garbage when trying to download a fasta file. The correct URL is not documented by RCSB but can be inferred from the RCSB web site when downloading a fasta file. I've updated the URL in fetch.py accordingly. Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus Wellcome Trust/MRC Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234

2 1

please read before your next commit
by markus.gerstel＠diamond.ac.uk 27 Jun '17

27 Jun '17

Good morning, recently a couple of large files were erroneously committed to the cctbx repository and then deleted thereafter. Git always keeps a full history, so to avoid cruft aggregating over time we decided to cut those files out of the repository history. Unfortunately this means everyone will have to do a couple of steps: 1. Please ensure you have run libtbx.git_avoid_merges on all of your development machines 2. Run git pull --rebase the next time you update your repository instead of git pull - again, do this on all development machines. 3. If you keep any local branches you may need to rebase them onto master (git fetch; git checkout $branchname; git rebase origin/master) If you don't follow these instructions you may end up recommitting those large files, and we have to start again. Thanks -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

1 0

Housekeeping / roadmap for future
by Graeme.Winter＠diamond.ac.uk 21 Jun '17

21 Jun '17

Hello all, Appreciate a phenix release coming shortly & not suggesting anything any time soon However we were discussing earlier what the road map is for cctbx - making it ready for the future - and I was wondering what is already on the table? Is there a road map somewhere? I recall Billy mentioning Python3 recently, as an example... Thanks Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

4 5

Re: [cctbxbb] bootstrap.py on Windows
by R. D. Oeffner 20 Jun '17

20 Jun '17

Instructions for building on windows are here: https://github.com/cctbx/cctbx_project/wiki Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus Wellcome Trust/MRC Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -----Original Message----- From: R. D. Oeffner Sent: Tuesday, June 20, 2017 10:37 PM To: horst.puschmann(a)gmail.com ; cctbxbb(a)phenix-online.org Cc: oleg(a)olexsys.org Subject: Re: [cctbxbb] bootstrap.py on Windows Dear Horst, There's a writeup of instructions on how to do CCTBX builds from bootstrap.py on Windows. No authentication is needed for building CCTBX on any platform. Please follow these instructions and get in touch if you have further problems. CCTBX and other projects depending on CCTBX is tested through regression test nightly on LBL machines on all platforms supported. See http://cci-vm-6.lbl.gov:8010/one_line_per_build for log files of these tests. Regards, Robert -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus Wellcome Trust/MRC Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234 -----Original Message----- From: markus.gerstel(a)diamond.ac.uk Sent: Tuesday, June 20, 2017 9:25 PM To: horst.puschmann(a)gmail.com ; cctbxbb(a)phenix-online.org Cc: oleg(a)olexsys.org Subject: Re: [cctbxbb] bootstrap.py on Windows Hello Horst, I think the automated tests aren't really covering the bootstrap process. While we extensively test bootstrap at our place, we do not run any tests on Windows. I don't know if LBL runs Windows tests, but if they do they probably run it within buildbot, which means it is not run the same way you run it. Best wishes -Markus ________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of Horst Puschmann [horst.puschmann(a)gmail.com] Sent: Tuesday, April 04, 2017 09:59 To: cctbxbb(a)phenix-online.org Cc: Oleg Dolomanov (oleg(a)olexsys.org) Subject: [cctbxbb] bootstrap.py on Windows Hello I am trying to install the cctbx on a Windows 7 64 bit machine using the following bootstrap.py file: https://raw.githubusercontent.com/cctbx/cctbx_project/master/libtbx/auto_bu… The script will fail, unless I comment out lines 1101 and 1102: #if self.isPlatformWindows(): #tarurl, arxname, dirpath = MODULES.get_module(module)().get_tarauthenticated(auth=self.get_auth()) If it tries to execute the commented lines, the error will be "KeyError: 'cciuser' in line 590" After commenting this out, it will fail in line 271, with "AttributeError: 'module' object has no attribute '_create_unverified_context'" I can get round that with disabling line 247: if sys.platform == "win32": (i.e. change this to if sys.platform == "xxx":) After that, things start downloading. All appears well until the same thing happens again in another bootstrap.py file in \modules\cctbx_project\libtbx\auto_build\bootstrap.py If I disable *that*, it goes further, but fails finally with ===== Running in build: run configure.py Traceback (most recent call last): File "bootstrap.py", line 2113, in <module> run() File "bootstrap.py", line 2108, in run enable_shared=options.enable_shared, File "bootstrap.py", line 1036, in run i.run() File "bootstrap.py", line 189, in run raise RuntimeError("Could not run %s: File not found" % executable) RuntimeError: Could not run base\bin\python: File not found I guess the automatic tests haven't picked this up, because no authentication is needed, maybe? Greetings Horst -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org http://phenix-online.org/mailman/listinfo/cctbxbb --- This email has been checked for viruses by AVG. http://www.avg.com

1 0