[phenixbb] Phenix-Rosetta integration tests fail with Rosetta MPI on RHEL 6.3

Terwilliger, Thomas Charles terwilliger at lanl.gov
Fri Sep 12 09:58:02 PDT 2014


Hi Sina,

The phenix_regression.wizards.test_command_line_rosetta_quick tests all use a single processor, so they do not, in themselves test the use of a cluster.  However each of these tests is set up so that a little directory with the name of the test is created, and the job is run by a script in that directory.  You can edit that script, add your queue commands, and test it that way.

Yes, nproc determines the number of jobs that mr_rosetta will submit at any one time.

I'm not familiar with SLURM so I am not certain of the answer to your question.  I'm hoping that you can specify the commands you need with the queue_commands keywords though.

Yes, the queue technology used in phenix.rosetta_refine and phenix.mr_rosetta are different.  I'm sorry about that.

All the best,
Tom T

On Sep 11, 2014, at 10:43 PM, Sina Masoud-Ansari wrote:

> Hi Tom,
> 
> Thanks for your reply. In that case, that's good news because I have a working non-mpi version of Rosetta 3.5 that I would like to make available to researchers on our SLURM cluster while I work on getting more recent versions of Rosetta working.
> 
> Are any of the cases in 'phenix_regression.wizards.test_command_line_rosetta_quick' suitable for submitting against a cluster so that I can check that the basic workflow is correct?
> 
> My impression from the documentation for mr_rosetta was that I would need to use additional options like:
> 
> nproc=2 \
> group_run_command="sbatch " \
> queue_commands="#SBATCH --time=01:00:00" \
> queue_commands="#SBATCH --mem-per-cpu=2048"
> queue_commands="module load Rosetta/3.5-gcc-4.4.6" \
> queue_commands="module load phenix/1.9-16926" \
> queue_commands="export PHENIX_ROSETTA_PATH=\$MODULE_ROSETTA_ROOT" \
> queue_commands="source phenix_env.sh" \
> queue_commands="srun \\"
> 
> I have a few questions about this I was hoping this community could help with:
> 
> Does nproc determine the number of jobs submitted by Phenix?
> 
> Also, SLURM is a bit different to PBS in that we need to prefix commands with 'srun', for example 'srun echo hello'. Is this handled automatically by Phenix (phenix.mr_rosetta)?
> 
> Finally I've noticed that rosetta_refine and mr_rosetta have slightly different options with regards to queue submission in the documentation.
> 
> http://www.phenix-online.org/documentation/reference/rosetta_refine.html
> http://www.phenix-online.org/documentation/reference/mr_rosetta.html
> 
> For example, rosetta_refine allows the option 'technology = slurm' but this is not listed as an option for mr_rosetta.
> Are they indeed different?
> 
> Thanks again, I appreciate your help
> 
> On 12/09/14 1:48 am, Terwilliger, Thomas Charles wrote:
>> i Sina,
>> 
>> Thanks for the question!   We have not tried Phenix mr_rosetta with Rosetta MPI so I am not too surprised that it did not work.  We have focused our effort on making it work with clusters using qsub and similar commands. Normally an individual Rosetta job does not take a really long time; rather it is the many Rosetta jobs that collectively take a long time.  Consequently using MPI to parallelize individual jobs seems to me to be likely to take about the same clock time as using many individual processors for separate jobs.
>> 
>> I'll check on the Rosetta error that you mention separately.
>> 
>> All the best,
>> Tom T
>>   
>>   ________________________________________
>> From: phenixbb-bounces at phenix-online.org [phenixbb-bounces at phenix-online.org] on behalf of Sina Masoud-Ansari [s.ansari at auckland.ac.nz]
>> Sent: Wednesday, September 10, 2014 10:29 PM
>> To: phenixbb at phenix-online.org
>> Subject: [phenixbb] Phenix-Rosetta integration tests fail with Rosetta MPI      on RHEL 6.3
>> 
>> Hi there,
>> 
>> I'm wondering anyone has managed to successfully run Phenix with Rosetta
>> MPI on Red Hat Enterprise Linux 6.3 or similar?
>> 
>> I currently have a non-MPI build of Rosetta that passes all of the tests
>> in phenix_regression.wizards.test_command_line_rosetta_quick
>> 
>> However when running these tests with the MPI build of Rosetta the
>> following tests fail:
>> 
>> Running test_relax
>> ------------------------------------------------------------
>> Numbers do not match: 6 1
>> Relaxed solutions  6
>> ---
>> Relaxed solutions  1
>> 
>> ------------------------------------------------------------
>> 
>> FAILED
>> See log files test_relax/test_relax.log test_relax/test_relax_current.log
>> =================================================================
>> Running test_relax_double
>> ------------------------------------------------------------
>> Numbers do not match: 6 1
>> Relaxed solutions  6
>> ---
>> Relaxed solutions  1
>> 
>> ------------------------------------------------------------
>> 
>> ------------------------------------------------------------
>> Numbers do not match: 6 1
>> Relaxed solutions  6
>> ---
>> Relaxed solutions  1
>> 
>> ------------------------------------------------------------
>> 
>> FAILED
>> See log files test_relax_double/test_relax_double.log
>> test_relax_double/test_relax_double_current.log
>> =================================================================
>> Running test_relax_no_sol
>> ------------------------------------------------------------
>> Numbers do not match: 6 1
>> Relaxed solutions  6
>> ---
>> Relaxed solutions  1
>> 
>> ------------------------------------------------------------
>> 
>> FAILED
>> See log files test_relax_no_sol/test_relax_no_sol.log
>> test_relax_no_sol/test_relax_no_sol_current.log
>> 
>> I have attempted to attach an archive of the logs.
>> 
>> I'm working with Rosetta 3.5, more recent versions of Rosetta fail other
>> tests in phenix_regression.wizards.test_command_line_rosetta_quick ,
>> please see https://www.rosettacommons.org/node/3721 for details.
>> 
>> Rosetta 3.5 was compiled with GCC 4.4.6 using OpenMPI 1.6.4
>> 
>> Any help or advice is appreciated.
>> 
>> --
>> Sina Masoud-Ansari
>> Centre for eResearch
>> 409.G21 24 Symonds Street
>> University of Auckland
>> 
>> e: s.ansari at auckland.ac.nz
>> p: +64 9 3737599 ext 89369
>> w: http://www.eresearch.auckland.ac.nz
>> 
> 
> -- 
> Sina Masoud-Ansari
> Centre for eResearch
> 409.G21 24 Symonds Street
> University of Auckland
> 
> e: s.ansari at auckland.ac.nz
> p: +64 9 3737599 ext 89369
> w: http://www.eresearch.auckland.ac.nz
> 



More information about the phenixbb mailing list