[IPython-dev] SciPy Sprint summary

[Justin Riley]

Hi Matthieu,

I forgot to cc the list on my last email. Details below. Thanks for
testing this.

~Justin

On 07/22/2010 10:20 AM, Matthieu Brucher wrote:
> 2010/7/22 Justin Riley <justin.t.riley at gmail.com>:
>> On Wed, Jul 21, 2010 at 7:08 AM, Matthieu Brucher
>> <matthieu.brucher at gmail.com> wrote:
>>> I've tried just a few minutes ago, but I got this:
>>>
>>> /JOB_SPOOL_DIR/1279710223.17444: line 8: /tmp/tmphM4RKl: Permission denied
>>>
>>> It seems that you may have to add some authorizations before excuting the file.
>>
>> You're right about needing to set permissions for LSF and I merged
>> your fork code. I found the following detail about using job scripts
>> with LSF (retrieved from
>> http://www.cisl.ucar.edu/docs/bluefire/lsf.html):
>>
>> --------
>> LSF command bsub allows submission of an executable, e.g.
>>
>>  $ bsub -i infile -o outfile -e errfile a.out
>>
>> LSF can also be used to submit a job script. However, then the LSF
>> command bsub requires redirection of its command file, specifically
>>
>>  $ bsub < myscript
>> --------
>>
>> I can't test this but I suspect LSF is ignoring our job script
>> variables given that we're submitting the job as an executable. If
>> this is the case then we'll either need to use redirection of the
>> script or just pass the -J option to the bsub command we're using now.
>>
>> With that said, would you mind running the following test?
>>
>> 1. download this job script: http://gist.github.com/485618
>> 2. run ipcontroller manually in a separate shell
>> 3. chmod +x the script and then bsub it the "executable" way (bsub -i
>> infile -o outfile -e errfile testscript.sh)
>> 4. check the output/error files for errors. you can also use the job's
>> id to tail it's output (bpeek -J jobid -f) and look at its history
>> info (bhist jobid)
>> 5. run ipython, obtain a mec and see how many ids you get (gist should
>> request 4)
>> 6. re-run 1-5 and use the "redirection" way (bsub < testscript.sh)
>>
>> If the redirection approach works and it's possible to do this with
>> twisted we could avoid the chmod'ing (although it doesn't hurt) and it
>> would "fit the mold" so far. Otherwise we'll need to chmod +x AND pass
>> in the -J option to the bsub command. This isn't too bad but would
>> require remolding.
>>
>> Thanks for testing,
>>
>> ~Justin
>>
> 
> Hi Justin,
> 
> I will try this, but I think you're right. I've just checked my own
> launch framework, and I submit the script by doing bsub <
> complex_temporaray_file. It should be done this way because we can set
> a lot of things in the script as arguments for LSF that we can't if we
> use bsub -i ...
> 
> I'll keep you posted.
> 
> Matthieu