<div dir="ltr"><div>The problem with using CircleCI is the memory restriction. I think it is 4GB. Please correct me if wrong. <br></div><div>Just some of the tractography files we load is of this size. Even for running our doc examples we need more than 8 GB.</div><div><br></div><div>We need a machine that has 12-16GB of RAM. Or we need to refactor some of the methods that require larger amounts of memory to use </div><div>memmaps or similar. But still 4GB will be a strong restriction. We are often dealing with big datasets.</div><div><br></div><div>Ariel, as I said in my previous e-mail at this stage we need a machine to process for 24 hours continuously and do that 1 to 2 times a week. But of course if there are important</div><div>issues we will need to do that sooner. Also it would be good to have some multiprocessing involved. The machine used should have multiple cores if possible.</div><div>So, what do you think will be the logistics for that?</div><div><br></div><div>Also we need a system which will allow to report and send feedback after the execution of the workflows. In a way Matthew has already setup some of these things</div><div>in the Berkeley bots. I want to hear his opinion too on this.</div><div><br></div><div>I also like the idea of having a docker that can be moved anywhere. </div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 3, 2016 at 2:26 PM, vanessa sochat <span dir="ltr"><<a href="mailto:vsochat@stanford.edu" target="_blank">vsochat@stanford.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">+1</div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 3, 2016 at 11:23 AM, Chris Filo Gorgolewski <span dir="ltr"><<a href="mailto:krzysztof.gorgolewski@gmail.com" target="_blank">krzysztof.gorgolewski@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Ah yes that makes sense. You can however split your tests (for example one workflow per line) and easily run a battery of tests that takes more than 2h.</p><div><div>
<div class="gmail_quote">On Mar 3, 2016 11:16 AM, "vanessa sochat" <<a href="mailto:vsochat@stanford.edu" target="_blank">vsochat@stanford.edu</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The upper limit for each test is 2 hours (120 minutes).</div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 3, 2016 at 11:02 AM, Chris Filo Gorgolewski <span dir="ltr"><<a href="mailto:krzysztof.gorgolewski@gmail.com" target="_blank">krzysztof.gorgolewski@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Yes, longer jobs (AFAIK there is no limit as long as the job is printing something every 10 minutes) and more RAM (4GB) are the main advantages.</p>
<div class="gmail_quote">On Mar 3, 2016 10:46 AM, "Yaroslav Halchenko" <<a href="mailto:lists@onerussian.com" target="_blank">lists@onerussian.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
On Thu, 03 Mar 2016, Chris Filo Gorgolewski wrote:<br>
<br>
> Have a look at waht we are doing for Nipype on CircleCI (on the free open<br>
> source tier):<br>
<br>
> <a href="https://github.com/nipy/nipype/blob/master/circle.yml" rel="noreferrer" target="_blank">https://github.com/nipy/nipype/blob/master/circle.yml</a><br>
> <a href="https://circleci.com/gh/nipy/nipype" rel="noreferrer" target="_blank">https://circleci.com/gh/nipy/nipype</a><br>
<br>
> All of the workflows we run for tests take over 3h to finish. Similar set<br>
> up is implemented in nilearn project.<br>
<br>
sorry for ignorance -- haven't used circleci yet. So is that the<br>
crucial advantage of circle ci (over travis) that it allows for<br>
longer jobs and more caching space or what?<span><font color="#888888"><br>
<br>
--<br>
Yaroslav O. Halchenko<br>
Center for Open Neuroscience <a href="http://centerforopenneuroscience.org" rel="noreferrer" target="_blank">http://centerforopenneuroscience.org</a><br>
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755<br>
Phone: <a href="tel:%2B1%20%28603%29%20646-9834" value="+16036469834" target="_blank">+1 (603) 646-9834</a> Fax: <a href="tel:%2B1%20%28603%29%20646-1419" value="+16036461419" target="_blank">+1 (603) 646-1419</a><br>
WWW: <a href="http://www.linkedin.com/in/yarik" rel="noreferrer" target="_blank">http://www.linkedin.com/in/yarik</a><br>
_______________________________________________<br>
Neuroimaging mailing list<br>
<a href="mailto:Neuroimaging@python.org" target="_blank">Neuroimaging@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/neuroimaging" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/neuroimaging</a><br>
</font></span></blockquote></div>
<br>_______________________________________________<br>
Neuroimaging mailing list<br>
<a href="mailto:Neuroimaging@python.org" target="_blank">Neuroimaging@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/neuroimaging" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/neuroimaging</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div>Vanessa Villamia Sochat<br>Stanford University<br><div><div><div><a href="tel:%28603%29%20321-0676" value="+16033210676" target="_blank">(603) 321-0676</a></div></div></div></div>
</div>
<br>_______________________________________________<br>
Neuroimaging mailing list<br>
<a href="mailto:Neuroimaging@python.org" target="_blank">Neuroimaging@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/neuroimaging" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/neuroimaging</a><br>
<br></blockquote></div>
</div></div><br>_______________________________________________<br>
Neuroimaging mailing list<br>
<a href="mailto:Neuroimaging@python.org" target="_blank">Neuroimaging@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/neuroimaging" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/neuroimaging</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div>Vanessa Villamia Sochat<br>Stanford University<br><div><div><div>(603) 321-0676</div></div></div></div>
</div>
</div></div><br>_______________________________________________<br>
Neuroimaging mailing list<br>
<a href="mailto:Neuroimaging@python.org">Neuroimaging@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/neuroimaging" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/neuroimaging</a><br>
<br></blockquote></div><br></div>