[Cython] Speedup hudson job with ccache

[Vitja Makarov]

2011/7/7 Stefan Behnel <stefan_ml at behnel.de>:
> Stefan Behnel, 07.07.2011 09:29:
>>
>> Vitja Makarov, 07.07.2011 08:54:
>>>
>>> 2011/7/7 Stefan Behnel
>>>
>>>> Vitja Makarov, 07.07.2011 08:32:
>>>>
>>>>> 2011/7/7 Stefan Behnel
>>>>>
>>>>> Robert Bradshaw, 06.07.2011 21:32:
>>>>>>
>>>>>> We should get ccache on sage.math to start using it on hudson.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>> It's just a single binary when installed. I have a copy in my
>>>>>> ~/ccache/bin.
>>>>>> Changing the Jenkins jobs now.
>>>>>>
>>>>>>
>>>>>> Is that already used?
>>>>>
>>>>
>>>> Actually, no. I noticed that it would a) require changes to all test
>>>> jobs,
>>>> and b) prevent us from getting clean test runs. I prefer a safe and
>>>> clean
>>>> run over a fast one.
>>>>
>>>>
>>> a) you can set CYTHON_RUNTESTS_CCACHE globaly somewhere inside ~/.profile
>>
>> ;) I actually set it globally in Jenkins. However, that doesn't
>> automatically remove the "rm -fr *" in the job configs - I initially
>> thought that would be a problem, but, apparently, it isn't.
>>
>>
>>>> Also, the test runner clears the target directory on startup. I wonder
>>>> where you got your numbers from...
>>>>
>>> By default ccache stores object files inside ~/.ccache directory
>>
>> Hmm, I guess changing that directory into a job local directory would
>> still
>> require changing each job... Are there any concurrency issues with ccache?
>>
>> Anyway, I can't see a .ccache in my home directory, but if that's how it
>> works, it should at least be properly enabled now.
>
> Clearly makes a difference.
>
> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests-py27-c/buildTimeTrend
>
> The .ccache directory is also there now, size is ~750M. I'll move it to the
> local scratch disc, CCACHE_DIR should do that. Does it make sense to delete
> it once a week, or should that be done more often?
>

Probably it's better to increase cache size, that will be shared
between projects:

$ ccache -s 5G (I don't know what's the right size to go)

>From man ccache:

"""
CACHE SIZE MANAGEMENT
       By  default  ccache has a one gigabyte limit on the cache size
and no maximum number of files. You can set a different limit using
the "ccache -M"
       and "ccache -F" options, which set the size and number of files limits.

       When these limits are reached ccache will reduce the cache to
20% below the numbers you specified in order to avoid doing the cache
clean  opera‐
       tion too often.
"""

Btw it's a good idea to cleanup cache once a week:

$ ccache --cleanup (or --clear?)

-- 
vitja.