[Speed] Ideas for speed.python.org

[Da_Blitz]

Hi

i have been watching speed since it was announced and was expecting 
some form of rough plan for the server would be discussed but haven't 
seen one proposed to date. since no one has got the ball rolling i 
thought i would put some questions out there to get it all started

What sort of raid level will be used with the server, raid10? raid5?
How is the hard drive going to be divided up?
Any opinions on what OS should be installed?
What services need to be running
What services need to be remotely accessible
is there any other infrastructure we need to integrate to? eg ldap for logins (unlikely but worth asking)
what are we doing about turbo boost? disabling in the BIOS or just leaving it in there
Who will need regular accesses to the box?
what is the box for besides benchmarking, will we allow users to log on and run their own compile jobs?

to answer some of my own questions:
Personally i think raid5 should be sufficient i don't see IO being a 
significant bottle neck and having 900GB vs 600GB may be an advantage 
down the road. raid 6 was ruled out as there are only 4 drives in the 
machine. there should be 2 unused hard drive slots we can use in the 
future if we need to. IIRC there is online raid5 -> raid6 migration 
code in the MD device subsystem but i can confirm that if anyone needs 
me to

as for hard drives, i am a fan of LVM as long as its don't right. VG 
name being <system name>-<Year><Month><day><count> where count is the 
install number for the day. if you don't do it this way you WILL have 
issues when the machine crashes and you need to plug those drives into 
another machine with the same hostname. as for partitioning, 10-15GB 
for / is normally sufficient. a couple of GB's for log files 
(/var/log) to avoid a misconfigured logrotate cron job from eating all 
the disk space (seen this more times than i want to count in 
production). 5GB for /tmp and the remaining space apart from 20GB 
allocated to /home

the reason for the 20GB of "slosh" space is so that if something does 
fill up and you cant afford to delete it you can resize the partition 
and then extend the filesystem while the machine is still running 
(only works on some filesystems and as such i would recommend only 
using a filesystem with only resize, online shrinking is not a high 
priority as one may think).

another tactic i have seen is a separate partition to install 
daily/weekly backups (eg /backups) that are rsyncd to a remote server. 
cheap and effective.

as for OS choice i assume that means ubuntu for 90% of the people on 
this mailing list, this is basically a matter of personal preference 
but if anyone has specific constructive comments then i recommend 
mentioning it

that covers most of the presetup work that cant be changed easily at 
another date

is this just a benchmarking server, or a server people can also log 
into and use as a generic build machine? i always assumed that one 
socket would be used to benchmarking and the other socket would be 
given over to normal users who needed a machine with a bit more speed 
as it is some nice hardware and it would be a shame to see it go to 
waste. this leads into the next question of who needs access to the 
machine and what level of accesses do they need.

remotely accessible serveries, i know apache was mentioned and someone 
wanted something a tiny bit more light weight. as long as you tune it 
apache is fine, we used to run 2000+ domains on machines with a 
quarter as much ram and only 2 cpus and still had them humming along 
with no issue. i would doubt you would hit the kind of traffic to that 
machine that would justify not using the default config and would put 
this as premature optimization. if you still want to look at 
alternatives, i have had experience with nginx. on my machine it only 
uses about 2MB but i would have to recommend lighttpd due to its cgi 
support and its fcgi process support (no external monitoring app 
required for fcgi processes unlike nginx)

i know someone mentioned doing some virtualisation which i assume by 
that they meant xen or virtual box. the cgroups support is one half of 
the light weight virtualisation stack for linux (the resource counting 
part) with LXC being the other half. but full virtualisation is 
something that should also be discussed.

i will stop here as my original list of "points to be discussed" came 
to about 50 items and even then i would have considered that a short 
list for server deployment, hopefully someone has some more input or 
if people like these ideas i can start putting this together as a more 
concrete list of things that need to get done

Man with a plan
Da_Blitz