[python-uk] Suggestions / best practices for deployment
MikedePlume
mike at mikedeplume.com
Mon May 20 16:40:28 CEST 2013
On Mon, 2013-05-20 at 14:51 +0100, Harry Percival wrote:
> Interesting points both. I think it's insightful to think about the
> line beween provisioning and deployment... The idea that deployment
> shouldn't require root permissions is a good stake in the ground.
>
>
> In the book I've decided I'm going to take people through spinning up
> a server with Apache, and configuring two virtualhosts with mod_wsgi,
> one for staging and one for production. We negotiate the gotchas,
> including setting up static files, using absolute paths and getting
> the permissions right for the (sqlite) database, etc.
>
>
> Then, deployment is a matter of updating the source code, updating any
> static files, and bouncing apache.
>
>
> Provisioning:
> - spinning up a server
> - installing apache
> - configuring apache
>
> Deployment:
> -updating source code
> -updating static files
> -migrate database
> -restart apache
> For those, the root/non-root perms thing holds pretty well, apart from
> the "restart apache" bit... And, up to a certain point, what I talk
> about under "provisioning" is stuff that might vary from platform to
> platform, but stuff under "deployment" should be more universal...
Interesting that the subject veers towards system design. I use FCGI
myself and the restart can be entirely non-root. Provisioning then
includes adding the FCGI reference into the server config.
> For automating, I think they can be two separate scripts... Although
> we might keep "spinning up a server" manual...
>
> What about testing? Obviously both scripts will be tested indirectly
> via my functional tests (which I'm planning to run against staging,
> and they will show up any, eg, CSS problems)... But I've been pretty
> hot in the book on unit testing everything too. At PA, our deploy
> scripts aren't unit tested, and as a result they're a bit of a mess.
>
Yup. That's where I'm coming from. But maybe there is a reasonable
cut-off? If a deployment script does not have too many paths (which
sounds reasonable) then one test (the just-do-it one) may be sufficient?
> But, unit testing fabric scripts? Won't it end up being a series of
> mocks? I find those kinds of tests a bit pointless... then again,
> maybe if we'd had mocky unit tests for all our deploy scripts since
> the beginning, perhaps they'd be better structured now... Thoughts?
It _is_ possible to test deployment scripts (against a VM perhaps) to
check that directories are created and so on, but I've tried to make
sure that pretty much all of the files (config or other set-up, and
static files) are part of the deployment itself and will be copies of
whatever happened in the testing environment (perhaps, maybe, up to a
point ... ) so testing might just boil down to "Oh looky here - there's
a web site". But that might be a bit simplistic.
Testing _will_ be required if the deployment updates a database - or is
that part of migration?
>
> Also, question for django people: static files, as collected by
> "manage.py collectstatic": in the repo, or not?
>
>
>
>
> On 20 May 2013 13:03, Daniel Pope <lord.mauve at gmail.com> wrote:
> It doesn't need to be SSH-based, there are orchestration
> systems that are RPC or pubsub based (eg. Jenkins,
> MCollective).
>
> As per the line between provisioning and deployment I've
> argued in the past that provisioning should be for state that
> spans many minor releases of an application. I also believe
> it's preferable to require that deployment cannot use root
> permissions, as this is less likely to impact independent
> services hosted on the same machine. It also allows better
> auditability - it prevents deployments making changes that are
> rightly the domain of the provisioning system.
>
> On 20 May 2013 11:17, "MikedePlume" <mike at mikedeplume.com>
> wrote:
> On Sat, 2013-05-18 at 17:41 +0100, Andy Robinson
> wrote:
> > Daniel, who are you disagreeing with? Everyone
> here agrees on
> > automation, I think.
> >
> > - Andy
>
> As a small user (one or two servers, one or two
> packages to deploy):
>
> I started by doing a lot of typing of commands, got
> bored with that,
> moved on to command line scripts and I now use a bunch
> of (python)
> scripts, some fabric based, to manage the server farm
> api and the ssh
> stuff. I think the point is that the underlying
> process is always going
> to involve ssh and some server farm api and it is the
> automation of all
> of that where the cornucopia of choice (?) is
> displayed.
>
> From a teaching perspective (that's what the book is
> about, I guess) I'd
> recognise the basics of ssh and so on and then build
> up from there. The
> basic principles behind testing a fabric deploy should
> be extendable to
> other environments, I'd have thought?
>
> I must say, I'd love to see something about testing
> bash scripts, and,
> perhaps, where to draw the line (if at all) between
> server provisioning
> and ci.
>
> Cheers
>
> Mike S.
> >
> >
> >
> > On 18 May 2013 12:24, Daniel Pope
> <lord.mauve at gmail.com> wrote:
> > > I thoroughly disagree with those who are saying
> that for small
> > > installations, it's less time-consuming to do
> things manually. A
> > > deployment/provisioning system gives you
> reproducibility - an executable
> > > record of how to re-create a server configuration
> or re-run a deployment
> > > without missing anything. The number of times I've
> spent 20 minutes
> > > wondering what I've missed all add up - that is
> why it automation breaks
> > > even so rapidly.
> > >
> > > Of course, you (or your employers) might have
> other reasons to want
> > > automation:
> > >
> > > - so that your deployment/provisioning is subject
> to automated testing,
> > > version control, code review, etc
> > > - so that you can build replicas of production
> servers for development,
> > > testing or disaster recovery
> > > - so that you can smash and rebuild a compromised
> or faulty machine without
> > > wasting time
> > > - so that you can deploy a dozen times a day and
> get features or fixes into
> > > the hands of your users
> > >
> > >
> > > On 17 May 2013 15:39, M.-A. Lemburg
> <mal at egenix.com> wrote:
> > >>
> > >> On 16.05.2013 17:46, Andy Robinson wrote:
> > >> > Speaking as a relatively obsolete dinosaur, I
> would suggest that if
> > >> > you are going to discuss specific deployment
> practices, you start with
> > >> > the most fundamental ones: SSH, the unix shell
> and so on.
> > >> >
> > >> > We have had issues over the years with people
> coming in and
> > >> > introducing sexy new deployment tools, but
> ultimately they all just
> > >> > run unix commands. Anyone managing a web
> application in the
> > >> > non-microsoft world is ultimately depending on
> this. Some key skills
> > >> > (assuming a Linux/Mac/Unix-ish environment):
> > >> > - know about SSH keys and logging into remote
> machines
> > >> > - know the basics of at least one command line
> editor (e.g. vi)
> > >> > - basic shell knowledge: environment
> variables, testing for existence
> > >> > of files and directories etc
> > >> > - how to interact with your database from the
> command line, if you use
> > >> > one (including dump and restore)
> > >> > - how your web server works: starting,
> stopping, configuration files,
> > >> > where log files live and how it talks to Python
> > >> >
> > >> > Fabric may be useful if you want to control
> half a dozen machines from
> > >> > your desktop, and it might add a lot of value
> if you want to control a
> > >> > hundred of them. But to update one server, you
> deploy by logging into
> > >> > it and then running commands or short scripts.
> > >> >
> > >> > For example, we have a 'demo site' we rebuild
> pretty often that uses
> > >> > Django, MYSQL, Celery and a few other things.
> It runs on plain
> > >> > vanilla Ubuntu machines we build ourselves.
> The sequence is...
> > >> >
> > >> > 1. Log in via SSH
> > >> > 2. CD to correct directory
> > >> > 3. activate virtual environment
> > >> > 4. stop any celery worker processes
> > >> > 5. stop web server processes (* in our setup,
> we leave Apache running)
> > >> > 6. pull latest code from mercurial - both the
> app, and 3-4 libraries
> > >> > it depends on
> > >> > 7. run a management command to rebuild the
> database
> > >> > 8. run a smallish in-place test suite
> > >> > 9. restart celery workers
> > >> > 10.restart web server
> > >> > 11. log out
> > >> >
> > >> > All of this after the login and CD can be
> handled by a shell script on
> > >> > the path of the server, so you can just run a
> command called something
> > >> > like
> > >> > ./update_server
> > >> >
> > >> > More realistically, we tend to end up with a
> management shell script
> > >> > called 'server' with a bunch of
> commands/arguments like 'stop / start
> > >> > / restart / update-code-in-staging /
> copy-live-data-to-staging /
> > >> > run-health-checks / swap-live-and-staging' and
> so on. SSH can execute
> > >> > remote commands like this just fine with the
> right arguments, if
> > >> > actually logging in is too tedious.
> > >> >
> > >> > Production sites are complex and all
> different. You might want to do
> > >> > instantaneous swaps from live to staging (and
> be able to back out fast
> > >> > if stuff goes wrong); to switch DNS so the
> world is looking at another
> > >> > server while you update one; you might have
> large databases to copy or
> > >> > migrate that need significant time; it may or
> may not be acceptable to
> > >> > lose sessions and have downtime; and so on.
> > >> >
> > >> >
> > >> > It takes less time to learn the fundamentals
> than you will spend
> > >> > debugging why your fancy new deployment tool
> stopped working after
> > >> > some Python dependency upgrade somewhere. And
> it is less likely that
> > >> > your new hires will disagree if you stick with
> the lowest common
> > >> > denominator.
> > >>
> > >> Fully agreed.
> > >>
> > >> The new devops tools are nice when it comes to
> managing farms
> > >> of VMs, where each VM is setup in more or less
> the same way,
> > >> and you want to reduce repetitive tasks, but for
> a typical
> > >> setup with just a few VMs/servers it'll take you
> longer to
> > >> write and (most importantly) test your devops
> scripts than
> > >> it would to write a few scripts that you can run
> via SSH on
> > >> the servers.
> > >>
> > >> No matter how smart you make your devops scripts,
> Murphy's law
> > >> is inevitably going to strike and humans are so
> much better at
> > >> parsing weird intermittent error messages than
> machines ...
> > >> still, after all these years :-)
> > >>
> > >> --
> > >> Marc-Andre Lemburg
> > >> eGenix.com
> > >>
> > >> Professional Python Services directly from the
> Source (#1, May 17 2013)
> > >> >>> Python Projects, Consulting and Support ...
> http://www.egenix.com/
> > >> >>> mxODBC.Zope/Plone.Database.Adapter ...
> http://zope.egenix.com/
> > >> >>> mxODBC, mxDateTime, mxTextTools ...
> http://python.egenix.com/
> > >>
> ________________________________________________________________________
> > >> 2013-05-07: Released mxODBC Zope DA 2.1.2 ...
> http://egenix.com/go46
> > >> 2013-05-06: Released mxODBC 3.2.3 ...
> http://egenix.com/go45
> > >>
> > >> ::::: Try our mxODBC.Connect Python Database
> Interface for free ! ::::::
> > >>
> > >> eGenix.com Software, Skills and Services GmbH
> Pastor-Loeh-Str.48
> > >> D-40764 Langenfeld, Germany. CEO Dipl.-Math.
> Marc-Andre Lemburg
> > >> Registered at Amtsgericht Duesseldorf:
> HRB 46611
> > >>
> http://www.egenix.com/company/contact/
> > >> _______________________________________________
> > >> python-uk mailing list
> > >> python-uk at python.org
> > >> http://mail.python.org/mailman/listinfo/python-uk
> > >
> > >
> > >
> > > _______________________________________________
> > > python-uk mailing list
> > > python-uk at python.org
> > > http://mail.python.org/mailman/listinfo/python-uk
> > >
> >
> >
> >
>
>
> _______________________________________________
> python-uk mailing list
> python-uk at python.org
> http://mail.python.org/mailman/listinfo/python-uk
>
> _______________________________________________
> python-uk mailing list
> python-uk at python.org
> http://mail.python.org/mailman/listinfo/python-uk
>
>
>
>
> --
> ------------------------------
> Harry J.W. Percival
> ------------------------------
> Twitter: @hjwp
> Mobile: +44 (0) 78877 02511
> Skype: harry.percival
> _______________________________________________
> python-uk mailing list
> python-uk at python.org
> http://mail.python.org/mailman/listinfo/python-uk
More information about the python-uk
mailing list