[CentralOH] Django + Celery + RabbitMQ

[Nick Albright]

Hello Brandon!

I've never really worked with Celery or RabbitMQ (though, I want to!) so I
can't really address those issues, but I have done a bit of ajax and django,
so I can try to help out. (Uncertain based on your Qs if I can help, but I
prolly can't hurt! :)

1) Sorry, sounds like a Celery Q, I can't help there.
2) I might be able to help here! = )

So I assume you are doing some browser side ajax polling (using the task_id
returned by Celery) to a django view which in turn will query to see if
there are results to return.  Once you get the list from celery in the
django view.  I'd format the list to HTML (seen below) and return that to
the browser, which can then insert that in the umm.. div/list of the
browser. (I use jQuery for all my ajax and dom manipulation)  So the django
side might look something like:

def check_for_results( request, task_id = None ):
    response = {}

    # Could be passing the task_id in the GET params
    task_id = int( request.GET.get( 'task_id', task_id ) )  # Assumes
task_id is really an int.  If not, don't cast it to one! :)

    results = CheckCeleryForResults( task_id )    # Whatever celery magic
goes here

    if results:
        # Convert the list to HTML for display
        response[ 'list' ] = render_to_string(
'appname/inclues/ist_display.html',
                                                           { 'results_list':
results,
                                                            },
context_instance = RequestContext( request ) )
        response[ 'done' ] = 1    # If you know there are more results
coming later, could make this 0
    else:
        response[ 'done' ] = 0

    return( HttpResponse( simplejson.dumps( response ),
mimetype='application/json' )


Note, I'm a huge fan of letting django format the HTML using a template, as
I can use that same template on initial page generation if I have any list
elements initially when the page is generated.  It's DRY and all that. :)

And in your HTML, you have a call like:


<script language="javascript" type="text/javascript">
function CheckForResults( task_id ) {
    $.getJSON( "{% url check_for_results %}",
                      { task_id: task_id },
                      function( json, textStatus ) {
                          // Hey!  We got results! Let's show them to our
beloved user!
                          if ( json[ 'list' ] ) {
                             $('#results_list').append( json[ 'list'] );
                          }
                          // Looks like we aren't done, let's poll again in
a bit
                          if ( json[ 'done' ] and !json[ 'done' ] ) {
                             setTimeout( function() { CheckForResults(
task_id ); }, 5000 );   // Wait 5 seconds and try again
                          }
                      }
  );
}
</script>


I hope that helps!
 -Nick


On Wed, Jul 20, 2011 at 8:51 AM, coding.solo <coding.solo at gmail.com> wrote:

> Hi all, I hope you guys can help me on this:
>
>   I have a developing a Django app that processes many <reallyLongTask>.
> My current implementation to get everything up and running was to block for
> <reallyLongTask> and then deliver the results in a view.  That works great,
> but was really long.  Now I want to take it 2.0 with the auspicious notion
> of asynchronous back-end processing whilst polling with AJAX on the
> front-end to populate the results as they are ready.  Currently, a
> <reallyLongTask> returns a list of lists.  Instead of returning a list of
> lists, I want to factor out the enveloping list and just want to return
> lists.
>
>   Now begins my questions.  I have Celery successfully working with
> RabbitMQ.  In my Django view, I execute my <reallyLongTask>.delay() and
> catch its AsyncResult.  I get the AsyncResult.task_id over to the front-end
> browser so that it can poll this task.
>
>    1)  In my AJAX Django view, how am I supposed to query the task with
> only this task_id?
>    2)  Since my original view no longer returns a list of lists, what do
> you guys suggest I do to pass my *lists* back to the browser front-end
> when each individual one is ready?  I would suspect RabbitMQ could handle
> such an exchange somehow, possibly even Celery.  I am thinking I want to
> populate a list for the AJAX view to draw from when the front-end polls that
> view.
>
>
>
>   Thanks all!  Can't wait for my first PyOhio!
>
>
>     Brandon Lorenz
>
> _______________________________________________
> CentralOH mailing list
> CentralOH at python.org
> http://mail.python.org/mailman/listinfo/centraloh
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/mailman/private/centraloh/attachments/20110720/52015aad/attachment.html>