[CentralOH] Django + Celery + RabbitMQ

[coding.solo]

Thanks, Nick!  Being new to Django, I have built my views to
render_to_response().  I like your example to get this over the wire using
JSON with a customized response.  As I see in your code, you are relying on
Django to markup the results as opposed to letting JS do this -- that
definitely steered me in a new direction.  I am grasping this DRY principle,
fondly. :)   I haven't done any JSON with Django yet, so this provides a
concrete example how to accomplish it.  Thank you for your time and
contribution, Nick!!


Now just for that Celery part, and I think I'll be sitting pretty.

    Brandon



On Wed, Jul 20, 2011 at 10:18 AM, Nick Albright <nick.albright at gmail.com>wrote:

> Hello Brandon!
>
> I've never really worked with Celery or RabbitMQ (though, I want to!) so I
> can't really address those issues, but I have done a bit of ajax and django,
> so I can try to help out. (Uncertain based on your Qs if I can help, but I
> prolly can't hurt! :)
>
> 1) Sorry, sounds like a Celery Q, I can't help there.
> 2) I might be able to help here! = )
>
> So I assume you are doing some browser side ajax polling (using the task_id
> returned by Celery) to a django view which in turn will query to see if
> there are results to return.  Once you get the list from celery in the
> django view.  I'd format the list to HTML (seen below) and return that to
> the browser, which can then insert that in the umm.. div/list of the
> browser. (I use jQuery for all my ajax and dom manipulation)  So the django
> side might look something like:
>
> def check_for_results( request, task_id = None ):
>     response = {}
>
>     # Could be passing the task_id in the GET params
>     task_id = int( request.GET.get( 'task_id', task_id ) )  # Assumes
> task_id is really an int.  If not, don't cast it to one! :)
>
>     results = CheckCeleryForResults( task_id )    # Whatever celery magic
> goes here
>
>     if results:
>         # Convert the list to HTML for display
>         response[ 'list' ] = render_to_string(
> 'appname/inclues/ist_display.html',
>                                                            {
> 'results_list': results,
>                                                             },
> context_instance = RequestContext( request ) )
>         response[ 'done' ] = 1    # If you know there are more results
> coming later, could make this 0
>     else:
>         response[ 'done' ] = 0
>
>     return( HttpResponse( simplejson.dumps( response ),
> mimetype='application/json' )
>
>
> Note, I'm a huge fan of letting django format the HTML using a template, as
> I can use that same template on initial page generation if I have any list
> elements initially when the page is generated.  It's DRY and all that. :)
>
> And in your HTML, you have a call like:
>
>
> <script language="javascript" type="text/javascript">
> function CheckForResults( task_id ) {
>     $.getJSON( "{% url check_for_results %}",
>                       { task_id: task_id },
>                       function( json, textStatus ) {
>                           // Hey!  We got results! Let's show them to our
> beloved user!
>                           if ( json[ 'list' ] ) {
>                              $('#results_list').append( json[ 'list'] );
>                           }
>                           // Looks like we aren't done, let's poll again in
> a bit
>                           if ( json[ 'done' ] and !json[ 'done' ] ) {
>                              setTimeout( function() { CheckForResults(
> task_id ); }, 5000 );   // Wait 5 seconds and try again
>                           }
>                       }
>   );
> }
> </script>
>
>
> I hope that helps!
>  -Nick
>
>
> On Wed, Jul 20, 2011 at 8:51 AM, coding.solo <coding.solo at gmail.com>wrote:
>
>> Hi all, I hope you guys can help me on this:
>>
>>   I have a developing a Django app that processes many <reallyLongTask>.
>> My current implementation to get everything up and running was to block for
>> <reallyLongTask> and then deliver the results in a view.  That works great,
>> but was really long.  Now I want to take it 2.0 with the auspicious notion
>> of asynchronous back-end processing whilst polling with AJAX on the
>> front-end to populate the results as they are ready.  Currently, a
>> <reallyLongTask> returns a list of lists.  Instead of returning a list of
>> lists, I want to factor out the enveloping list and just want to return
>> lists.
>>
>>   Now begins my questions.  I have Celery successfully working with
>> RabbitMQ.  In my Django view, I execute my <reallyLongTask>.delay() and
>> catch its AsyncResult.  I get the AsyncResult.task_id over to the front-end
>> browser so that it can poll this task.
>>
>>    1)  In my AJAX Django view, how am I supposed to query the task with
>> only this task_id?
>>    2)  Since my original view no longer returns a list of lists, what do
>> you guys suggest I do to pass my *lists* back to the browser front-end
>> when each individual one is ready?  I would suspect RabbitMQ could handle
>> such an exchange somehow, possibly even Celery.  I am thinking I want to
>> populate a list for the AJAX view to draw from when the front-end polls that
>> view.
>>
>>
>>
>>   Thanks all!  Can't wait for my first PyOhio!
>>
>>
>>     Brandon Lorenz
>>
>> _______________________________________________
>> CentralOH mailing list
>> CentralOH at python.org
>> http://mail.python.org/mailman/listinfo/centraloh
>>
>>
>
> _______________________________________________
> CentralOH mailing list
> CentralOH at python.org
> http://mail.python.org/mailman/listinfo/centraloh
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/mailman/private/centraloh/attachments/20110720/549b7bfc/attachment-0001.html>