String concatenation - which is the fastest way ?
stefan_ml at behnel.de
Thu Aug 11 13:59:50 CEST 2011
Chris Angelico, 11.08.2011 12:59:
> On Thu, Aug 11, 2011 at 7:40 AM,<przemolicc at poczta.fm> wrote:
>> I am not a database developer so I don't want to change the whole process
>> of data flow between applications in my company. Another process is
>> reading this XML from particular Oracle table so I have to put the final XML there.
> I think you may be looking at a submission to
> http://www.thedailywtf.com/ soon. You seem to be working in a rather
> weird dataflow. :( Under the circumstances, you're probably going to
> want to go with the original ''.join() option.
>> This server has 256 GB of RAM so memory is not a problem.
>> Also the select which fetches the data is sorted. That is why I have to
>> carefully divide into subtasks and then merge it in correct order.
> There's no guarantee that all of that 256GB is available to you, of course.
> What may be the easiest way is to do the select in a single process,
> then partition it and use the Python multiprocessing module to split
> the job into several parts. Then you need only concatenate the handful
> of strings.
Or join them using an n-way merge.
More information about the Python-list