[Chennaipy] Reduce the search time on a huge text file
Vignesh Ragupathy
me at vikki.in
Thu Jun 24 06:49:54 EDT 2021
Hi,
Another approach is to convert the data as Pandas dataframe and use some
vector operations.
A sample code is attached.
Regards,
Vignesh
On Thu, Jun 24, 2021 at 2:23 PM Saravanan Muthuramalingam <
saravana4285 at gmail.com> wrote:
> Hi Abirami,
> Please find the attached sample to use yield , when you call yield
> with next keyword, it will iterate and send you next batch of lines , so
> that your memory will be cleaned and the execution can be fast, try
> tweaking this to meet your requirement, thanks
>
> Sara
>
> On Thu, Jun 24, 2021 at 12:12 PM Abirami Moorthy <mabirami8799 at gmail.com>
> wrote:
>
>>
>>
>> On Thu, 24 Jun 2021 at 09:41, Thirunarayanan Srinivasan <
>> tstnarayan at gmail.com> wrote:
>>
>>> On Thu, 24 Jun 2021 at 6:22 AM Saravanan Muthuramalingam <
>>> saravana4285 at gmail.com> wrote:
>>>
>>>>
>>>> Hello,
>>>>
>>>>
>>> Here 25 lakhs tamil words collected link in below
>>
>> https://github.com/KaniyamFoundation/all_tamil_words.
>>
>> i have done a simple program using above the text file
>>>>>
>>>>
>>
>>> https://github.com/KaniyamFoundation/all_tamil_words
>>>>> it takes 4 min to execute the program.
>>>>>
>>>>
>> thankyou for your all kind replies, i'll keep trying what you
>> said.
>>
>>
>>
>>> Regards,
>>>>> M.Abirami
>>>>>
>>>>> _______________________________________________
>> Chennaipy mailing list
>> Chennaipy at python.org
>> https://mail.python.org/mailman/listinfo/chennaipy
>>
> _______________________________________________
> Chennaipy mailing list
> Chennaipy at python.org
> https://mail.python.org/mailman/listinfo/chennaipy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/chennaipy/attachments/20210624/7cb04b43/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tamil_words.ipynb
Type: application/x-ipynb+json
Size: 2665 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/chennaipy/attachments/20210624/7cb04b43/attachment-0001.bin>
More information about the Chennaipy
mailing list