Re: [Ncr-Python.in] developing a data mining tool, along with a webserver
Copying python mailing list for more suggestions :-) Start with installing webserver like apache and db. For web development, use a framework like web2py or Django. I have been using web2py for quite some time and it is very simple and follows the MVC architecture and pretty good integration with database layer. Django has been around for a long time and has pretty big user base when compared to web2py. Web2py is comparatively new. You can choose either of these frameworks. You can of course avoid using any framework all together but frameworks makes your life little simpler and it will be easier for you as you have not done anything for web before. For data mining tool, you need good knowledge of db. Processing of data can be done very easily in python layer. You can check out http://wiki.python.org/moin/NumericAndScientific/Plotting for graphs/charts etc. -- Regards Rohan On Thu, Feb 24, 2011 at 4:19 PM, vijay shanker <deontics@gmail.com> wrote:
hi, hav been coding with python for arn 2 yrs.. never did nething for web.. bascally into application development..coding frm arn 4 years.. vijay
On Thu, Feb 24, 2011 at 1:34 PM, Rohan Malhotra <yourbuddyrohan@gmail.com>wrote:
can you tell me at about your conform level with python? Are you just getting started with python? do you have general programming experience?
It will be good, if you post this question to entire list. More gurus are present on list ;)
Regards
On Thu, Feb 24, 2011 at 1:30 PM, vijay shanker <deontics@gmail.com>wrote:
hi rohan,
i want to use python for developing a data mining tool, along with a webserver ..will be using mysql.. provide me some inputs on how to stat with this problem.. regards vijay shanker
-- Regards Rohan
On Thu, Feb 24, 2011 at 1:30 PM, vijay shanker <deontics@gmail.com> wrote:
hi rohan,
i want to use python for developing a data mining tool, along with a webserver ..will be using mysql.. provide me some inputs on how to stat with this problem.. regards vijay shanker
What is this project going to be licensed in have you put up a wiki page or some write up somewhere ? -Satya Ps: guys please please adhere to this http://lug-iitd.org/Mailing_List_Guidelines i do sound like a broken record , but i have to please ....
Hi, I have been doing some data mining/ scrapping work for sometime using Python. I donot know the kind of documents you are going handle. Since you mentioned webserver, I think it would be of HTML and XML documents mostly (May be I am wrong). But if thats what you are doing, then BeautifulSoup is an important Library for DOM traversing. And there is a website named http://www.scraperwiki.com, which supports data scrapping from web and online storage of the data you collect on their datastore. You can license the data as per your wish. It supports multiple languages like Python, PHP, Ruby..etc., This site is helpful, as it reduces your burden of setting up your own database. If you want data for other applications, they provide a API for datastore access. Hope your tool was as I guessed and the information is useful. -- P.Arunmozhi Twitter: @tecoholic Website: http://www.arunmozhi.in On Thu, Feb 24, 2011 at 5:03 PM, satyaakam goswami <satyaakam@gmail.com>wrote:
On Thu, Feb 24, 2011 at 1:30 PM, vijay shanker <deontics@gmail.com>wrote:
hi rohan,
i want to use python for developing a data mining tool, along with a webserver ..will be using mysql.. provide me some inputs on how to stat with this problem.. regards vijay shanker
What is this project going to be licensed in have you put up a wiki page or some write up somewhere ?
-Satya Ps: guys please please adhere to this http://lug-iitd.org/Mailing_List_Guidelines i do sound like a broken record , but i have to please ....
_______________________________________________ Ncr-Python.in mailing list Ncr-Python.in@python.org http://mail.python.org/mailman/listinfo/ncr-python.in Mailing list guidelines : http://lug-iitd.org/Mailing_List_Guidelines
i'll have tons of excel sheets with data .. to generate cluster and display them on a web page under diferent catgories.. welcomes suggestion.. vijay shanker On Thu, Feb 24, 2011 at 10:32 AM, aruntheguy@gmail.com <aruntheguy@gmail.com
wrote:
Hi,
I have been doing some data mining/ scrapping work for sometime using Python. I donot know the kind of documents you are going handle. Since you mentioned webserver, I think it would be of HTML and XML documents mostly (May be I am wrong). But if thats what you are doing, then BeautifulSoup is an important Library for DOM traversing.
And there is a website named http://www.scraperwiki.com, which supports data scrapping from web and online storage of the data you collect on their datastore. You can license the data as per your wish. It supports multiple languages like Python, PHP, Ruby..etc., This site is helpful, as it reduces your burden of setting up your own database. If you want data for other applications, they provide a API for datastore access.
Hope your tool was as I guessed and the information is useful.
-- P.Arunmozhi Twitter: @tecoholic Website: http://www.arunmozhi.in
On Thu, Feb 24, 2011 at 5:03 PM, satyaakam goswami <satyaakam@gmail.com>wrote:
On Thu, Feb 24, 2011 at 1:30 PM, vijay shanker <deontics@gmail.com>wrote:
hi rohan,
i want to use python for developing a data mining tool, along with a webserver ..will be using mysql.. provide me some inputs on how to stat with this problem.. regards vijay shanker
What is this project going to be licensed in have you put up a wiki page or some write up somewhere ?
-Satya Ps: guys please please adhere to this http://lug-iitd.org/Mailing_List_Guidelines i do sound like a broken record , but i have to please ....
_______________________________________________ Ncr-Python.in mailing list Ncr-Python.in@python.org http://mail.python.org/mailman/listinfo/ncr-python.in Mailing list guidelines : http://lug-iitd.org/Mailing_List_Guidelines
_______________________________________________ Ncr-Python.in mailing list Ncr-Python.in@python.org http://mail.python.org/mailman/listinfo/ncr-python.in Mailing list guidelines : http://lug-iitd.org/Mailing_List_Guidelines
On Fri, Feb 25, 2011 at 1:39 AM, vijay shanker <deontics@gmail.com> wrote:
i'll have tons of excel sheets with data .. to generate cluster and display them on a web page under diferent catgories.. welcomes suggestion..
explain in detail , show some code snippets if you do not code its still ok , but still you need to share some details that will elicit interest and attention from others. also everyone on this thread and otherwise please look at the Mailing list guidelines before posting http://lug-iitd.org/Mailing_List_Guidelines , in order to bring sanity to the conversations we are having.
_______________________________________________ Ncr-Python.in mailing list Ncr-Python.in@python.org http://mail.python.org/mailman/listinfo/ncr-python.in Mailing list guidelines : http://lug-iitd.org/Mailing_List_Guidelines
on limitation with BeautifulSoup though - if there is some javascript in your page, it won't be executed. BeautifulSoup simply fetches the soucre of page not DOM per se. On Fri, Feb 25, 2011 at 12:02 AM, aruntheguy@gmail.com <aruntheguy@gmail.com> wrote:
Hi,
I have been doing some data mining/ scrapping work for sometime using Python. I donot know the kind of documents you are going handle. Since you mentioned webserver, I think it would be of HTML and XML documents mostly (May be I am wrong). But if thats what you are doing, then BeautifulSoup is an important Library for DOM traversing.
And there is a website named http://www.scraperwiki.com, which supports data scrapping from web and online storage of the data you collect on their datastore. You can license the data as per your wish. It supports multiple languages like Python, PHP, Ruby..etc., This site is helpful, as it reduces your burden of setting up your own database. If you want data for other applications, they provide a API for datastore access.
Hope your tool was as I guessed and the information is useful.
-- P.Arunmozhi Twitter: @tecoholic Website: http://www.arunmozhi.in
On Thu, Feb 24, 2011 at 5:03 PM, satyaakam goswami <satyaakam@gmail.com> wrote:
On Thu, Feb 24, 2011 at 1:30 PM, vijay shanker <deontics@gmail.com> wrote:
hi rohan,
i want to use python for developing a data mining tool, along with a webserver ..will be using mysql.. provide me some inputs on how to stat with this problem.. regards vijay shanker
What is this project going to be licensed in have you put up a wiki page or some write up somewhere ?
-Satya Ps: guys please please adhere to this http://lug-iitd.org/Mailing_List_Guidelines i do sound like a broken record , but i have to please ....
_______________________________________________ Ncr-Python.in mailing list Ncr-Python.in@python.org http://mail.python.org/mailman/listinfo/ncr-python.in Mailing list guidelines : http://lug-iitd.org/Mailing_List_Guidelines
_______________________________________________ Ncr-Python.in mailing list Ncr-Python.in@python.org http://mail.python.org/mailman/listinfo/ncr-python.in Mailing list guidelines : http://lug-iitd.org/Mailing_List_Guidelines
-- Regards Rohan
On Fri, Feb 25, 2011 at 10:31 AM, Rohan Malhotra <yourbuddyrohan@gmail.com> wrote:
on limitation with BeautifulSoup though - if there is some javascript in your page, it won't be executed. BeautifulSoup simply fetches the soucre of page not DOM per se.
Oh dear, what a mess of a thread. @Rohan, please read what Satyakaam posted: http://lug-iitd.org/Mailing_List_Guidelines @Vijay, as Satyakaam wrote, give more details about where you are stuck. We cannot do complete homework for you.
participants (5)
-
aruntheguy＠gmail.com -
Nandeep Mali -
Rohan Malhotra -
satyaakam goswami -
vijay shanker