Python script not mapping our site correctly?

[Paul McNett]

michael.buonomo at gmail.com wrote:
> We have been using the Google recommended python script for about a
> year. 

Which script would that be? Googling for 'python script' yields approx. 
27 million hits.

> We recently realized that the script was not crawling our sites
> url's, but just our folders which reside on the server. 

The behavior of the script recently changed, or you were running the 
script for a year not realizing what its purpose was?

> The python
> script seems to be designed for 'non database' sites, not a site which
> is using .asp, and has dynamic pages.

It sounds like your script is just traversing a directory structure on 
disk, presumably indexing the text in the files found there. I think it 
sounds like (but I'm guessing, here) that you want what is known as a 
web crawler, that communicates via http with your site, follows links, 
and indexes the resulting pages.

> We are an ecommerce site. What are other ecommerce sites using to
> create an xml file?

XML is mostly used to persist data of one sort or another. What kind of 
XML file do you want to create?

> Are they using the python script?

We aren't going to be able to help you with this question until you 
become *much more specific*:

+ Which python script? Where did you download it from and what is it called?

+ What is the purpose of the XML you want generated? (WAG: Submit to 
Froogle?)

+ What pages do you want indexed?

Usually, for database-driven ecommerce sites, for developing lists of 
products for submission to places like Froogle, I don't go via the web 
interface at all: I write python scripts (there's that word again!) to 
connect to the database, and run queries to determine the results, run 
that through a template for each line that figures out things such as 
the URL of the page that represents the product, etc.)

But I hesitate to say much more until we understand what you want your 
python script to do.


-- 
pkm ~ http://paulmcnett.com