[Tutor] Runtime error while Test data creation

[bob gailer]

On 10/7/2011 2:19 PM, Guess?!? wrote:
> Hello all,
>
> I am trying to create some test data for a search module that I am 
> building. Since I dont want to hit performance related issues late in 
> the game, I decided to create half a million records in the DB.
>
> My approach is create a csv file with valid UUIDs which I already 
> have. Read each record from csv and generate random data and do DB 
> insert as long the csv file has records in them. This crashes near 
> 100000 record insertions.
>
> The problem is that my program eventually fails with this error. Any 
> ideas what I can improve to make it go all the way?
>
> Runtime Error!
> Program: C:\Python27\pythonw.exe
> This app has requested the Runtime to terminate it in an unusual 
> way. Please contact application support team

Please just post ordinary text. No colors, no fancy formatting.

Please always post the entire traceback so we tell where the code failed.
>
> Here is the code that I wrote to do this.
>
> =======================================================================
> import pyodbc, random, datetime, uuid
>
> #INT
> conn = pyodbc.connect('DRIVER={SQL 
> Server};SERVER=SERVERNAME\INT_FOUNDATIONS;DATABASE=membership_service;UID=int_usr;PWD=blah')
>
>
> c = conn.cursor()
>
> # "enterprise_person_id" TEXT NOT NULL,
>
> # read file
> f = open('C:\\Users\\garora\\Desktop\\INTPPLHALFMIL.csv','r')
>
> # Hardcoded valid seed values
>
> # INT
> programId = 50801
> packageId = 46101
> pkg_optionId = 40301
>
>
> seed_data = "TEST_DATA"
>
> numberPrePend = "AAAA-"
>
> create_dt = datetime.datetime.now();
> modified_dt = datetime.datetime.now();
>
> i = 0
>
> for each in f:
> if (i == 0):
> i = i + 1
> continue;
> (id, entrpid,) = each.split(',')
> print id
> id = id.replace('\n', '')
> print id
> id = uuid.UUID(id)
> #enterprise_person_id
> print entrpid
> entrpid = entrpid.replace('\n', '')
> print entrpid
> entrpid = uuid.UUID(entrpid)
> #random values for few columns
> statusRandom = random.randint(1,4)
> sourceRandom = random.randint(1,7)
> inputChannelRandom = random.randint(1,5)
> marketRandom= random.randint(1,10)
> trialRandom = random.randint(0,1)
> monthRandom = random.randint(3,12) # prevent month of feb
> dateRandom = random.randint(1,30) # prevent 31st on invalid month
> yearRandom = random.randint(2007,2010)
> member_start_dt = datetime.datetime(yearRandom, monthRandom, 
> dateRandom,0,0,0)
> current_term_start_dt = datetime.datetime(yearRandom, monthRandom, 
> dateRandom,0,0,0)
>
> print "id: " + str(i)
> print " programId: " + str(programId)
> print " packageId: " + str(packageId)
> print " pkgoptionId: " + str(pkg_optionId)
> print " status id: " + str(statusRandom)
> print " source id: " + str(sourceRandom)
> print " input channel id: " + str(inputChannelRandom)
> print " market id: " + str(marketRandom)
> print " person id: " + str(id)
> print "membership id: " + numberPrePend + str(i)
> print " member_start_dt " + str(member_start_dt)
> print " current_term_start_dt " + str(current_term_start_dt)
> print " started_as_trial " + str(trialRandom)
> print " enterprise_person_id " + str(entrpid)
>
> record = (i, programId, packageId, pkg_optionId, statusRandom, 
> sourceRandom, inputChannelRandom,
> marketRandom, str(id), numberPrePend + str(i), member_start_dt, 
> current_term_start_dt, trialRandom
> ,seed_data, seed_data, create_dt, modified_dt, str(entrpid))
> print record
> c.execute("""insert into Members(id, membership_program_id, 
> membership_package_id,
> membership_pkg_option_id, membership_status_id, membership_source_id, 
> src_input_channel_id,
> src_market_id, person_id_ref, membership_id, member_start_dt, 
> current_term_start_dt, started_as_trial,
> created_by,  modified_by, create_dt, modified_dt, enterprise_person_id)
> values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""", record)
> conn.commit()
> i = i + 1
>
> c.close()
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


-- 
Bob Gailer
919-636-4239
Chapel Hill NC

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20111007/d462a3b2/attachment.html>