[PyGreSQL] Will PygreSQL help me to solve my problem

[Christoph Zwerschke]

Hi Kumar,

I don't think you will really need Python or PyGreSQL here. You can proceed
as follows:

- create a "staging table" corresponding 1:1 to your flat file
- load your flat file into that staging table with COPY command
- vacuum analyze your staging table
- create primary key and indexes on the staging table
in order to speed up the following selects
- drop indexes and foreign constraints on your actual tables
in order to speed up the following inserts
- populate your acutal database tables with
insert into actual tables select complex query from staging table
- vacuum analyze your actual tables
- create indexes and foreign constraints on the actual tables
- drop the staging table

Only if you really need to do a lot of data processing that cannot be done
with SQL, or your flat file is in a format that cannot be loaded with COPY,
you need to resort to Python.

- you can read the flat file directly with Python
- you can process the data with Python
- insert the data into your tables with PyGreSQL

Please note that PyGreSQL has an inserttable method for bulk inserts that
corresponds to the COPY command.

Gtx
Christoph

----- Original Message ----- 
From: "Kumar S" <ps_postgres at yahoo.com>
To: <pygresql at Vex.Net>
Sent: Thursday, September 16, 2004 1:05 AM
Subject: [PyGreSQL] Will PygreSQL help me to solve my problem


> Dear Group,
> I am a new member to this group and I have
> low-intermediate skill in python and I really want to
> brush up my skills.
>
> I am currently designing a database to include data
> from a tab delimitted text file into many tables (1
> file to Many tables).
>
> my schema is in 1-normal form. There are 9 tables over
> all.
>
> I cannot populate the tables using COPY command and
> insert due to column variation between file and tables
> and laborious process respectively.
>
> In order to proceed I wanted to populate a table
> initially that has no foreign key defined.
>
> After this I want to take the primary_key_id from that
> table and fill associated data into other tables.
>
> Will PygreSQL help me in this situation. Please help
> me I am in a grave situation. I am research student in
> medical university.
>
> Thank you.
>
> Kumar