[PyGreSQL] Improvements and directions for the future

[Christopher Sean Hilton]

On Feb 7, 2010, at 4:23 PM, Christoph Zwerschke wrote:

> Am 07.02.2010 21:17 schrieb Christopher Hilton:
>> Would it not be better for performance to leave the classic (import
>> pg) interface as a wrapper around the C library and have the DB-API2
>> interface as a wrapper around classic? E.g. doesn't the object oriented
>> nature of the DB-API2 interface make it more easier to build extensions
>> like copy in/copy out in a way that's most portable for developers?
> 
> This would be better for the performance of pg, but not of pgdb. Note that pg is already a thin wrapper around the C lib, with some additions which are of no interest for pgdb.

I will have to reacquaint myself with the details of pgdb's implementation.

>> A perceived lack of a DB-API2 interface in part driven by PyGreSQL's
>> lack of documentation of that interface;
> 
> Of course the DB-API2 documentation already exists, but we should really document the differences and additions - particularly which of the optional parts we implemented, which types are supported and how they work. A few examples would also not hurt.
> 

Yes, I recently jumped back into a pgdb project and the only Doco I found aside from the code itself was some generic DB-API2 docu which was aimed at pyscopg2. thus I ran into small but easily cleared issues regarding the parameters to the connect object. Etc. That's what I think the psotgresql people would like to see.

>> A handful of missing features, some of which are present: copy
>> in/copy  out and some of which aren't? bytea
> 
> Copy is partially implemented in pg via "inserttable". But we should also provide copy_from() and copy_to() methods for pgdb cursors. Bytea is actually supported by pg and pgdb; again it seems the lacking docs are the only problem here.
> 

A while back I had a project that needed to import large amounts of data from an excel spreadsheet. I believe that I made some patches to PyGreSQL to either expose the underlying database connection object or allow you to create a pgdb connect object from a pg connection. My goal was to be able to use copy_in from a DB-API2 connection object. I'm certain I have that running now and I have no problem giving back the patches if you are interested. 

I think that the PostgreSQL team sees DB-API2 as too tightly bound to ANSI SQL and thus a bit anemic. If I read the postgresql mailing list thread correctly I believe that they would like to see a DB-API2 compliant database driver for python with appropriate extensions to properly service PostgreSQL. Being a little obvious and redundant, it's not enough to have a feature in pg, they want a documented extension in pgdb. 

> Btw, we already have identified a couple of issues on our todo list:
> http://www.pygresql.org/future.html
> 

I saw this and it looks good. I believe that the problems here for both PyGreSQL are more perceptual than real. Perhaps it's time for a project representative to get the real picture to the PostgreSQL developers.

>> Some developer friendly features like a bug reporting database and
> > more modern revision control.
> 
> A bug tracker would also allow people to vote for features and make it easier to submit patches.

Yes. And down the road a wiki would allow users to submit documentation and examples. 

> 
> -- Christoph


-- Chris Hilton

Chris Hilton                                   e: chris|at|vindaloo|dot|com
----------------------------------------------------------------------------
                  "The pattern juggler lifts his hand; The orchestra begin.
      As slowly turns the grinding wheel in the court of the crimson king."
                                           -- Ian McDonald / Peter Sinfield