Hi All,
I'm trying to add support for bulk copying to postmodern. Does anyone have any ideas on how it should be done. Here's my suggestion...
1) Add message primitives (we seem to already have copy-done-message)
(define-message copy-data-message #\d (data) (string data))
(define-message copy-fail-message #\f (reason) (string reason))
2) Add a "bulk-copier" class with slots for db ; the connection we're using table ; the table we're copying from/to from/to ; whether we're copying from or to a table header ; whether to include a header row in output/input ;; plus slots for the CSV options delimiter null-str quote-str escape-str
3) A function open-copier creates an instance of the above class and sends the corresponding copy statement to Postgres before returning the object.
4) A function copy-row takes a row (i.e. a simple list of Lisp values), and writes them to the stream attached to the copier.
5) A function copy-done lets Postgres know we're finished and commits the change to the database.
Anyone got any hints for debugging this low-level stuff? I seem to be able to open a copier (and send the copy statement to pg) but when I send the complete-command message in copy-done, my code is hanging.
Hi Andy,
Your approach sounds good. I've never used bulk data copying, so I can't really give very useful feedback. As for debugging, well, have the code log what it's sending and then double-check with the protocol docs—there are often subtle gotchas described in the 'message flow' document that aren't mentioned in the message-by-message reference.
Best, Marijn
postmodern-devel@common-lisp.net