[postmodern-devel] using dao-class on database with column names like "addr:city"
Hi All; Sorry if this is a dumb newbie question. I have a PostGIS database with column names that are rather unfortunate (and created not by me but by/via many different bits of GIS software, and they sadly seem not to have mechanisms to specify rewriting of column names on the fly during the workflow). Some of the tables have column names which have colons in them. When I try and do the first Obvious Thing I can think of (name my dao-defclass slots e.g., |addr:city| in an attempt to match the column name), get-dao complains thusly: --- cut here --- No slot named addr:city in class CM-LOCATION. DAO out of sync with table, or incorrect query used. --- cut here A trip into the debugger leads me to believe that the slot name |addr:city| in the dao-class defclass gets rewritten as "addr_city", and then the get-dao from the table with column name "addr:city" (which does not seem to likewise get passed through to-sql-name, as it IS in fact an sql name) fails. (FWIW I have never had any trouble whatsoever with normally-named columns in almost all other databases I have processed.) Another Obvious Thing to try - setting *escape-sql-names-p* around the defclass (as suggested in another mailing to this list circa 2008) didn't seem to affect things either (but that could be Pilot Error). The final Obvious Thing to try was figuring out how to set the sql-name in the slot definition, but there seemed to be no initarg or other mechanisms I could spot. If some kind soul out there would point me in the right direction, I'd greatly appreciate it. Thanks in advance, -jm p.s., and then there are the database column names (e.g., "type") that are locked in various CL packages... -- --- John Morrison --- john.nmi.morrison@gmail.com
A trip into the debugger leads me to believe that the slot name |addr:city| in the dao-class defclass gets rewritten as "addr_city",
If colons are safe in SQL field names, you can change s-sql::to-sql-name to leave them intact (there seems to already by an exception for * characters).
p.s., and then there are the database column names (e.g., "type") that are locked in various CL packages...
Slot names can be locked? I've never heard of that. How?
On Tuesday 14 June 2011, Marijn Haverbeke wrote:
If colons are safe in SQL field names, you can change s-sql::to-sql-name to leave them intact (there seems to already by an exception for * characters).
Thanks, and will do... As regards colons in column names: while I am by no means an experienced SQL guy, AFAICT the various GIS tools I am using all seem (so far) to be functioning correctly on databases with those column names.
Slot names can be locked? I've never heard of that. How?
IIRC I got a package lock violation when naming the dao-class' slot "type" to mirror the SQL column name. I will go back and look again - only spent 2 seconds kludging around it, as I ran into the colon problem on the same table.... Thanks again for the help! -jm
_______________________________________________ postmodern-devel mailing list postmodern-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel
-- --- John Morrison --- john.nmi.morrison@gmail.com
John Morrison <john.nmi.morrison@gmail.com> writes:
On Tuesday 14 June 2011, Marijn Haverbeke wrote:
If colons are safe in SQL field names, you can change s-sql::to-sql-name to leave them intact (there seems to already by an exception for * characters).
Thanks, and will do...
As regards colons in column names: while I am by no means an experienced SQL guy, AFAICT the various GIS tools I am using all seem (so far) to be functioning correctly on databases with those column names.
For what it's worth, a column (or other object) may use any kind of string as its name if enclosed in double-quotes. They're a bit like CL symbols and ||s in that regard. db=> create table foo ("I am a Column" int); CREATE TABLE db=> insert into foo values (1); INSERT 0 1 db=> select * from foo; I am a Column --------------- 1 (1 row) Zach
On Tue, Jun 14, 2011 at 08:33:34AM -0400, John Morrison wrote:
On Tuesday 14 June 2011, Marijn Haverbeke wrote:
If colons are safe in SQL field names, you can change s-sql::to-sql-name to leave them intact (there seems to already by an exception for * characters).
Thanks, and will do...
As regards colons in column names: while I am by no means an experienced SQL guy, AFAICT the various GIS tools I am using all seem (so far) to be functioning correctly on databases with those column names.
Sorry, but colons aren't a avalid part of an SQL identifier. A <regular identifier> is a character string, up to 128 characters long, that consists only of letters, digits, and underscore characters. It must begin with a letter. In standard-conforming SQL 'addr:city' only works as a delimited identifier: A <delimited identifier> is a character string, up to 128 characters long, surrounded by a pair of double quote marks. (The delimiting double quotes aren't part of the <identifier>, so they're not included in the calculation of its size.) Two consecutive double quotes within the character string (i.e., "") represent one double quote mark; together, they count as one character when calculating the size of the <identifier>. A <delimited identifier> is case sensitive. That is, uppercase and lowercase letters within a <delimited identifier> are not interchangeable; This smells like open streetmap ... must have been SQL gurus that designed that schema :-/
Slot names can be locked? I've never heard of that. How?
IIRC I got a package lock violation when naming the dao-class' slot "type" to mirror the SQL column name. I will go back and look again - only spent 2 seconds kludging around it, as I ran into the colon problem on the same table....
This seems wrong. The symbol 'type' only causes lock-problems when you try to bind its function value. Are you shure you didn't: (defclass foo () ((type :accessor type))) ??? That _would_ try to create a function '(type instance)' and trigger the package lock violation error. HTH Ralf Mattes
Thanks again for the help!
-jm
_______________________________________________ postmodern-devel mailing list postmodern-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel
-- --- John Morrison --- john.nmi.morrison@gmail.com
_______________________________________________ postmodern-devel mailing list postmodern-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/postmodern-devel
On Tuesday 14 June 2011, rm@tuxteam.de wrote:
Sorry, but colons aren't a avalid part of an SQL identifier.
I would be happy with any mechanism at all which would enable me to cope with such non-conforming databases. What's the Right Thing to do?
This smells like open streetmap ... must have been SQL gurus that designed that schema :-/
Cloudmade.com produced the shapefiles (which presumably contain OSM data - not sure of the toolchain used). I used PostGIS 2.0 and its shp2pgsql (built from source) to put the shapefiles into PostGreSQL 9.0.4-2. Regarding OpenStreetMap: FYI when I was using Osmosis to import OSM data myself (into PostGIS 1.5x), I didn't see this problem as the schema was completely different and a lot simpler. And I cannot easily go back to Osmosis under PostGIS 2.0 as a lot of the PostGIS function names (upon which Osmosis relies) have changed.
This seems wrong. The symbol 'type' only causes lock-problems when you try to bind its function value. Are you shure you didn't:
(defclass foo () ((type :accessor type)))
At one point I am sure I did. Thanks for the help!! -jm -- --- John Morrison --- john.nmi.morrison@gmail.com
I would be happy with any mechanism at all which would enable me to cope with such non-conforming databases. What's the Right Thing to do?
The problem is that Postmodern went with a 'simplify Lisp symbols to get credible SQL names' scheme that assumes those credible SQL names are not escaped. Some major touching up would be necessary, both in the dao metaclass code and in s-sql, to make it possible to get around this assumption.
participants (4)
-
John Morrison -
Marijn Haverbeke -
rm＠tuxteam.de -
Zach Beane