hello list,
i've recently found postmodern on cl.net, and i was wondering that there seems to be a massive amount of duplication of efforts among the following projects:
- pg is a long-time postgresql socket based lib: http://common-lisp.net/project/pg/
- cl-rdbms (our fresh clsql replacement, http://common-lisp.net/project/cl-rdbms/ ) which aims to be a backend independent sql lib, just like clsql, with sexp sql syntax ( see http://common-lisp.net/cgi-bin/darcsweb/darcsweb.cgi?r=cl-rdbms-cl-rdbms;a=h... for a quick impression ). it has a single backend for now, based on pg.
- and postmodern which is something like a mixture of pg and cl-rdbms
we should bring out the best of these... my first impression of postmodern is that it's more modern and better organized code then pg which is good.
and to also write actual suggestions besides to this rant, i'd use http://common-lisp.net/project/local-time/ as the date/time abstraction data type. it supports timezones and has a string parser/printer.
and fyi, trivial-sockets seems to be a dead project. i've tried to contact the author several times with patches and there wasn't any response; #lisp is on the same opinion. usocket at least welcomes patches, but it's not really important if t-s can already do everything needed.
any toughts?
Hello Attila,
I was aware of PG when I wrote Postmodern -- but as you say, its code is not in a very good state so I opted to re-write it. I didn't find out about cl-rdbms until very recently, and since it seemed in a rather expermental and undocumented state I didn't pay too much heed to it.
The part of Postmodern that does SQL-as-s-expressions is something that I wrote half a year ago and has been in use at the place where I work for a while now, so I chose to stick with that instead of similar existing project. There's also a factor of 'it is more fun to write something myself than it is to bend existing existing into the shape I need' involved here, so I (intentionally) did not always make the most 'cost-effective' choices.
I checked out local-time (writing a date/time library was something I'd rather have avoided), but it did not seem to provide the necessary data types for representing the various SQL types, and it is Unix-only at the moment (uses the unix timezone files), and I want to be more portable than that. -- Of course, optional, unportable support for timezones is better than no support for them at all, so maybe we could add something like that.
Though I haven't found any real clean way to model optional dependencies in ASDF yet -- see the hideous #.-hacks in the asd files for cl-postgres and postmodern. I know there are ASDF extensions for things like that, but nobody uses them so it's rather inconvenient to require them.
Trivial sockets... I know (you've told me multiple times on IRC), but until usocket has proper and portable support for byte streams this seems the only alternative (I'll switch to usocket at the drop of a hat when it gets byte streams).
Anyway, thanks for the reactions, I'll look a bit further into cl-rdbms as soon as I have the time, and we'll see if some kind of merging is in order. But maybe you're better off just using the parts of Postmodern that you need (cl-postgres) in that project? Or a branch of it, if you must... My experiences with you in UCW context have kind of suggested that we've got a bit of a different approach towards library building -- I'm something of a minimalist, you like to throw in all kind of undeniably neat but very experimental and expensive features ;)
Regards, Marijn
hi,
we had enough of pg and implemented a Postmodern backend for cl-rdbms. it's using only the cl-postgres package from Postmodern and works like a charm.
in the process i've added blob and null support and made some other small cleanups. please find the changes at
darcs pull http://common-lisp.net/project/cl-wdim/darcs/postmodern
you may prefer not to pull one of the changes, which moves to local-time for time representation. it works fine, but there's no interval support in local-time, yet.
i think the other changes are generally useful, please take a look, i hope you'll like them!
and i also hope you don't mind that i've "set up" a Postmodern repo for our changes. some people are more sensitive in these things, so please let me know if you have any objections and i'll remove the repo from the public space.
cl-postgres is one of the cleaner libs out there, it was very easy to feel at home in the code, thanks for this nice lib!
regards,
Hey,
Thanks for the patches! All have been applied to the main repo, except for the local-time one. (Incidentally, my efforts to fix/merge/extend local-time for use with postmodern have more or less ceased -- I came across some very hairy problems, and I'm not currently doing anything that would benefit from this work, so motivation ran out.)
The separate repo is perfectly fine, as long as you don't go promoting it as a 'better alternative' or something, but seeing how little noise you usually make about your software, I doubt that that will be a problem.
Cheers, Marijn
you can find the following patch at: darcs pull http://common-lisp.net/project/cl-wdim/darcs/postmodern
Tue Aug 21 15:34:46 CEST 2007 attila.lendvai@gmail.com * Use ironclad for md5 hashing (much better optimized)
we were profiling our code and saw that postgres connecting was among the consumers. the main bottleneck was the md5 hashing/printing, so tried with ironclad and it was a lot faster.
you may consider moving to ironclad and then pulling the patch. i hope you'll like it.
at your service,
Hello Attila,
Finally getting around to looking at cl-rdbms, I found it to be very well thought out -- its syntax probably supports more different constructs than s-sql does at this point. Its syntax is probably also quite a bit more thought out. If, a month ago, you had had some minimal documentation, I'd probably have realized this earlier, heh. Also, one of the reasons I stuck to Postgres-specificness is that I don't know enough about other databases, and do not have the time, to write stuff in a multi-backend way. Maybe the cl-rdbms code can help there (though it's also still only tested on pg). So now the burning question presents itself -- do we merge?
I haven't played with cl-rdbms enough to know how much I like it. Its DAO (records) approach seems more primitive than Postmodern's. How stable are the various parts of the library? Is anybody planning to put some effort into documentation anytime soon?
I guess what it boils down to is: A) I'm not in the possession of a lot of time for merging these libraries myself at the moment, but f you or someone else would be interested in putting work into this I'd be happy to cooperate. B) I'm rather proud of some of the stuff I did in the Postmodern code, and of the docs, so I want to at least have some influence in the merging process. But of course, C) It's no good to have multiple similar libraries, we should all join together and make a solid CLSQL-replacement.
So, let me know what you think.
Regards, Marijn
So, let me know what you think.
sorry for being quiet, i promise to write a more detailed answer later.
until then a few quick toughts: - i didn't know it's you Marijn (hint for the website :) - cl-rdbms is only 7 weeks old... :) - unfortunately i doubt there will be any docs for cl-rdbms. we belive that the code must be written so that it's self documenting (by using long names without abbreviations, factor out everything feasible, etc) with an easy-to-play-with test suite that covers/demonstrates all the features. - the multi backend stuff is only a theoretical framework with a single pg backend for now. it's only based on our previous programming experience, so until the second backend is made... - after a first impression i like many things in postmodern - i'm also open for merging and in no way want to assimilate postmodern without you being involved - not too much time here either (the rewrite of clsql was a sacrifice itself that didn't worth it in the short term. let's hope there'll be a long-term... :) - the ucw ajax branch was an experiment of mine and i'm not especially proud of the internal code quality. (ucw is basically a repo of random web-related features with more connections between them than necessary, but hopefully Drew's cleanup will handle that)
happy new year, i'll be back later
Hi,
In case anybody is hesitating to start using Postmodern because this thread seems to indicate that there is a huge overhaul/merge imminent... do not worry about this. It seems that both Attila and me have our hands too full to spend serious time on merging the libraries at the moment. (Also, we have difficulty agreeing whether documentation and premature optimization are a good thing or a bad thing.)
Cheers, Marijn
Hi Marijn,
Marijn Haverbeke wrote:
In case anybody is hesitating to start using Postmodern because this thread seems to indicate that there is a huge overhaul/merge imminent...
I have been meaning to write about my experience using Postmodern for a while, I'll take this opportunity. Only one thing to say - thank you! I was using CLSQL for a web project of mine, it was not really working out for me. There are little problems that used to leave my head scratching. No knock against CLSQL, it's an incredibly useful library, but there are some little things which I thought could have been simpler to do.
I found about Postmodern in Zach's blog, and I was initially skeptical about using it as the version is still 0.23 and the project page says there are still some things to be done. But I was totally wrong. Apart from a bit of initial confusion (about auto-id - perhaps an example would do well there - very good documentation, apart from that), I have to say it's very simple to use, and I find that it's very well thought out.
Perhaps automatic joins in select-dao/get-dao a la CLSQL select would be cool, but it's not really show stopper for me - using the query interface is just as simple, and good enough for me. Perhaps would be nice as a future feature, just for completeness.
I always preferred PostgreSQL over MySQL for my projects, so it's like almost the perfect library for me. And just in time! :). Thank you once again. I like Postmodern just the way it is, but I won't really complain if you're adding more features - but I hope you keep it as clean and simple to use as it is now :)
Cheers, Vamsee.
Hey,
Thanks for the encouraging words! About the DAO interface: The reason it is so simple now is that I couldn't think of a more complex approach that would be general enough to be really useful. I had some ideas, but they all ended up locking the library user into a certain way of working with databases, and I didn't want to do that. If you (or anyone on this list) knows of some library (lisp or otherwise) that manages to abstract DAOs in an elegant way, let me know. Also, if you have some proposal for a change or addition to the library, send it to this list. The current interface is by no means set in stone -- though I try not to introduce incompatible changes too often ;).
While we are at the subject: One thing that is missing from the DAOs is that you can not retrieve one, update its primary key, and then store the changes -- it will be saved as a new record, instead of updating the old one. One way around this would be to have a hidden field in every DAO object, and to store their primary keys (as a list) in there when they are retrieved from the database. But this feels a bit clunky, so any suggestions are welcome.
Cheers, Marijn
(or anyone on this list) knows of some library (lisp or otherwise) that manages to abstract DAOs in an elegant way, let me know. Also, if
http://common-lisp.net/project/cl-perec/ may be interesting when talking about DAOs and joins. it is the persistency part of cl-wdim. basically it's a CLOS metaclass that brings in rdbms based persistency, pretty much how rucksack or allegro cache works, but with an RDBMS backend. it also have a declarative query language that handles joins transparently among other things.
we only recently factored it out from cl-wdim and there are various patched dependencies, but there was already a user who could successfully run the test suite.
postmodern-devel@common-lisp.net
-
Attila Lendvai -
Marijn Haverbeke -
Vamsee Kanakala