As common-lisp.net has been pretty unreliable in the last days, we've moved development over to the bknr.net server maintained by Hans. If you're interested in trying out the soon-to-be-released Hunchentoot version, get all of Hunchentoot, Chunga, and FLEXI-STREAMS from here:
http://bknr.net/trac/browser/trunk/thirdparty
If you're using Drakma, you'll have to grab it from bknr.net as well, as there are incompatible changes in Chunga.
The previous development history can be seen at
http://trac.common-lisp.net/tbnl/browser/branches
whenever the server is up.
Edi.
To make your life easier, I have create an ediware package that you can use to check out Hunchentoot, Drakma and all dependencies with one command:
svn co svn://bknr.net/svn/ediware
If you use this, please let me know. If it is not used, I will not bother updating the externals list in the ediware package.
-Hans
On Mon, May 26, 2008 at 4:39 PM, Edi Weitz edi@agharta.de wrote:
As common-lisp.net has been pretty unreliable in the last days, we've moved development over to the bknr.net server maintained by Hans. If you're interested in trying out the soon-to-be-released Hunchentoot version, get all of Hunchentoot, Chunga, and FLEXI-STREAMS from here:
http://bknr.net/trac/browser/trunk/thirdparty
If you're using Drakma, you'll have to grab it from bknr.net as well, as there are incompatible changes in Chunga.
The previous development history can be seen at
http://trac.common-lisp.net/tbnl/browser/branches
whenever the server is up.
Edi. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
The idea of ediware package is really great!
On Mon, May 26, 2008 at 3:04 PM, Hans Hübner hans@huebner.org wrote:
To make your life easier, I have create an ediware package that you can use to check out Hunchentoot, Drakma and all dependencies with one command:
svn co svn://bknr.net/svn/ediware
If you use this, please let me know. If it is not used, I will not bother updating the externals list in the ediware package.
-Hans
On Mon, May 26, 2008 at 4:39 PM, Edi Weitz edi@agharta.de wrote:
As common-lisp.net has been pretty unreliable in the last days, we've moved development over to the bknr.net server maintained by Hans. If you're interested in trying out the soon-to-be-released Hunchentoot version, get all of Hunchentoot, Chunga, and FLEXI-STREAMS from here:
http://bknr.net/trac/browser/trunk/thirdparty
If you're using Drakma, you'll have to grab it from bknr.net as well, as there are incompatible changes in Chunga.
The previous development history can be seen at
http://trac.common-lisp.net/tbnl/browser/branches
whenever the server is up.
Edi. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
Hi,
On 26-May-08, at 3:04 PM, Hans Hübner wrote:
To make your life easier,
It definitely does.
I have create an ediware package that you can use to check out Hunchentoot, Drakma and all dependencies with one command:
svn co svn://bknr.net/svn/ediware
If you use this, please let me know.
I'll be using it.
If it is not used, I will not bother updating the externals list in the ediware package.
There's stuff in there that I don't think is Edi's code. Will you be keeping these up to date as well? How is this supposed to work?
Thanks for doing this. Great idea!
Cheers, Bob
-Hans
On Mon, May 26, 2008 at 10:09 PM, Bob Hutchison hutch-lists@recursive.ca wrote:
On 26-May-08, at 3:04 PM, Hans Hübner wrote:
I have create an ediware package that you can use to check out Hunchentoot, Drakma and all dependencies with one command:
svn co svn://bknr.net/svn/ediware
If you use this, please let me know.
I'll be using it.
Okay, so I will be maintaining it.
If it is not used, I will not bother updating the externals list in the ediware package.
There's stuff in there that I don't think is Edi's code. Will you be keeping these up to date as well? How is this supposed to work?
The idea is that by checking out from the ediware URL, you will always get all dependencies. The URL may change slightly, and we may introduce a mechanism so that one can check out releases. I will announce changes on this list.
-Hans
So what's the canonical location for Hans' branch now?
Thanks,
Cyrus
On May 26, 2008, at 12:04 PM, Hans Hübner wrote:
To make your life easier, I have create an ediware package that you can use to check out Hunchentoot, Drakma and all dependencies with one command:
svn co svn://bknr.net/svn/ediware
If you use this, please let me know. If it is not used, I will not bother updating the externals list in the ediware package.
-Hans
On Mon, May 26, 2008 at 4:39 PM, Edi Weitz edi@agharta.de wrote:
As common-lisp.net has been pretty unreliable in the last days, we've moved development over to the bknr.net server maintained by Hans. If you're interested in trying out the soon-to-be-released Hunchentoot version, get all of Hunchentoot, Chunga, and FLEXI-STREAMS from here:
http://bknr.net/trac/browser/trunk/thirdparty
If you're using Drakma, you'll have to grab it from bknr.net as well, as there are incompatible changes in Chunga.
The previous development history can be seen at
http://trac.common-lisp.net/tbnl/browser/branches
whenever the server is up.
Edi. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
On Tue, May 27, 2008 at 7:59 AM, Cyrus Harmon ch-tbnl@bobobeach.com wrote:
So what's the canonical location for Hans' branch now?
My former branch now lives in svn://bknr.net/svn/trunk/thirdparty/hunchentoot/ - Note that you can also use http://bknr.net/svn/... if your firewall does not like outgoing svn connections.
-Hans
On Mon, 26 May 2008 22:59:09 -0700, Cyrus Harmon ch-tbnl@bobobeach.com wrote:
So what's the canonical location for Hans' branch now?
The bknr.net one I sent to the list yesterday (or the "ediware" package which references the same content). The folder called "hunchentoot" contains what started as Hans' branch and what both Hans and I have been working on (and are still working on) since. This will be the basis for the next release. Same goes for Chunga/Drakma.
Part 1 ("hans" being renamed to "hunchentoot") is here:
http://trac.common-lisp.net/tbnl/browser/branches
The sequel is now here:
http://bknr.net/trac/browser/trunk/thirdparty
The same three folders.
As I said, you are encouraged to check it out and play with it (and report bugs of course). Expect a couple of incompatible changes, though.
Hans Hübner wrote:
To make your life easier, I have create an ediware package that you can use to check out Hunchentoot, Drakma and all dependencies with one command:
svn co svn://bknr.net/svn/ediware
If we're going to start using public version control for 'Ediware' libraries, may I gently suggest that a distributed VCS might be a better fit for the project. Distributed version control better enables and encourages a broad base of contributors with one or a limited number of committers. It also allows fast offline operation.
The signs all seem to be pointing to GIT as the DVCS of choice for OSS projects [1]. As git hosts the linux kernel, and has experienced an explosive community growth of useful tools, it won't be going away anytime soon.
As an example, I merged the development history of previous hunchentoot releases I had tracked with the SVN histories at clnet and bknr to produce a unified history you can clone with:
git clone git://repos.tcross.org/hunchentoot.git # [2]
If we're interested in going this direction, I'll publish git conversions of Edi's other libraries.
Finally, I do understand that you're probably more personally comfortable with SVN. I believe though that the long-term benefits of moving to a DVCS now would be worth the investment in transition, especially as the project is already undergoing major changes.
Cheers,
-- Travis
For more on GIT, see Bill Clementson's piece at http://bc.tech.coop/blog/080118.html
[1] For what it's worth, my personal VCS history over the last decade has been:
CVS --> SVN --> Mercurial --> Darcs --> GIT
As git is dominating the DVCS world, git may be the last stop for awhile.
[2] This is just an example; I haven't done extensive checking yet to ensure that my conversion and merge here is flawless.
On Fri, 30 May 2008 09:24:02 +0000, Travis Cross tc@travislists.com wrote:
If we're going to start using public version control for 'Ediware' libraries, may I gently suggest that a distributed VCS might be a better fit for the project.
The short answer is no. For a longer answer please search the mailing list. This has been discussed ad nauseam before.
Edi.
Edi Weitz wrote:
On Fri, 30 May 2008 09:24:02 +0000, Travis Cross tc@travislists.com wrote:
If we're going to start using public version control for 'Ediware' libraries, may I gently suggest that a distributed VCS might be a better fit for the project.
The short answer is no. For a longer answer please search the mailing list. This has been discussed ad nauseam before.
Not a problem. :)
I don't have to search; I was probably the one who brought it up first (only once though). At that time, the answer was that there was going to be no public version control at all. One of the reasons given was related to the need for offline operation. Another was consideration of Windows support. And finally, as I recall, was the consideration that managing version control would mean more time spent maintaining the project.
So I was a bit surprised to see the introduction of a seemingly public and permanent version-controlled repository for hunchentoot that doesn't support offline operation and has a relatively high management complexity (SVN).
That's why I brought up git. Things looked to have changed. I wasn't looking to kick a dead horse. :)
Cheers,
-- Travis
On Fri, 30 May 2008 09:48:45 +0000, Travis Cross tc@travislists.com wrote:
I don't have to search; I was probably the one who brought it up first (only once though). At that time, the answer was that there was going to be no public version control at all. One of the reasons given was related to the need for offline operation. Another was consideration of Windows support. And finally, as I recall, was the consideration that managing version control would mean more time spent maintaining the project.
The reasons are still the same. Hans and I are currently joining efforts because Hans had suggested a bunch of changes (that we had talked about before) and I used his activities as an excuse to work on a redesign of Huchentoot that I had planned for some time. Some of these changes are pretty radical and they "span" several libraries at once, so it would have been tricky to release them piecemeal and we thought it might make sense to show the work in progress so people can play with it already.
Once the dust has settled a bit, I'll very likely return to private version control and regular (deemed-stable) releases for the reasons you correctly enumerated above.
Edi Weitz wrote:
As common-lisp.net has been pretty unreliable in the last days, we've moved development over to the bknr.net server maintained by Hans. If you're interested in trying out the soon-to-be-released Hunchentoot version, get all of Hunchentoot, Chunga, and FLEXI-STREAMS from here:
http://bknr.net/trac/browser/trunk/thirdparty
If you're using Drakma, you'll have to grab it from bknr.net as well, as there are incompatible changes in Chunga.
The previous development history can be seen at
http://trac.common-lisp.net/tbnl/browser/branches
whenever the server is up.
Edi.
Hi, I've been playing around with the new branch and the new CONNECTION-MANAGER etc. Being able to control threads or handling of incoming connections is nice.
But something I'm missing from HT is the ability to control the stream or connection after this. SEND-HEADERS did not seem to work as I needed because if I "collect" the stream returned from that function for usage later, and let the request-thread it originated from die it seems the old HT has been closing that stream which of course causes problems when I later wanted to use the "collected" streams.
In the new HT this doesn't seem to happen (I think?), but the stream is of a binary type. I was wondering -- how do I send text to this stream? .. and I'm also wondering whether what I'm seeing is a random glitch by ways of timing or something, and that HT in fact will close the stream for me in this version also?
I'm working on a "comet" server. Hence the somewhat unusual way of dealing with connections/streams.
Thanks,
On Tue, 10 Jun 2008 11:23:44 +0200, Lars Rune Nøstdal larsnostdal@gmail.com wrote:
But something I'm missing from HT is the ability to control the stream or connection after this. SEND-HEADERS did not seem to work as I needed because if I "collect" the stream returned from that function for usage later, and let the request-thread it originated from die it seems the old HT has been closing that stream which of course causes problems when I later wanted to use the "collected" streams.
The stream you get from SEND-HEADERS is only good for handling the current request, it is up to Hunchentoot what it does with this stream later on and it depends on what has been negotiated between the client and the server. There are situations where the server must close the stream and there are situations where the server must re-use it and read the next request from it. The server must also be able to switch chunking on and off between requests if needed.
So, if you're trying to "keep" the stream between requests, that won't work.
With respect to this, the old and the new Hunchentoot should behave the same. If they're not, that's probably an error.
In the new HT this doesn't seem to happen (I think?), but the stream is of a binary type. I was wondering -- how do I send text to this stream?
Yes, this is new. The rationale is that if you want maximum convenience and protection from errors, you just send content to Hunchentoot and let it take care of sending the right data to the client. If you request a stream, you're likely doing this for performance reasons or because you want to do something special, so it seems right to give you the "raw" binary stream. If you want to send text to this stream, wrap it with a flexi stream, see for example stream-direct-utf-8 here:
http://bknr.net/trac/browser/trunk/thirdparty/hunchentoot/test/test.lisp
.. and I'm also wondering whether what I'm seeing is a random glitch by ways of timing or something, and that HT in fact will close the stream for me in this version also?
See above. If you think there's a situation where Huchentoot should close a stream but it doesn't, that's an error. A test case would be nice.
I'm working on a "comet" server. Hence the somewhat unusual way of dealing with connections/streams.
I don't know what theat is, unfortunately,
Cheers, Edi.
Edi Weitz wrote:
On Tue, 10 Jun 2008 11:23:44 +0200, Lars Rune Nøstdal larsnostdal@gmail.com wrote:
But something I'm missing from HT is the ability to control the stream or connection after this. SEND-HEADERS did not seem to work as I needed because if I "collect" the stream returned from that function for usage later, and let the request-thread it originated from die it seems the old HT has been closing that stream which of course causes problems when I later wanted to use the "collected" streams.
The stream you get from SEND-HEADERS is only good for handling the current request, it is up to Hunchentoot what it does with this stream later on and it depends on what has been negotiated between the client and the server. There are situations where the server must close the stream and there are situations where the server must re-use it and read the next request from it. The server must also be able to switch chunking on and off between requests if needed.
So, if you're trying to "keep" the stream between requests, that won't work.
I am keeping it so I can respond to the current request - but to do so later, from a different thread. This might seem strange, I know, but it is useful here. x)
Think of it as a sort of "hanging" or waiting response on the server end, and there are many of them -- a lot, so one thread pr. client with a wait condition or something isn't going to work. The threads (connection-manager again) are to be fetched from a pool and they got to end up back in it when there isn't any more "real" work to do.
I tried to save a closure with the stream (returned from send-headers) in it and I was intending to call it later when the server has something it wants to push to that client. I'd let the thread that read the headers of the request originally "die out" opening a bit more room for other clients.
..this might be too different from what HT currently does, or is. So maybe an option for me would be to add something that dispatches to this behavior internally (based on GET/POST parameters, I think) in HT. How close are you guys to finishing work on the branch by the way? I'm guessing it would make sense for me to work vs. the branch now.
Comet is used to push data from the server to the client like this by the way. Think of stock data in real time; using JS to update the HTML elements in the DOM on the client.
With respect to this, the old and the new Hunchentoot should behave the same. If they're not, that's probably an error.
Ok.
In the new HT this doesn't seem to happen (I think?), but the stream is of a binary type. I was wondering -- how do I send text to this stream?
Yes, this is new. The rationale is that if you want maximum convenience and protection from errors, you just send content to Hunchentoot and let it take care of sending the right data to the client. If you request a stream, you're likely doing this for performance reasons or because you want to do something special, so it seems right to give you the "raw" binary stream. If you want to send text to this stream, wrap it with a flexi stream, see for example stream-direct-utf-8 here:
http://bknr.net/trac/browser/trunk/thirdparty/hunchentoot/test/test.lisp
Thanks!
On Tue, 10 Jun 2008 12:25:22 +0200, Lars Rune Nøstdal larsnostdal@gmail.com wrote:
I tried to save a closure with the stream (returned from send-headers) in it and I was intending to call it later when the server has something it wants to push to that client. I'd let the thread that read the headers of the request originally "die out" opening a bit more room for other clients.
That won't work with the current code. Right now, the thread who gives the stream to you (via SEND-HEADERS) "owns" the stream and if you kill it, you can't just keep the stream.
You'll have to invent your own connection manager class for that. The exact plumbing of what happens when when requests are handled is still in flux, but I hope the current state will give you an idea.
..this might be too different from what HT currently does, or is. So maybe an option for me would be to add something that dispatches to this behavior internally (based on GET/POST parameters, I think) in HT. How close are you guys to finishing work on the branch by the way? I'm guessing it would make sense for me to work vs. the branch now.
It would definitely make sense to do this vs. the branch. Hans is currently on vacation and I'm busy with other things, so work on the branch has stalled a bit, but there should be more activity in the next weeks. No planned release date yet.
Hi,
On 10-Jun-08, at 5:23 AM, Lars Rune Nøstdal wrote:
I'm working on a "comet" server. Hence the somewhat unusual way of dealing with connections/streams.
That sounds interesting. How is it going?
How does what you're doing compare to what has been done here:
http://groups.google.com/group/symbolicweb
Anyway, please keep us updated on this list. I'm sure there are a bunch of us who are interested.
Cheers, Bob
Bob Hutchison wrote:
Hi,
On 10-Jun-08, at 5:23 AM, Lars Rune Nøstdal wrote:
I'm working on a "comet" server. Hence the somewhat unusual way of dealing with connections/streams.
That sounds interesting. How is it going?
Hi, It's progressing well and getting it to scale better is soon on the top of things to do.
How does what you're doing compare to what has been done here:
Yeah, that's what I need this for. I've currently implemented Comet in that thing by setting the request/response thread to sleep (wait-condition), and it works great; but it will not scale because there is a limit to the amount of threads a server can have hanging around like that.
..so again that's why I need a "threadless" socket/stream from HT.. :)
Anyway, please keep us updated on this list. I'm sure there are a bunch of us who are interested.
Ok. Yeah, maybe a Comet module for HT would be of interest. I really hate promising things, but I might give it a try; I mean, attempt make it modularized and not hardwired into SymbolicWeb .... x)
On 2008-06-10, at 9:00 PM, Lars Rune Nøstdal wrote:
Ok. Yeah, maybe a Comet module for HT would be of interest. I really hate promising things, but I might give it a try; I mean, attempt make it modularized and not hardwired into SymbolicWeb .... x)
It's interesting to read about your efforts as we discussed Comet as a way to implement real time streaming to our Ajaxy mobile application yesterday. In fact I started writing a Comet module for it almost a year ago (only a naive test), but the interest vaned as the only mobile device that worked with it was the iPhone.
But I was just thinking of looking at it again as the situation on the client side may have improved since then. And, of course, the iPhone is finally coming to Norway.
Anyway, my point is that since there are two developers in Norway that have interest in Hunchentoot/Comet, there has to be hundreds if not thousands! worldwide. :-)
Bjørn Nordbø wrote:
On 2008-06-10, at 9:00 PM, Lars Rune Nøstdal wrote:
Ok. Yeah, maybe a Comet module for HT would be of interest. I really hate promising things, but I might give it a try; I mean, attempt make it modularized and not hardwired into SymbolicWeb .... x)
It's interesting to read about your efforts as we discussed Comet as a way to implement real time streaming to our Ajaxy mobile application yesterday. In fact I started writing a Comet module for it almost a year ago (only a naive test), but the interest vaned as the only mobile device that worked with it was the iPhone.
But I was just thinking of looking at it again as the situation on the client side may have improved since then. And, of course, the iPhone is finally coming to Norway.
Anyway, my point is that since there are two developers in Norway that have interest in Hunchentoot/Comet, there has to be hundreds if not thousands! worldwide. :-)
--Bjørn Nordbø
..hehe x)
Ok, way early, but something like this: http://paste.lisp.org/display/62069
It works, but it's not finished. .. I'll document it later and provide a small example (that depends on HT only). There are really just two new exported symbols; COMET-SERVER and WITH-DELAYED-RESPONSE.
Instead of using START-SERVER one have got to do:
(tbnl::start (make-instance 'tbnl:comet-server :port 6001))
..the reason for this is that it seems START-SERVER doesn't accept a custom server designator at the moment.
The other exported symbol, WITH-DELAYED-RESPONSE, works something like this from inside a normal handler function:
(with-delayed-response (stream *comet-delay-in-secs* closure (setf (comet-closure-of session) closure)) (if (comet-message-exists-p session) (princ (comet-message-of session) stream) (princ "no message - but keep on polling, mr. client!" stream)))
Then something like this:
(defun say (msg) (push msg (comet-message-of session)) (when (comet-closure-of session) (funcall (comet-closure-of session))))
..ok, now I gotta start dealing with the connections proper, and logging etc.
On Wed, 11 Jun 2008 14:35:16 +0200, Lars Rune Nøstdal larsnostdal@gmail.com wrote:
..the reason for this is that it seems START-SERVER doesn't accept a custom server designator at the moment.
This will be added before we make a release. It is not fully clear yet how the final API will look like, though.
Lars Rune Nøstdal wrote:
Bjørn Nordbø wrote:
On 2008-06-10, at 9:00 PM, Lars Rune Nøstdal wrote:
Ok. Yeah, maybe a Comet module for HT would be of interest. I really hate promising things, but I might give it a try; I mean, attempt make it modularized and not hardwired into SymbolicWeb .... x)
It's interesting to read about your efforts as we discussed Comet as a way to implement real time streaming to our Ajaxy mobile application yesterday. In fact I started writing a Comet module for it almost a year ago (only a naive test), but the interest vaned as the only mobile device that worked with it was the iPhone.
But I was just thinking of looking at it again as the situation on the client side may have improved since then. And, of course, the iPhone is finally coming to Norway.
Anyway, my point is that since there are two developers in Norway that have interest in Hunchentoot/Comet, there has to be hundreds if not thousands! worldwide. :-)
--Bjørn Nordbø
..hehe x)
Ok, way early, but something like this: http://paste.lisp.org/display/62069
..yeah, bit too early. I'll try again later, but the general idea is something like what's seen there..
..sleep now..
Lars Rune Nøstdal wrote:
Lars Rune Nøstdal wrote:
Bjørn Nordbø wrote:
On 2008-06-10, at 9:00 PM, Lars Rune Nøstdal wrote:
Ok. Yeah, maybe a Comet module for HT would be of interest. I really hate promising things, but I might give it a try; I mean, attempt make it modularized and not hardwired into SymbolicWeb .... x)
It's interesting to read about your efforts as we discussed Comet as a way to implement real time streaming to our Ajaxy mobile application yesterday. In fact I started writing a Comet module for it almost a year ago (only a naive test), but the interest vaned as the only mobile device that worked with it was the iPhone.
But I was just thinking of looking at it again as the situation on the client side may have improved since then. And, of course, the iPhone is finally coming to Norway.
Anyway, my point is that since there are two developers in Norway that have interest in Hunchentoot/Comet, there has to be hundreds if not thousands! worldwide. :-)
--Bjørn Nordbø
..hehe x)
Ok, way early, but something like this: http://paste.lisp.org/display/62069
..yeah, bit too early. I'll try again later, but the general idea is something like what's seen there..
..sleep now..
.. Ok, back again x) .. I gave it a try but I'll stick with the old thread + wait-condition method for now. It works well; the only problem is that it does not scale, so I'll work on something else for SW than scalability for a while.
There are too many changes needed to pull this off properly and I think it'll basically end up a complete rewrite of HT instead of just an addon/module for it. I can try to describe some of the problems even though many of you are probably already aware of some of them.
A major problem is global variables. I suggest that instead of having things like *REPLY* *REQUEST* *SERVER* and *HUNCHENTOOT-STREAM*, a concept of or class ROUND-TRIP is added:
(defclass round-trip () ((request) (response) (socket) (stream) (server) (session)))
..and that for, perhaps, backward compatibility things like this would do:
(define-symbol-macro *session* (session-of *round-trip*))
This would "close over" a single round-trip cycle (request->response) and maintain everything in one place so it can be passed around easily ("response is to be delayed, thus end this thread and pass it to another thread for execution later!").
Also, this means handling of a round-trip cycle cannot be hardcoded into a single function/method or into a single path of execution (a single thread). Using things like UNWIND-PROTECT makes the path of execution static; "when thread ends close the stream" (but we, well I, don't want this!).
I think things needs to be more split up for this to happen; this is hard to get just right.
I recall removal of globals mentioned on this list for the new HT, and to avoid stepping on peoples toes and wasting time I'll wait around a bit. It also seems things are moving from DEFUN to DEFMETHOD which would enable things to be more split up by the user.
On Sat, 14 Jun 2008 07:21:05 +0200, Lars Rune Nøstdal larsnostdal@gmail.com wrote:
A major problem is global variables.
As you already mentioned, this will be partly addressed by upcoming changes. Still, I'd be interested in how far this is really a "major problem" when you try to implement certain things. You aren't forced to use the special variables, are you?
Also, this means handling of a round-trip cycle cannot be hardcoded into a single function/method or into a single path of execution (a single thread). Using things like UNWIND-PROTECT makes the path of execution static; "when thread ends close the stream" (but we, well I, don't want this!).
The goal of the current development obviously is to make Hunchentoot more flexible and at the same time to give its users more rope to hang themselves. In the end, it will still be a web server, though, with an eye on simplicity and backwards compatibility, it won't become an all singing and dancing General Problem Solver. I hope you won't be disappointed.
(Just out of curiosity, are there general purpose web servers out there where one single request is not handled within one thread/process? I'm aware that Apache for example has a pretty sophisticated model of providing different ways of hooking into the request handling architecture. Still, I think, the thread that accepts the request will always be the one which is in charge of sending the reply and cleaning up, or am I wrong?)
On 14-Jun-08, at 10:04 AM, Edi Weitz wrote:
On Sat, 14 Jun 2008 07:21:05 +0200, Lars Rune Nøstdal <larsnostdal@gmail.com
wrote:
The goal of the current development obviously is to make Hunchentoot more flexible and at the same time to give its users more rope to hang themselves. In the end, it will still be a web server, though, with an eye on simplicity and backwards compatibility, it won't become an all singing and dancing General Problem Solver. I hope you won't be disappointed.
Edi, you've set the expectations :-)
(Just out of curiosity, are there general purpose web servers out there where one single request is not handled within one thread/process? I'm aware that Apache for example has a pretty sophisticated model of providing different ways of hooking into the request handling architecture. Still, I think, the thread that accepts the request will always be the one which is in charge of sending the reply and cleaning up, or am I wrong?)
My limited understanding of what they are doing is to separate socket handling from the thread that does the work (in increasingly sophisticated ways). So when a request comes a thread is assigned to it, handles the request and returns to the pool of available threads. The socket isn't necessarily closed (does HT handle keep-alive?) and is somehow named if it is kept open. If you tie this in with threads/ processes that are independent of the request-handling task, you can imagine one of these threads asking for a specific socket (or maybe a bunch of them) that is still open and shoving something down the socket/s, still not closing it/them. This also works with the new fangled event driven server stuff. Apache, jetty, tomcat, and a bunch of python, perl, and even a plugin for Rails (and <http://groups.google.com/group/symbolicweb
for lisp). There are problems with doing this using long lived HTTP
connections (e.g. firewalls), but people are doing it, so...
I didn't really take this too seriously until someone managed to comfortably handle many thousands (20000??) concurrent comet connections (the article went through what was done to achieve this number). It was nicely documented in an article, that, naturally, I can't find at the moment. I think this article might talking about the same thing, but it is certainly talking of 20000 connections, but it is mostly just reporting it:
http://cometdaily.com/2008/01/07/20000-reasons-that-comet-scales/.
This is possible useful:
<http://ajaxexperience.techtarget.com/assets/documents/Carter_Michael_Scalabl...
Cheers, Bob
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
2008/6/14 Bob Hutchison hutch-lists@recursive.ca:
...
I didn't really take this too seriously until someone managed to comfortably handle many thousands (20000??) concurrent comet connections (the article went through what was done to achieve this number). It was nicely documented in an article, that, naturally, I can't find at the moment. I think this article might talking about the same thing, but it is certainly talking of 20000 connections, but it is mostly just reporting it:
http://cometdaily.com/2008/01/07/20000-reasons-that-comet-scales/. ...
I think I've seen this; if it's the same article, it was done in Erlang, using that language's lightweight processes. As such, it's not using quite the same techniques that any Hunchentoot implementation would. Rob
On Sat, 14 Jun 2008 11:11:52 -0400, Bob Hutchison hutch-lists@recursive.ca wrote:
My limited understanding of what they are doing
With "they" you mean Apache?
does HT handle keep-alive?
Yes.
On 14-Jun-08, at 12:53 PM, Edi Weitz wrote:
On Sat, 14 Jun 2008 11:11:52 -0400, Bob Hutchison <hutch-lists@recursive.ca
wrote:
My limited understanding of what they are doing
With "they" you mean Apache?
No, I meant every server/group/person who is trying to do something like comet. That includes Apache, or possibly some groups contributing modules to Apache, or both.
does HT handle keep-alive?
Yes.
I'm not sure what all Lars needs here, but it at a first pass, if you keep those sockets alive and name them and so allowing some independent thread or event handler to find the socket and use it again... Well, as I said, I don't know what Lars needs, but I think I could do some fun stuff with that capability.
Cheers, Bob
tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
Bob Hutchison wrote:
On 14-Jun-08, at 10:04 AM, Edi Weitz wrote:
On Sat, 14 Jun 2008 07:21:05 +0200, Lars Rune Nøstdal larsnostdal@gmail.com wrote:
The goal of the current development obviously is to make Hunchentoot more flexible and at the same time to give its users more rope to hang themselves. In the end, it will still be a web server, though, with an eye on simplicity and backwards compatibility, it won't become an all singing and dancing General Problem Solver. I hope you won't be disappointed.
Edi, you've set the expectations :-)
(Just out of curiosity, are there general purpose web servers out there where one single request is not handled within one thread/process? I'm aware that Apache for example has a pretty sophisticated model of providing different ways of hooking into the request handling architecture. Still, I think, the thread that accepts the request will always be the one which is in charge of sending the reply and cleaning up, or am I wrong?)
My limited understanding of what they are doing is to separate socket handling from the thread that does the work (in increasingly sophisticated ways). So when a request comes a thread is assigned to it, handles the request and returns to the pool of available threads. The socket isn't necessarily closed (does HT handle keep-alive?) and is somehow named if it is kept open. If you tie this in with threads/processes that are independent of the request-handling task, you can imagine one of these threads asking for a specific socket (or maybe a bunch of them) that is still open and shoving something down the socket/s, still not closing it/them. This also works with the new fangled event driven server stuff. Apache, jetty, tomcat, and a bunch of python, perl, and even a plugin for Rails (and http://groups.google.com/group/symbolicweb for lisp). There are problems with doing this using long lived HTTP connections (e.g. firewalls), but people are doing it, so...
Just a quick note that reducing the timeout of the comet-response (the empty one) works in these cases. I've confirmed this with more than one corporate firewall to the surprise of people expecting any networked application in similar style of something SW'ish to fail. :)
... I might add some heuristics that does some automatic adjusting and detecting on "boot-up time" for the comet timeout later ...
Edi Weitz wrote:
On Sat, 14 Jun 2008 07:21:05 +0200, Lars Rune Nøstdal larsnostdal@gmail.com wrote:
A major problem is global variables.
As you already mentioned, this will be partly addressed by upcoming changes. Still, I'd be interested in how far this is really a "major problem" when you try to implement certain things. You aren't forced to use the special variables, are you?
No, I guess not -- maybe not. It starts with a socket/stream at the connection-manager end, and functions seem to accept &optional (reply *reply*) etc. .. so I can build on that from the lowest levels and up, I think.
I'll give this a new try a bit later and go about it a bit slower and build something from the ground up instead. I figured I could hack this together quickly with the parts that where there already by moving them around and splitting them up a bit, but maybe that's not such a good idea currently. It just gets more complicated that way I think.
..I should wait a bit and see what happens though..
I still have a feeling the global variables should be removed or, that is, gathered together into a "single blob" so the whole concept of a request-response cycle (round-trip) can be passed around easily if the user really needs to do this.
I know global vars aren't "dangerous" in Lisp like in other languages when they are bound (LET) within a single thread where the execution path is predefined or stays within a single thread; that's not what I mean =) .. but in this case the bindings or data needs to be accessed from multiple threads (or well, moved to a new thread) as the cycle "walks forward".
But, OK, never mind .. globals vars are being addressed - and/or I can just not used them for my stuff; right. It will work out.
Also, this means handling of a round-trip cycle cannot be hardcoded into a single function/method or into a single path of execution (a single thread). Using things like UNWIND-PROTECT makes the path of execution static; "when thread ends close the stream" (but we, well I, don't want this!).
The goal of the current development obviously is to make Hunchentoot more flexible and at the same time to give its users more rope to hang themselves. In the end, it will still be a web server, though, with an eye on simplicity and backwards compatibility, it won't become an all singing and dancing General Problem Solver. I hope you won't be disappointed.
Yeah, I don't think I'll be disappointed but it would be great not having to write huge parts from scratch like mentioned above even though it would actually be possible to do so.
If things are split up and work more on their own not depending on different context bits (global variables, let-bindings etc.) at the same time this will enable code-bits to be shared between multiple server backends, modules or connection-managers, I think/hope.
(Just out of curiosity, are there general purpose web servers out there where one single request is not handled within one thread/process? I'm aware that Apache for example has a pretty sophisticated model of providing different ways of hooking into the request handling architecture. Still, I think, the thread that accepts the request will always be the one which is in charge of sending the reply and cleaning up, or am I wrong?)
The thread cannot wait around for the server to have anything to send as a response; it will not scale in any Lisp I know of.
-
Andrei Stebakov -
Bjørn Nordbø -
Bob Hutchison -
Cyrus Harmon -
Edi Weitz -
Hans Hübner -
Lars Rune Nøstdal -
Robert Synnott -
Travis Cross