Hi,
Yesterday I completed an abcl-dev blogpost about 0.18 which handles out of memory (OOM) conditions much better than earlier versions. However, as the post concludes, it would be even lispier to have a low-memory signal instead of OOM, because in that case the condition system can be used to handle the situation right in the call stack where the condition occurred, in order to be able to continue execution after handling the condition.
Douglas Miles suggested we use JMX memory management notifications to create this behaviour. Before we can implement such behaviour, a number of questions need to be answered:
* How does an application tell the difference between our current OOM condition and the new behaviour? -> this is important, because the current behaviour unwinds the stack; the future one won't * Do we remove the current behaviour? * Do we let the user select the parameters for signalling the requested condition? * Should we let the user install handlers to be called when a condition like this occurs? * <others>?
Regards,
Erik.
On Sat, Apr 3, 2010 at 5:17 PM, Erik Huelsmann ehuels@gmail.com wrote:
Hi,
Yesterday I completed an abcl-dev blogpost about 0.18 which handles out of memory (OOM) conditions much better than earlier versions. However, as the post concludes, it would be even lispier to have a low-memory signal instead of OOM, because in that case the condition system can be used to handle the situation right in the call stack where the condition occurred, in order to be able to continue execution after handling the condition.
Douglas Miles suggested we use JMX memory management notifications to create this behaviour. Before we can implement such behaviour, a number of questions need to be answered:
Just my .02 €:
* How does an application tell the difference between our current OOM condition and the new behaviour? -> this is important, because the current behaviour unwinds the stack; the future one won't
- the new behaviour should not be an ERROR but a WARNING, in my opinion. You can react to it or ignore it. After all, low-memory is not the same as no-more-memory.
* Do we remove the current behaviour?
- no! Having the new behaviour does not make OOM impossible to occur, and we still need to react to that in a meaningful manner.
* Do we let the user select the parameters for signalling the requested condition?
- If possible, I'd give the user the possibility to select those parameters.
* Should we let the user install handlers to be called when a condition like this occurs?
- yes, why not? It's a condition like any other.
* <others>?
- JMX is fully available out of the box from Java 1.6. In 1.5 you have to launch the VM with some properties set to enable it. Since low-memory signaling is a kind of help to the programmer, but it's not mandatory, we could simply warn if JMX is not enabled and never signal any low-memory condition in that case.
Cheers, Alessio
On Apr 8, 2010, at 11:25, Alessio Stalla wrote:
- the new behaviour should not be an ERROR but a WARNING, in my
opinion. You can react to it or ignore it. After all, low-memory is not the same as no-more-memory.
Actions to be taken in low memory can also be (might particularly be) unrelated to what's currently on the stack (dynamic scope), such as clearing caches. On the other hand, condition handlers are always dynamically scoped. It might be good to have a way to register global handlers for the low memory warning.
On the other hand, what thread is a global handler called in, when? For that matter, when might a condition be signaled -- what if it's the result of an allocation in the middle of a subsystem in an inconsistent state? OOM always has this problem, but in the low memory case it's more feasible to recover well (since you have some room to calculate your cleanup), so you need to think more about not accidentally breaking invariants.
On Thu, Apr 8, 2010 at 5:51 PM, Kevin Reid kpreid@switchb.org wrote:
On Apr 8, 2010, at 11:25, Alessio Stalla wrote:
- the new behaviour should not be an ERROR but a WARNING, in my
opinion. You can react to it or ignore it. After all, low-memory is not the same as no-more-memory.
Actions to be taken in low memory can also be (might particularly be) unrelated to what's currently on the stack (dynamic scope), such as clearing caches. On the other hand, condition handlers are always dynamically scoped. It might be good to have a way to register global handlers for the low memory warning.
On the other hand, what thread is a global handler called in, when? For that matter, when might a condition be signaled -- what if it's the result of an allocation in the middle of a subsystem in an inconsistent state? OOM always has this problem, but in the low memory case it's more feasible to recover well (since you have some room to calculate your cleanup), so you need to think more about not accidentally breaking invariants.
The issues associated with threading are indeed subtle; I hadn't considered them. OOM being an exception means that it will be handled in the thread(s) that raised it. With JMX, instead, you either poll the memory MXBean from a dedicated thread, or register a listener that will be automatically invoked when the threshold is exceeded; see [1]. The thread that gets to call the listener seems to be a dedicated one ([1] says not to do lengthy operations in the listener to avoid blocking further notifications). So, in either case, a global handler is more appropriate.
Regards, Alessio
[1] http://download.java.net/jdk7/docs/technotes/guides/management/mxbeans.html (search for "Detecting Low Memory")
-- Kevin Reid http://switchb.org/kpreid/
armedbear-devel mailing list armedbear-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
armedbear-devel@common-lisp.net
-
Alessio Stalla -
Erik Huelsmann -
Kevin Reid