Re: Parsing big XML files with klacks and sbcl

On Tue, May 29, 2018 at 9:20 PM, Attila Lendvai attila@lendvai.name wrote:

...

(loop while t do (klacks:consume *src*))

try to add (sb-ext:gc :full t) inside the loop. if that helps, then you're overwhelming SBCL's gc algorithm by allocating too much garbage between two gc's (or something along that line, maybe someone else with more knowledge of the details can elaborate).

This indeed keeps the memory usage in check. However a forced gc on every loop sounds less than ideal. I am a bit surprised that a streaming parser generates so much garbage, considering on of the main use cases is handling large files. Also I am wondering if the GC can be configured to run more aggressively without further explicit calls in the rest of the code.