On Sun, 28 Sep 2008 15:28:07 -0500, "Matthew D. Swank" akopa.gmane.poster@gmail.com wrote:
Well the regexes are defined in the lexers in this file: http://common-lisp.net/~mswank/apache-ppcre.lisp
The lexer api is in this file: http://common-lisp.net/~mswank/cl-ppcre-lexer.lisp
Finally, the log file I'm lexing: http://lcpug.asternix.com/pub/Main/ApacheLogProject/access.log
Compare (with-open-file (in "access.log") (let ((foo (stream-gen *apache-pcrelex-line* in))) (time (loop :for x := (funcall foo) :unless x :return nil))))
with
(with-open-file (in "access.log") (let ((foo (stream-gen *apache-pcrelex* in))) (time (loop :for x := (funcall foo) :unless x :return nil))))
When I slurp the entire file into a string the matches seem to be taking about a tenth of a second for each token.
Sorry, I don't have the time to read the entire application right now. Can you boil this down to a single application of PPCRE:SCAN which is too slow?
Thanks, Edi.