July 2005 - cl-ppcre-devel@common-lisp.net

[cl-ppcre-devel] New release 1.2.10
by Edi Weitz 19 Jul '05

19 Jul '05

Changelog: Version 1.2.10 2005-07-20 Fixed bug in CHAR-SEARCHER-AUX (caught by Peter Schuller) Don't redefine what's already there (for LispWorks) Download: <http://weitz.de/files/cl-ppcre.tar.gz> Cheers, Edi.

1 0

Re: [cl-ppcre-devel] Byte vectors instead of strings
by pete-cl-ppcre＠kazmier.com 17 Jul '05

17 Jul '05

On Sun, Jul 17, 2005 at 06:36:48PM -0600, Jim Prewett wrote: > > i'm just cureous, what sort of log processing are you doing? I'm responsible for all of the network management systems for a VoIP telecom company. Part of our architecture is the real-time monitoring of various logs such as the syslog messages generated by about 1000+ Cisco devices as well as various application log files. Currently, I use my own Python software called LogWrap[1] for this purpose. Another part of our architecture is the post processing of log files for trend analysis, intrusion detection analysis, etc ... This analysis is done with a whole bunch of Python scripts. Over the past year, I've been learning CL in my free time and have been trying to slowly introduce CL at work in both of the above areas. My first attempt was to write some of the post processing tools in CL because I thought that CL coupled with cl-ppcre would be much faster than my existing Python tools. This was not the case because the open-source CL implementations were slow due to the IO processing. Now I am now trying to use byte vectors with cl-ppcre to see if this will significantly speed up the processing. > I've been working on a "generic log analysis" application for a couple of > years now that (very) strongly suggests CL-PPCRE called LoGS. I'm reading about it now and it sounds very interesting and familiar as my Python LogWrap does some of the same, rules, actions, suppression, generic handlers, etc. Its time for me to go to bed now, but I will read more about this tomorrow as it may help me with the real-time part of my architecture. I was going to write LogWrap in Lisp, but it sounds like you've saved me the trouble. > Is there a collaboration here? Perhaps, but it seems at first glance that LoGS has everything I need. After I go through the documentation more thoroughly, I'll be able to determine if there are any missing pieces of functionality I might want to add and contribute if wanted. Thanks, Pete [1] http://www.kazmier.com/computer/logwrap

2 1

[cl-ppcre-devel] Byte vectors instead of strings
by pete-cl-ppcre＠kazmier.com 17 Jul '05

17 Jul '05

Hi Edi, How hard would it be to modify cl-ppcre to work on byte vectors instead of strings? I'm trying to obtain faster performance when parsing large log files. Most of the time spent processing the logs is wasted on the creation of strings. I want to use read-sequence with unsigned-byte as the external format to avoid that processing. Of course, this means I need a regexp library that can handle byte vectors. As a newbie, is it even worth hacking cl-ppcre to use byte vectors or is the difficulty level too high? I am also considering learning FFI and just making an interface to a standard C regexp library which will work with bytes. However, if I can use cl-ppcre, I'd prefer as its written in CL. Thanks, Pete

2 3