March 2004 - cl-ppcre-devel@common-lisp.net

RE: [cl-ppcre-devel] Problems with a regexp using CL-PPCRE 0.7.4
by Kick Damien-DKICK1 17 Mar '04

17 Mar '04

Sorry I've been away for a bit... between my sister-in-law's wedding last weekend, an aikido seminar <http://www.barnswood.com/arlaiki/> this coming weekend, and work, I haven't had any time to spare for CL-PPCRE. I probably won't be able to get to this until sometime next week/weekend but I haven't forgotten about it.

1 0

RE: [cl-ppcre-devel] Problems with a regexp using CL-PPCRE 0.7.4
by Kick Damien-DKICK1 12 Mar '04

12 Mar '04

Edi Weitz [edi(a)agharta.de] wrote: > 1. Maybe this is an error that can only be seen on Sun OS. I don't > have a Sun to test so you might want to use an x86 machine as > your client if possible and see if it exposes the same problem. That will take a bit longer but I think I should be able to find a GNU/Linux box on an x86. I'll have to start from installing CMUCL, etc. I'll let you know how things go... > Well, not exactly the same error, but close. Maybe you could try > again with CL-PPCRE:*USE-BMH-MATCHERS* set to T and TRACE the > CL-PPCRE::CREATE-BMH-MATCHER function to see which arguments it > gets? * (in-package #:break-cl-ppcre) #<The BREAK-CL-PPCRE package, 30/44 internal, 0/2 external> * (setf *test-expect-login* (setf *test-expect-password* #| ... |#)) #| ... |# * cl-ppcre:*use-bmh-matchers* T * (trace CL-PPCRE::CREATE-BMH-MATCHER) (CL-PPCRE::CREATE-BMH-MATCHER) * (test-telnet #| ... |#) 0: (CL-PPCRE::CREATE-BMH-MATCHER "login:" NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40A67039}> Trying #| ... |#... Connected to #| ... |#. Escape character is '^]'. SunOS 5.6 login: 0: (CL-PPCRE::CREATE-BMH-MATCHER "assword:" NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40AE4689}> #| ... |# Password: 0: (CL-PPCRE::CREATE-BMH-MATCHER "tekelec:[" NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40AE82C9}> Last login: Thu Mar 11 21:20:26 from 10.17.193.24 Sun Microsystems Inc. SunOS 5.6 Generic August 1997 tekelec:[/tekelec/users/#| ... |#] 1 % 0: (CL-PPCRE::CREATE-BMH-MATCHER "tekelec:[" NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40AEF2E9}> ls SEdisplaylJ_1N_ log mgts_cit_csh auto.sh mgts.Xdefaults mgts_cit_env auto_datafiles.tar mgts.cshrc mgts_gsr6.tar datafile.6.0.1.0.4.tar mgts.login mgts_run datafiles mgts.profile mgts_umt_csh datafiles.bak mgts.xinitrc set_mgts_env install.errors mgts.xsession tekelec:[/tekelec/users/#| ... |#] 2 % T * (test-telnet #| ... |# :prompt +default-mgts-server-prompt--break-cl-ppcre+) 0: (CL-PPCRE::CREATE-BMH-MATCHER "login:" NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40B15FF9}> Trying #| ... |#... Connected to #| ... |#. Escape character is '^]'. SunOS 5.6 login: 0: (CL-PPCRE::CREATE-BMH-MATCHER "assword:" NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40B1A079}> #| ... |# Password: 0: (CL-PPCRE::CREATE-BMH-MATCHER " % " NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40B1CCB1}> 0: (CL-PPCRE::CREATE-BMH-MATCHER " tekelec:[" NIL) 0: CL-PPCRE::CREATE-BMH-MATCHER returned #<Closure Over Function "DEFUN CREATE-BMH-MATCHER" {40B1ECE1}> Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER: -1 is not of type (MOD 536870911) Restarts: 0: [ABORT] Return to Top-Level. Debug (type H for help) ("DEFUN CREATE-BMH-MATCHER" -2) Source: ; File: /usr/vob/2gtt/lti/cl-ppcre-0.7.4/scanner.lisp (AREF SKIP (CHAR-CODE (SCHAR PATTERN K))) 0]

2 1

RE: [cl-ppcre-devel] Problems with a regexp using CL-PPCRE 0.7.4
by Kick Damien-DKICK1 11 Mar '04

11 Mar '04

Well, if all of the optimizations are still on, I don't see the SIGSEGV on the first try. It doesn't come until later, after trying a run a few times. If I recompile CL-PPCRE 0.7.4 with all the optimizations off, the type checking kicks into ERROR on the first try. -----Original Message----- From: cl-ppcre-devel-bounces(a)common-lisp.net [mailto:cl-ppcre-devel-bounces@common-lisp.net] On Behalf Of Edi Weitz Sent: Thursday, 11 March, 2004 16:22 To: Kick Damien-DKICK1 Cc: cl-ppcre-devel(a)common-lisp.net Subject: Re: [cl-ppcre-devel] Problems with a regexp using CL-PPCRE 0.7.4 So I've tried to reproduce the error you get and I didn't succeed. I've temporarily enabled telnet on a local Linux machine and set the bash PS1 prompt to be 'tekelec:[\w] \! % ' to be as close to your example as possible. The whole transcript is below with only the password removed. I had to change "Login" to "ogin" and "ls" to "ls --color=never" but everything else is unchanged. CL-PPCRE is 0.7.4 compiled without modifications, CMUCL is 18e on a Linux (SuSE 9.0) system. Hmmm..... Edi. edi@bird:/tmp > cat expect.lisp (defpackage #:break-cl-ppcre (:use #:common-lisp #:extensions #:cl-ppcre)) (in-package #:break-cl-ppcre) ;; I believe that the following regular expression is causing CL-PPCRE ;; to choke. (defconstant +default-mgts-server-prompt--break-cl-ppcre+ `(:sequence #\Newline "tekelec:[" (:greedy-repetition 0 nil :everything) "] " (:greedy-repetition 1 nil :digit-class) " % " :end-anchor)) (defvar *test-expect-login* nil) (defvar *test-expect-password* nil) (declaim (inline string-cat)) (defun string-cat (&rest args) (apply #'concatenate 'string args)) (defmacro with-default-spawn ((default-spawn) &body code) `(flet ((expect (expected &optional (spawn ,default-spawn) &key (echo *standard-output*)) (expect expected spawn :echo echo)) (send (message &optional (spawn ,default-spawn)) (send message spawn))) (macrolet ((send1 (&rest message-parts) `(send* ,',default-spawn ,@message-parts))) ,@code))) (defmacro with-spawn-process ((id exec-name &optional exec-args &key without-default-spawn) &body code) (let ((exec-args-value (gensym "EXEC-ARGS-VALUE-"))) `(let* ((,exec-args-value ,exec-args) (,id (spawn ,exec-name ,exec-args-value))) (unwind-protect ,(if without-default-spawn `(progn ,@code) `(with-default-spawn (,id) ,@code)) (process-close ,id))))) (defmacro with-spawn-stream ((stream exec-name &optional exec-args &key without-default-spawn) &body code) (let ((exec-args-value (gensym "EXEC-ARGS-VALUE-")) (id (gensym "SPAWN-PROCESS-"))) `(let ((,exec-args-value ,exec-args)) (with-spawn-process (,id ,exec-name ,exec-args-value :without-default-spawn t) (let ((,stream (process-pty ,id))) ,(if without-default-spawn `(progn ,@code) `(with-default-spawn (,stream) ,@code))))))) (defgeneric expect (expected spawn &key echo) (:documentation "ARGS: EXPECTED SPAWN &KEY ECHO This is a CMU CL version of Don Libes' expect. EXPECTED is what one expects to find on SPAWN, created by the function SPAWN.")) (defgeneric send (message spawn) (:documentation "ARGS: MESSAGE SPAWN A CMU CL version of Don Libe's send. Send MESSAGE to SPAWN, created by the function SPAWN.")) (defun send* (spawn &rest messages) (send (apply #'concatenate 'string (mapcar #'string messages)) spawn)) (defun spawn (program &optional args) "ARGS: PROGRAM &OPTIONAL ARGS A CMU CL version of Don Libes' spawn. PROGRAM is the name of the program to be exec'd in a pseudo-terminal." (run-program program args :wait nil :pty t :input t :output t :error t)) (defmethod expect ((expected string) (spawn extensions::process) &key (echo *standard-output*) regexp case-insensitive-mode multi-line-mode single-line-mode extended-mode) (expect (create-scanner (if regexp expected (quote-meta-chars expected)) :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode) (process-pty spawn) :echo echo)) (defmethod expect ((expected string) (spawn stream) &key (echo *standard-output*) regexp case-insensitive-mode multi-line-mode single-line-mode extended-mode) (expect (create-scanner (if regexp expected (quote-meta-chars expected)) :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode) spawn :echo echo)) ;; expected is a parse-tree (defmethod expect ((expected t) (spawn extensions::process) &key (echo *standard-output*) case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive) (expect (create-scanner expected :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode :destructive destructive) (process-pty spawn) :echo echo)) ;; expected is a parse-tree (defmethod expect ((expected t) (spawn stream) &key (echo *standard-output*) case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive) (expect (create-scanner expected :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode :destructive destructive) spawn :echo echo)) ;; expected is a scanner (defmethod expect ((expected function) (spawn extensions::process) &key (echo *standard-output*)) (expect expected (process-pty spawn) :echo echo)) ;; expected is a scanner (defmethod expect ((expected function) (spawn stream) &key (echo *standard-output*)) (let ((buffer (make-array '(0) :element-type 'base-char :fill-pointer 0 :adjustable t))) (with-output-to-string (match buffer) (let ((io (make-echo-stream spawn (if echo (make-broadcast-stream match echo) match)))) ;; I know that this is going to be a horribly inefficient ;; algorithm; i.e. reading a single character at a time ;; and re-scanning the BUFFER every time a new character ;; is added. I'll work on fixing this later. For know, I ;; just want to get something working. -- Damien Kick (loop (read-char io) (multiple-value-bind (match-start match-end reg-starts reg-ends) (scan expected buffer) (when match-start (return (values buffer match-start match-end reg-starts reg-ends))))))))) (defmethod send ((message string) (spawn extensions::process)) (send message (process-pty spawn))) (defmethod send ((message string) (spawn stream)) (write-string message spawn) (force-output spawn) message) (defun test-telnet (address &key (login *test-expect-login*) (password *test-expect-password*) (prompt '(:sequence "tekelec:[" (:greedy-repetition 0 nil :everything) #\] (:greedy-repetition 1 nil #\Space) (:greedy-repetition 1 nil (:char-class (:range #\0 #\9))) (:greedy-repetition 1 nil #\Space) #\%))) (with-spawn-stream (stream "telnet" (list address)) (expect "ogin:") (send (string-cat (string login) (string #\Newline))) (expect "assword:") (send (string-cat (string password) (string #\Newline))) (expect prompt) (send (string-cat "ls --color=never" (string #\Newline))) (expect prompt) t)) edi@bird:/tmp > cmucl ; Loading #p"/home/edi/.cmucl-init". ; loading system definition from /usr/local/lisp/Registry/asdf-install.asd into ; #<The ASDF1015 package, 0/9 internal, 0/9 external> ; registering #<SYSTEM ASDF-INSTALL {48513F25}> as ASDF-INSTALL CMU Common Lisp 18e, running on bird With core: /usr/local/lib/cmucl/lib/lisp.core Dumped on: Thu, 2003-04-03 15:47:12+02:00 on orion See <http://www.cons.org/cmucl/> for support information. Loaded subsystems: Python 1.1, target Intel x86 CLOS 18e (based on PCL September 16 92 PCL (f)) * (mk:load-system :cl-ppcre) ; Loading #p"/usr/local/lisp/source/cl-ppcre/packages.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/specials.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/util.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/errors.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/lexer.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/parser.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/regex-class.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/convert.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/optimize.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/closures.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/repetition-closures.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/scanner.x86f". ; Loading #p"/usr/local/lisp/source/cl-ppcre/api.x86f". (#<FILE: api> #<FILE: scanner> #<FILE: repetition-closures> #<FILE: closures> #<FILE: optimize> #<FILE: convert> #<FILE: regex-class> #<FILE: parser> #<FILE: lexer> #<FILE: errors> #<FILE: util> #<FILE: specials> #<FILE: packages>) * (load (compile-file "expect.lisp")) ; Python version 1.1, VM version Intel x86 on 11 MAR 04 11:11:39 pm. ; Compiling: /tmp/expect.lisp 11 MAR 04 11:09:42 pm ; Byte Compiling Top-Level Form: ; Converted STRING-CAT. ; Compiling DEFUN STRING-CAT: ; Converted WITH-DEFAULT-SPAWN. ; Compiling DEFMACRO WITH-DEFAULT-SPAWN: ; Converted WITH-SPAWN-PROCESS. ; Compiling DEFMACRO WITH-SPAWN-PROCESS: ; Converted WITH-SPAWN-STREAM. ; Compiling DEFMACRO WITH-SPAWN-STREAM: ; Byte Compiling Top-Level Form: ; Converted SEND*. ; Compiling DEFUN SEND*: ; Converted SPAWN. ; Compiling DEFUN SPAWN: ; Compiling DEFMETHOD EXPECT (STRING EXTENSIONS::PROCESS): ; Compiling DEFMETHOD EXPECT (STRING STREAM): ; Compiling DEFMETHOD EXPECT (T EXTENSIONS::PROCESS): ; Byte Compiling Top-Level Form: ; Compiling DEFMETHOD EXPECT (T STREAM): ; Compiling DEFMETHOD EXPECT (FUNCTION EXTENSIONS::PROCESS): ; Compiling DEFMETHOD EXPECT (FUNCTION STREAM): ; Compiling DEFMETHOD SEND (STRING EXTENSIONS::PROCESS): ; Compiling DEFMETHOD SEND (STRING STREAM): ; Converted TEST-TELNET. ; Compiling DEFUN TEST-TELNET: ; Byte Compiling Top-Level Form: ; expect.x86f written. ; Compilation finished in 0:00:00. T * (in-package :break-cl-ppcre) #<The BREAK-CL-PPCRE package, 44/44 internal, 0/2 external> * (setf *test-expect-login* "edi") "edi" * (setf *test-expect-password* "********") "********" * (test-telnet "miles") Trying 62.159.208.81... Connected to miles. Escape character is '^]'. Welcome to SuSE Linux 7.1 (i386) - Kernel 2.4.0-4GB (5). miles login: edi Password: Last login: Thu Mar 11 23:09:28 from trane.agharta.de Have a lot of fun... tekelec:[/tmp/foo] 501 % ls --color=never 08_zappa.mp3 genera-handbook.pdf fpga-technology_charts.pdf my-exchange-file-changed-functions.el tekelec:[/tmp/foo] 502 % T * (test-telnet "miles" :prompt +default-mgts-server-prompt--break-cl-ppcre+) Trying 62.159.208.81... Connected to miles. Escape character is '^]'. Welcome to SuSE Linux 7.1 (i386) - Kernel 2.4.0-4GB (5). miles login: edi Password: Last login: Thu Mar 11 23:12:44 from trane.agharta.de Have a lot of fun... tekelec:[/tmp/foo] 501 % ls --color=never 08_zappa.mp3 genera-handbook.pdf fpga-technology_charts.pdf my-exchange-file-changed-functions.el tekelec:[/tmp/foo] 501 % T * (quit) edi@bird:/tmp > _______________________________________________ cl-ppcre-devel site list cl-ppcre-devel(a)common-lisp.net http://common-lisp.net/mailman/listinfo/cl-ppcre-devel

2 1

RE: [cl-ppcre-devel] Problems with a regexp using CL-PPCRE 0.7.4
by Kick Damien-DKICK1 11 Mar '04

11 Mar '04

Edi Weitz [edi(a)agharta.de] wrote: > 2. If that doesn't help you might want to try the whole thing again > with CL-PPCRE:*USE-BMH-MATCHERS* set to NIL globally. Do you > still get an error? Is it different? I just tried and I got the same error. Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER: -2 is not of type (MOD 536870911) Restarts: 0: [ABORT] Return to Top-Level. Debug (type H for help) (SEARCH " % " " " :FROM-END NIL ...) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/seq.lisp. 0] cl-ppcre:*use-bmh-matchers* NIL 0] (break) Break Restarts: 0: [CONTINUE] Return from BREAK. 1: [ABORT ] Return to debug level 1. 2: Return to Top-Level. Debug (type H for help) (DEBUG::DEBUG-EVAL-PRINT (BREAK)) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/debug.lisp. 0]] backtrace 0: (DEBUG::DEBUG-EVAL-PRINT (BREAK)) 1: (DEBUG::DEBUG-LOOP) 2: (DEBUG:INTERNAL-DEBUG) 3: (DEBUG::INVOKE-TTY-DEBUGGER #<TYPE-ERROR {8028D8FD}>) 4: (DEBUG::REAL-INVOKE-DEBUGGER #<TYPE-ERROR {8028D8FD}>) 5: (INVOKE-DEBUGGER #<TYPE-ERROR {8028D8FD}>) 6: (ERROR TYPE-ERROR :FUNCTION-NAME SEARCH :DATUM ...) 7: (KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER SEARCH #.(SYSTEM:INT-SAP #x38000308) #<Alien (* #) at #xFFBEDE78> (269 526)) 8: (KERNEL::INTERNAL-ERROR #.(SYSTEM:INT-SAP #xFFBEDE78) #<unused-arg>) 9: ("Foreign function call land") 10: (SEARCH " % " " " :FROM-END NIL ...) 11: ("DEFUN CREATE-SCANNER-AUX" #<lambda-list-unavailable>) 12: ("DEFMETHOD EXPECT (FUNCTION STREAM)" #<unused-arg> #<unused-arg> #<Closure Over Function "DEFUN CREATE-SCANNER-AUX" {8028D451}> #<Stream for descriptor 9> ...) 13: (TEST-TELNET-ARGSMMGTS5--CASE-2 "argsmmgts5" :LOGIN NIL :PASSWORD ...) 14: (INTERACTIVE-EVAL (TEST-TELNET-ARGSMMGTS5--CASE-2 "argsmmgts5" :PROMPT +DEFAULT-MGTS-SERVER-PROMPT--BREAK-CL-PPCRE+)) 15: (COMMON-LISP::%TOP-LEVEL) 16: (COMMON-LISP::RESTART-LISP) 0]]

2 2

RE: [cl-ppcre-devel] Problems with a regexp using CL-PPCRE 0.7.4
by Kick Damien-DKICK1 11 Mar '04

11 Mar '04

Edi Weitz [edi(a)agharta.de] wrote: Thanks for the reply. I'll try the things you mentioned but I just wanted to get to one thing, first. > Some ideas: > > 1. Your methods for the generic function EXPECT don't have congruent > lambda lists: > > > ;; expected is a parse-tree > > (defmethod expect ((expected t) (spawn stream) > > &key (echo *standard-output*) > > case-insensitive-mode > > multi-line-mode > > single-line-mode > > extended-mode > > destructive) > > #| ... |#) > > > > ;; expected is a scanner > > (defmethod expect ((expected function) (spawn extensions::process) > > &key (echo *standard-output*)) > > #| ... |#) > > This can lead to incosistent behaviour of all sorts, so you should > change that first and see if it makes a difference. When you write that the lambda lists are not congruent, I assume you mean that they have different keywords and are, therefore, not congruent. Consider the CLHS section 7.6.4 <http://www.lispworks.com/reference/HyperSpec/Body/07_fd.htm>: 4. If the generic function lambda list mentions &key, each method must accept all of the keyword names mentioned after &key, either by accepting them explicitly, by specifying &allow-other-keys, or by specifying &rest but not &key. Each method can accept additional keyword arguments of its own. [...] I believe that that all of the EXPECT methods are congruent by that definition. Or did you have something other than keyword argument equivalence in mind whne you wrote about the lambda lists not being congruent?

2 1

[cl-ppcre-devel] Problems with a regexp using CL-PPCRE 0.7.4
by Kick Damien-DKICK1 11 Mar '04

11 Mar '04

I have been using CMUCL to reimplement some functionality that one would find in Don Libes' Expect <http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=ovr7wx4…>, using CL-PPCRE for the regular expression engine. Thank you all for providing this excellent library to the CL community. I've especially enjoyed being able to use a sexp notation for regexes. However, I think I've found a regexp that breaks CL-PPCRE. Perhaps I've given CL-PPCRE an ill formed regexp or corrupted the Lisp image with some bad code? I've tried to par down the code that caused my original problem, though I'm afraid it is not nearly as concise as it could be. Please feel free to yell at me if I should spend more time trimming the irrelevant code out of the example and I'll post a leaner version. Using CL-PPCRE 0.7.4 "out of the box", I was seeing my CMUCL process handling a SIGSEGV. Because of this, I recompiled the CL-PPCRE code after removing all of the "(declare (optimize speed ...))" statements. The following is an edited transcript (to remove machine address, logins (urk! not a word), and passwords) of the results of using the unoptimized CL-PPCRE (again, using the optimized CL-PPCRE was giving me a SIGSEGV at the same point of error). Please let me know what you think; i.e. is this a CL-PPCRE bug or have I done something wrong? % cmucl ; Loading #p".../.cmucl-init.sparcf". ;; Loading #p".../src/clocc-20040206/clocc.sparcf". ;; Loading #p".../src/clocc-20040206/src/defsystem-3.x/defsystem.sparcf". CMU Common Lisp 18e, running on gsdapp04 With core: .../sparc-sun-solaris2.6/lib/cmucl/lib/lisp.core Dumped on: Tue, 2003-04-08 13:23:10-05:00 on achat See <http://www.cons.org/cmucl/> for support information. Loaded subsystems: Python 1.1, target SPARCstation/Solaris 2 CLOS 18e (based on PCL September 16 92 PCL (f)) * (load "break-cl-ppcre") ; Loading #p".../lti/break-cl-ppcre.sparcf". Type-error in COMMON-LISP::PACKAGE-OR-LOSE: "CL-PPCRE" is not of type PACKAGE Restarts: 0: [CONTINUE] Make this package. 1: Return NIL from load of "break-cl-ppcre". 2: [ABORT ] Return to Top-Level. Debug (type H for help) (COMMON-LISP::PACKAGE-OR-LOSE "CL-PPCRE") Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/package.lisp. 0] 0 T * (load "cl-ppcre-0.7.4/load") ; Loading #p".../lti/cl-ppcre-0.7.4/load.lisp". ;; Loading #p".../lti/cl-ppcre-0.7.4/packages.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/specials.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/util.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/errors.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/lexer.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/parser.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/regex-class.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/convert.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/optimize.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/closures.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/repetition-closures.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/scanner.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/api.sparcf". ;; Loading #p".../lti/cl-ppcre-0.7.4/ppcre-tests.sparcf". T * (load "break-cl-ppcre") ; Loading #p".../lti/break-cl-ppcre.sparcf". T * (setf break-cl-ppcre::*test-expect-login* #| ... |#) #| ... |# * (setf break-cl-ppcre::*test-expect-password* #| ... |#) #| ... |# * (break-cl-ppcre::test-telnet #| ... |#) Trying #| ... |#... Connected to #| ... |#. Escape character is '^]'. SunOS 5.6 login: #| ... |# Password: Last login: Fri Mar 5 04:44:33 from 10.17.193.24 Sun Microsystems Inc. SunOS 5.6 Generic August 1997 tekelec:[/tekelec/users/#| ... |#] 1 % ls SEdisplaylJ_1N_ log mgts_cit_csh auto.sh mgts.Xdefaults mgts_cit_env auto_datafiles.tar mgts.cshrc mgts_gsr6.tar datafile.6.0.1.0.4.tar mgts.login mgts_run datafiles mgts.profile mgts_umt_csh datafiles.bak mgts.xinitrc set_mgts_env install.errors mgts.xsession tekelec:[/tekelec/users/#| ... |#] 2 % T * (break-cl-ppcre::test-telnet #| ... |# :prompt break-cl-ppcre::+default-mgts-server-prompt--break-cl-ppcre+) Trying #| ... |#... Connected to #| ... |#. Escape character is '^]'. SunOS 5.6 login: #| ... |# Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER: -1 is not of type (MOD 536870911) Restarts: 0: [ABORT] Return to Top-Level. Debug (type H for help) ("DEFUN CREATE-BMH-MATCHER" -2) Source: ; File: .../lti/cl-ppcre-0.7.4/scanner.lisp (BMH-MATCHER-AUX) 0] (debugger) ; ; Warning: This function is undefined: ; DEBUGGER ; Error in KERNEL:%COERCE-TO-FUNCTION: the function DEBUGGER is undefined. Error flushed ... 0] (break) Break Restarts: 0: [CONTINUE] Return from BREAK. 1: [ABORT ] Return to debug level 1. 2: Return to Top-Level. Debug (type H for help) (DEBUG::DEBUG-EVAL-PRINT (BREAK)) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/debug.lisp. 0]] backtrace 0: (DEBUG::DEBUG-EVAL-PRINT (BREAK)) 1: (DEBUG::DEBUG-LOOP) 2: (DEBUG:INTERNAL-DEBUG) 3: (DEBUG::INVOKE-TTY-DEBUGGER #<TYPE-ERROR {4080E945}>) 4: (DEBUG::REAL-INVOKE-DEBUGGER #<TYPE-ERROR {4080E945}>) 5: (INVOKE-DEBUGGER #<TYPE-ERROR {4080E945}>) 6: (ERROR TYPE-ERROR :FUNCTION-NAME "DEFUN CREATE-BMH-MATCHER" :DATUM ...) 7: (KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER "DEFUN CREATE-BMH-MATCHER" #.(SYSTEM:INT-SAP #x38000308) #<Alien (* #) at #xFFBEDEC0> (429 526)) 8: (KERNEL::INTERNAL-ERROR #.(SYSTEM:INT-SAP #xFFBEDEC0) #<unused-arg>) 9: ("Foreign function call land") 10: ("DEFUN CREATE-BMH-MATCHER" -2) 11: ("DEFUN CREATE-SCANNER-AUX" " " 0 1) 12: ("DEFMETHOD EXPECT (FUNCTION STREAM)" #<unused-arg> #<unused-arg> #<Closure Over Function "DEFUN CREATE-SCANNER-AUX" {4080DA81}> #<Stream for descriptor 9> ...) 13: (BREAK-CL-PPCRE::TEST-TELNET #| ... |# :LOGIN NIL :PASSWORD ...) 14: (INTERACTIVE-EVAL (BREAK-CL-PPCRE::TEST-TELNET #| ... |# :PROMPT BREAK-CL-PPCRE::+DEFAULT-MGTS-SERVER-PROMPT--BREAK-CL-PPCRE+)) 15: (COMMON-LISP::%TOP-LEVEL) 16: (COMMON-LISP::RESTART-LISP) 0]] (ext:quit) Password: % cat break-cl-ppcre.lisp (defpackage #:break-cl-ppcre (:use #:common-lisp #:extensions #:cl-ppcre)) (in-package #:break-cl-ppcre) ;; I believe that the following regular expression is causing CL-PPCRE ;; to choke. (defconstant +default-mgts-server-prompt--break-cl-ppcre+ `(:sequence #\Newline "tekelec:[" (:greedy-repetition 0 nil :everything) "] " (:greedy-repetition 1 nil :digit-class) " % " :end-anchor)) (defvar *test-expect-login* nil) (defvar *test-expect-password* nil) (declaim (inline string-cat)) (defun string-cat (&rest args) (apply #'concatenate 'string args)) (defmacro with-default-spawn ((default-spawn) &body code) `(flet ((expect (expected &optional (spawn ,default-spawn) &key (echo *standard-output*)) (expect expected spawn :echo echo)) (send (message &optional (spawn ,default-spawn)) (send message spawn))) (macrolet ((send1 (&rest message-parts) `(send* ,',default-spawn ,@message-parts))) ,@code))) (defmacro with-spawn-process ((id exec-name &optional exec-args &key without-default-spawn) &body code) (let ((exec-args-value (gensym "EXEC-ARGS-VALUE-"))) `(let* ((,exec-args-value ,exec-args) (,id (spawn ,exec-name ,exec-args-value))) (unwind-protect ,(if without-default-spawn `(progn ,@code) `(with-default-spawn (,id) ,@code)) (process-close ,id))))) (defmacro with-spawn-stream ((stream exec-name &optional exec-args &key without-default-spawn) &body code) (let ((exec-args-value (gensym "EXEC-ARGS-VALUE-")) (id (gensym "SPAWN-PROCESS-"))) `(let ((,exec-args-value ,exec-args)) (with-spawn-process (,id ,exec-name ,exec-args-value :without-default-spawn t) (let ((,stream (process-pty ,id))) ,(if without-default-spawn `(progn ,@code) `(with-default-spawn (,stream) ,@code))))))) (defgeneric expect (expected spawn &key echo) (:documentation "ARGS: EXPECTED SPAWN &KEY ECHO This is a CMU CL version of Don Libes' expect. EXPECTED is what one expects to find on SPAWN, created by the function SPAWN.")) (defgeneric send (message spawn) (:documentation "ARGS: MESSAGE SPAWN A CMU CL version of Don Libe's send. Send MESSAGE to SPAWN, created by the function SPAWN.")) (defun send* (spawn &rest messages) (send (apply #'concatenate 'string (mapcar #'string messages)) spawn)) (defun spawn (program &optional args) "ARGS: PROGRAM &OPTIONAL ARGS A CMU CL version of Don Libes' spawn. PROGRAM is the name of the program to be exec'd in a pseudo-terminal." (run-program program args :wait nil :pty t :input t :output t :error t)) (defmethod expect ((expected string) (spawn extensions::process) &key (echo *standard-output*) regexp case-insensitive-mode multi-line-mode single-line-mode extended-mode) (expect (create-scanner (if regexp expected (quote-meta-chars expected)) :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode) (process-pty spawn) :echo echo)) (defmethod expect ((expected string) (spawn stream) &key (echo *standard-output*) regexp case-insensitive-mode multi-line-mode single-line-mode extended-mode) (expect (create-scanner (if regexp expected (quote-meta-chars expected)) :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode) spawn :echo echo)) ;; expected is a parse-tree (defmethod expect ((expected t) (spawn extensions::process) &key (echo *standard-output*) case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive) (expect (create-scanner expected :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode :destructive destructive) (process-pty spawn) :echo echo)) ;; expected is a parse-tree (defmethod expect ((expected t) (spawn stream) &key (echo *standard-output*) case-insensitive-mode multi-line-mode single-line-mode extended-mode destructive) (expect (create-scanner expected :case-insensitive-mode case-insensitive-mode :multi-line-mode multi-line-mode :single-line-mode single-line-mode :extended-mode extended-mode :destructive destructive) spawn :echo echo)) ;; expected is a scanner (defmethod expect ((expected function) (spawn extensions::process) &key (echo *standard-output*)) (expect expected (process-pty spawn) :echo echo)) ;; expected is a scanner (defmethod expect ((expected function) (spawn stream) &key (echo *standard-output*)) (let ((buffer (make-array '(0) :element-type 'base-char :fill-pointer 0 :adjustable t))) (with-output-to-string (match buffer) (let ((io (make-echo-stream spawn (if echo (make-broadcast-stream match echo) match)))) ;; I know that this is going to be a horribly inefficient ;; algorithm; i.e. reading a single character at a time ;; and re-scanning the BUFFER every time a new character ;; is added. I'll work on fixing this later. For know, I ;; just want to get something working. -- Damien Kick (loop (read-char io) (multiple-value-bind (match-start match-end reg-starts reg-ends) (scan expected buffer) (when match-start (return (values buffer match-start match-end reg-starts reg-ends))))))))) (defmethod send ((message string) (spawn extensions::process)) (send message (process-pty spawn))) (defmethod send ((message string) (spawn stream)) (write-string message spawn) (force-output spawn) message) (defun test-telnet (address &key (login *test-expect-login*) (password *test-expect-password*) (prompt '(:sequence "tekelec:[" (:greedy-repetition 0 nil :everything) #\] (:greedy-repetition 1 nil #\Space) (:greedy-repetition 1 nil (:char-class (:range #\0 #\9))) (:greedy-repetition 1 nil #\Space) #\%))) (with-spawn-stream (stream "telnet" (list address)) (expect "login:") (send (string-cat (string login) (string #\Newline))) (expect "assword:") (send (string-cat (string password) (string #\Newline))) (expect prompt) (send (string-cat "ls" (string #\Newline))) (expect prompt) t)) %

2 1

[cl-ppcre-devel] Re: cl-ppcre question
by Edi Weitz 10 Mar '04

10 Mar '04

Hi Arthur! Sorry for the late reply, I was on vacation. On Sun, 29 Feb 2004 22:36:46 -0600, Arthur Gold <artiegold(a)austin.rr.com> wrote: > A quick question from and old (out of work) programmer who has > suddenly had a little Common Lisp work fall in his lap (yea!): I've > been starting to work with cl-ppcre for a project, and the following > result confuses me -- > > (ppcre::scan "(\w)*" "A_B_C") > > returns NIL > > Shouldn't "(\w)*" match any sequence of letters or underscores? > > Many thanks for your time. Try (PPCRE::SCAN "(\\w)*" "A_B_C") instead. The first backslash is necessary because of CL's string syntax, the second backslash is the one that's actually "sent" to CL-PPCRE. * (princ "\w") w "w" * (princ "\\w") \w "\\w" If you think you're using too much backslashes you might want to check CL-INTERPOL. Cheers, Edi. PS: Please use the CL-PPCRE mailing list if possible.

1 0