July 2007 - cxml-devel - mailman3.common-lisp.net

[cxml-devel] suggestion and bug report
by Francis Leboutte 22 Jul '07

22 Jul '07

Hello, Below, a small suggestion and a bug report. Regards, Francis 1. In rod-to-utf8-string, element-type should be base-char, which is used for strings of characters having 8-bit encoding in some implementations (would take less memory in these implementations - LW, ACL and others maybe) (defun rod-to-utf8-string (rod) (let ((out (make-buffer :element-type 'character))) (runes-to-utf8/adjustable-string out rod (length rod)) out)) 2. While experimenting with the functions cxml:utf8-string-to-rod and cxml:rod-to-utf8-string, I found that the string returned by cxml:utf8-string-to-rod doesn't include the last input character when this character is not an ascii character. On LWw 5.0.2, an example with the micro sign: USER 8 > (cxml:rod-to-utf8-string (format nil "abc~C" (code-char #x03BC))) "abcÎ¼" USER 9 > (cxml:utf8-string-to-rod (cxml:rod-to-utf8-string (format nil "abc~C." (code-char #x03BC)))) "abcµ." USER 10 > (cxml:utf8-string-to-rod (cxml:rod-to-utf8-string (format nil "abc~C" (code-char #x03BC)))) "abc" The problem seems to come from the method (defmethod runes-encoding:decode-sequence ((encoding (eql :utf-8)) ...) ...) where the tests (< (%+ rptr ...) in-end) should probably be (<= (%+ rptr ...) in-end) instead. -- Francis Leboutte Algorithme, Rue de la Charrette 141, 4130 Tilff (Esneux), Belgique Service en informatique f.leboutte(a)algo.be www.algo.be +32-(0)4.388.3919

2 1

[cxml-devel] New release: cxml
by David Lichteblau 07 Jul '07

07 Jul '07

Hi, a new release tarball is available for download. The new release fixes the build on non-Unicode lisps and on OpenMCL, which were broken in the previous tarball. In addition, the following changes are part of the release: * New function cxml:parse. * Serialization no longer defaults to canonical form. * Fixed octet array argument to make-source. * XMLS compatibility is not bug-for-bug-compatible with XMLS any more. There is now a mode using pairs of local name and namespace URI, and a second mode using qualified names only. The old behaviour using pairs of prefix and local names was removed. (Thanks to Douglas Crosher.) * SCL support (thanks to Douglas Crosher). Includes support for implementations where URIs are valid namestrings, and a mode where normal streams are used instead of xstreams and ystreams (albeit both SCL-specific at this point). * new convenience serialization function cxml:doctype. Various DTD serialization fixes.

1 0

[cxml-devel] ASD files for cxml don't work for openmcl
by David Lichteblau 06 Jul '07

06 Jul '07

Hi, (sorry for starting a new thread, I don't have your original mail right now) Quoting Sunil Mishra (smishra at sfmishras.com) > I've recently updated to the latest cxml, to the CVS HEAD, and am having > trouble with openmcl64. All in all, it looks like rune-is-integer has > been poorly tested. I've had trouble with the asd files, compiling the > tester, and with parsing XML. I've included lots of detail below. I am > happy to help debug cxml, but I don't understand the system and will > need guidance. I have just checked in fixes for the two problems you reported (build broken without Unicode, and xml:base handling broken without Unicode). In addition, OpenMCL actually supports Unicode now, but cxml failed to detect that. (Also fixed.) Please test! Make sure to clean out fasls before doing so. If it works for you, I will make a new release. Thanks, David

2 1

[cxml-devel] ASD files for cxml don't work for openmcl
by Sunil Mishra 05 Jul '07

05 Jul '07

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I've recently updated to the latest cxml, to the CVS HEAD, and am having trouble with openmcl64. All in all, it looks like rune-is-integer has been poorly tested. I've had trouble with the asd files, compiling the tester, and with parsing XML. I've included lots of detail below. I am happy to help debug cxml, but I don't understand the system and will need guidance. I'm finding that the asd files for cxml don't work as written for #+rune-is-integer. Consider what happens in the script below: ? (asdf:operate 'asdf:load-op 'cxml) ; loading system definition from /usr/local/asdf-install/site-systems/cxml.asd into #<Package "ASDF0"> ; registering #<SYSTEM :CXML-XML #x300040E0D25D> as CXML-XML ; registering #<SYSTEM :CXML-DOM #x300040E4A05D> as CXML-DOM ; registering #<SYSTEM :CXML-KLACKS #x300040E78FAD> as CXML-KLACKS ; registering #<SYSTEM :CXML-TEST #x300040E67EAD> as CXML-TEST ; registering #<SYSTEM :CXML #x300040E9841D> as CXML ; loading system definition from /usr/local/asdf-install/site-systems/trivial-gray-streams.asd into #<Package "ASDF0"> ; registering #<SYSTEM :TRIVIAL-GRAY-STREAMS #x300040E8191D> as TRIVIAL-GRAY-STREAMS ; loading system definition from /usr/local/asdf-install/site-systems/puri.asd into #<Package "ASDF0"> ; registering #<SYSTEM PURI #x300040EAA8FD> as PURI ; registering #<SYSTEM PURI-TESTS #x300040ED2FAD> as PURI-TESTS ; loading system definition from /usr/local/asdf-install/site-systems/runes.asd into #<Package "ASDF0"> ;;; Checking for wide character support... no, reverting to octet strings. ;;; Building Closure with (UNSIGNED-BYTE 16) RUNES ; registering #<SYSTEM :RUNES #x300040F11DFD> as RUNES > Error: There is no package named "UTF8-DOM" . > While executing: CCL::%FASL-NVPACKAGE, in process listener(1). > Type :GO to continue, :POP to abort, :R for a list of available restarts. > If continued: Retry finding package with name "UTF8-DOM". > Type :? for other options. 1 > :pop ;;; *** At this point, go off to the shell and force reloading cxml.asd $ touch cxml.asd ;;; *** Now back to lisp... ? (asdf:operate 'asdf:load-op 'cxml) ; loading system definition from /usr/local/asdf-install/site-systems/cxml.asd into #<Package "ASDF0"> NIL ? I think what's happening here is that the feature rune-is-integer is not yet defined when cxml.asd is loaded. So the system definition cxml-dom is defined incorrectly. Only *after* cxml.asd is loaded, we load rune.asd, which is where rune-is-integer is defined. When I touch cxml.asd, it is reloaded and things work fine as rune-is-integer is now correctly defined. Splitting rune.asd and cxml.asd is currently clearly not working. I think you will be able to replicate this problem if you define cxml-dom as follows: (asdf:defsystem :cxml-dom :default-component-class closure-source-file :pathname (merge-pathnames "dom/" (make-pathname :name nil :type nil :defaults *load-truename*)) :components ((:file "package") #+rune-is-character (:file rune-impl :pathname "dom-impl" :depends-on ("package")) #+rune-is-character (:file rune-builder :pathname "dom-builder" :depends-on (rune-impl)) #+rune-is-integer (utf8dom-file utf8-impl :pathname "dom-impl" :depends-on ("package")) #+rune-is-integer (utf8dom-file utf8-builder :pathname "dom-builder" :depends-on (utf8-impl)) (:file "dom-sax" :depends-on ("package"))) :depends-on (:cxml-xml)) Concatenating the files runes.asd followed by cxml.asd, and using the result as cxml.asd, allows cxml to load. However, even parsing an XML document currently is non-functional. All this used to work fine in the previous build of cxml. In addition, the files test/domtest.lisp and test/xmlconf.lisp have repeated references to the package rune-dom, which does not work for rune-is-integer. Parsing an XML document breaks in the function compute-base. When #+rune-is-integer, sax:find-attribute is called with a string name of an attribute while document data is stored as an integer array. Here's a part of the stack trace: 0: (STRING) 1: (CCL::STRING-START-END "xml:base" #(120 109 108 110 115) 0) 2: (STRING= "xml:base" #(120 109 108 110 115)) 3: (MEMBER "xml:base" '(#S(SAX::ATTRIBUTE :NAMESPACE-URI #(104 116 116 112 58 47 47 119 119 119 ...) :LOCAL-NAME NIL :QNAME #(120 109 108 110 115) :VALUE #(104 116 116 112 58 47 47 119 119 119 ...) :SPECIFIED-P ...))) 4: (CCL::FIND-POSITIONAL-TEST-KEY "xml:base" #<Compiled-function SAX::%ROD= #x30004143E34F> #<Anonymous Function #x30004143F5AF> 'NIL) 5: (CXML::COMPUTE-BASE) Locals: CXML::ATTRS = (#S(SAX::ATTRIBUTE :NAMESPACE-URI #(104 116 116 112 58 47 47 119 119 119 ...) :LOCAL-NAME NIL :QNAME #(120 109 108 110 115) :VALUE #(104 116 116 112 58 47 47 119 119 119 ...) :SPECIFIED-P ...)) 6: (CXML::P/SZTAG #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429E545D>>))) Locals: CXML::INPUT = #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429C8E4D>>)) CXML::CAT = :STAG CXML::SEM = (#(104 116 109 108) (#(120 109 108 110 115) . #(104 116 116 112 58 47 47 119 119 119 ...))) #:ARGS3257 = #<DESTRUCTURE-STATE #x4216E6D> CXML::NAME = #(104 116 109 108) #:REST3258 = ((#(120 109 108 110 115) . #(104 116 116 112 58 47 47 119 119 119 ...))) CXML::*NAMESPACE-BINDINGS* = ((#()) (#(120 109 108 110 115) . #(104 116 116 112 58 47 47 119 119 119 ...)) (#(120 109 108) . #(104 116 116 112 58 47 47 119 119 119 ...))) CXML::NEW-NAMESPACES = ((NIL . #(104 116 116 112 58 47 47 119 119 119 ...))) 7: (CXML::P/ELEMENT #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429E545D>>))) Locals: 8: (CXML::P/DOCUMENT #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429E545D>>)) #<UTF8-DOM::DOM-BUILDER #x3000429EBCAD>) Locals: CXML::VALIDATE = NIL CXML::ROOT = NIL CXML::ENTITY-RESOLVER = #<Compiled-function HAXL::DEFAULT-ENTITY-RESOLVER #x30004161DECF> CXML::DISALLOW-INTERNAL-SUBSET = NIL CXML::RECODE = T RUNES:XSTREAM = #<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429C8E4D>> CXML::NAME = #<error printing CXML::STREAM-NAME #x3000429C8E4D> CXML::BASE = #<URI file://+/Users/smishra/lisp/wessal/templates/error.xml> CXML::*CTX* = NIL CXML::*VALIDATE* = T CXML::*NAMESPACE-BINDINGS* = ((#()) (#(120 109 108 110 115) . #(104 116 116 112 58 47 47 119 119 119 ...)) (#(120 109 108) . #(104 116 116 112 58 47 47 119 119 119 ...))) CXML::*DATA-BEHAVIOUR* = #<Unbound> CXML::*DATA-BEHAVIOUR* = :DTD 9: (#<Anonymous Function #x3000414D19AF> '(:ENTITY-RESOLVER #<Compiled-function HAXL::DEFAULT-ENTITY-RESOLVER #x30004161DECF>) #<UTF8-DOM::DOM-BUILDER #x3000429EBCAD> #<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429E545D>> #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429E545D>>))) Locals: CXML::ARGS = (:ENTITY-RESOLVER #<Compiled-function HAXL::DEFAULT-ENTITY-RESOLVER #x30004161DECF>) CXML::HANDLER = #<UTF8-DOM::DOM-BUILDER #x3000429CF69D> RUNES:XSTREAM = #<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429C8E4D>> CXML::ZSTREAM = #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429C8E4D>>)) CXML::*SCRATCH-PAD* = #<Unbound> CXML::*SCRATCH-PAD-2* = #<Unbound> CXML::*SCRATCH-PAD-3* = #<Unbound> CXML::*SCRATCH-PAD-4* = #<Unbound> 10: (CXML::CALL-WITH-ZSTREAM #<COMPILED-LEXICAL-CLOSURE #x3000429E53EF> #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429E545D>>))) Locals: CXML::FN = #<COMPILED-LEXICAL-CLOSURE #x3000429C8DDF> CXML::ZSTREAM = #S(CXML::ZSTREAM :TOKEN-CATEGORY NIL :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429C8E4D>>)) CCL::*INTERRUPT-LEVEL* = 0 CCL::*INTERRUPT-LEVEL* = -1 Catch-tags: NIL 11: (CXML::PARSE-XSTREAM #<RUNES:XSTREAM #<error printing CXML::STREAM-NAME #x3000429E545D>> #<UTF8-DOM::DOM-BUILDER #x3000429EBCAD>) Locals: CXML::HANDLER = #<UTF8-DOM::DOM-BUILDER #x3000429CF69D> CXML::ARGS = (:ENTITY-RESOLVER #<Compiled-function HAXL::DEFAULT-ENTITY-RESOLVER #x30004161DECF>) CXML::*CTX* = NIL #:G3378 = (RUNES-ENCODING:ENCODING-ERROR) CCL::%HANDLERS% = ((PROCESS-RESET)) Catch-tags: NIL 12: (CXML::CALL-WITH-OPEN-XFILE #<COMPILED-LEXICAL-CLOSURE #x3000429EB9FF>) Locals: CXML::CONTINUATION = #<COMPILED-LEXICAL-CLOSURE #x3000429CF3EF> CXML::OPEN-ARGS = (#P"/Users/smishra/lisp/wessal/templates/error.xml") CXML::INPUT = #<BASIC-FILE-BINARY-INPUT-STREAM ("/Users/smishra/lisp/wessal/templates/error.xml"/12 ISO-8859-1) #x3000429CF36D> CCL::*INTERRUPT-LEVEL* = 0 CCL::*INTERRUPT-LEVEL* = -1 Catch-tags: NIL I know that the particular invocation of parse-xml matters. Here's how I'm doing mine: (cxml:parse-file document-path #+allegro (rune-dom:make-dom-builder) #+openmcl (cxml-dom:make-dom-builder) :entity-resolver #'default-entity-resolver) Any help will be much appreciated. Thanks, Sunil -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGjTU8jRnqWMAEf1cRAueTAKDLo+dR9MWXtg6Sr78j340q4nGaHQCgsK8Q 4cfCfsF+BZQ6qd7y77sbAoA= =j+fv -----END PGP SIGNATURE-----

2 1