- cxml-devel - mailman3.common-lisp.net

[cxml-devel] seeking suggestion for handling malformed attribute names
by Russell Kliese 29 Oct '11

29 Oct '11

I have been using closure-html and cxml to extract data from web pages. Up until now it has worked quite well, but I have come across a web page that has quite extraordinarily malformed html markup. I am looking for suggestions on how to continue parsing even when attribute names are malformed. Here is a self-contained example containing some of the offending markup: (require :closure-html) (require :cxml-stp) (defvar *html-str*) (setf *html-str* "<html> <body> <table> <tr><td vertical-align:=\"\" ;=\"\"></td></tr> </table> </body> </html> ") (chtml:parse *html-str* (cxml-stp:make-builder)) This example give CXML:WELL-FORMEDNESS-VIOLATION. I guess I could write a custom SAX handler to skip attributes with these errors, but is there a more simple solution? Should the chtml parser be patched to handle this case? Are there any other ways to make parsing more robust? Or is this markup beyond hope of repair? Regards, Russell

2 3

[cxml-devel] parsing from a http stream (no EOF at the end)
by Attila Lendvai 03 Oct '11

03 Oct '11

cxml:parse is stuck waiting for an EOF even after the last closing tag when parsing from a stream (in my case it's a http stream in a server entry point. the client waits for the server to react, the server waits for the EOF). looking at the code i get the impression that it was never meant to be used like this, an EOF is pretty much mandatory. could someone comment on this? was cxml ever meant to support such use-case, just noone really needed it yet? any plans for supporting this? any hints are appreciated, -- attila Notice the erosion of your (digital) freedom, and do something about it! PGP: 2FA1 A9DC 9C1E BA25 A59C 963F 5D5F 45C7 DFCD 0A39 OTR XMPP: 8647EEAC EA30FEEF E1B55146 573E52EE 21B1FF06

3 3

[cxml-devel] Will there be a release since 2008?
by Kalyanov Dmitry 05 Aug '11

05 Aug '11

CXML had not seen a release for a long time (last release was in 2008). The released version has some problems (e.g., ASDF2 will not load it and SBCL ships with ASDF2). I think that it would be easier for distribution maintainers (like Gentoo Portage Lisp team) if there would be a recent release of CXML. So I kindly ask CXML developers to make a release of CXML.

1 0

[cxml-devel] Ho do I write the following on a sink?
by Marco Antoniotti 30 May '11

30 May '11

Hi I am trying to write "Λ" on a sink. (let ((sink (cxml:make-character-stream-sink *standard-output* :indentation 2 :canonical nil))) (sax:start-document sink) (xhtml-generator:write-doctype sink) (xhtml-generator:with-html sink (:html (:head (:title "Titel")) (:body ((:p "style" "font-weight: bold") "Inhalt") (:ul (:li "Eins") (:li "Zwei") (:li "Λ") (:li "Drei"))))) (sax:end-document sink)) <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html> <head> <title> Titel</title> </head> <body> <p style="font-weight: bold"> Inhalt</p> <ul> <li> Eins</li> <li> Zwei</li> <li> &Lambda;</li> <li> Drei</li> </ul> </body> </html> #<EDITOR::RUBBER-STREAM #<EDITOR:BUFFER CAPI interactive-pane 2> 219C7A3F> How do I get it "right"? Thank -- Marco Antoniotti

4 9

[cxml-devel] A bug fix and a refactoring for consideration
by Russ Tyndall 02 May '11

02 May '11

I made a small bugfix and a refactoring of dom:map-document that I was hoping could be pulled upstream. Without the change to cxml:attribute*, attributes created with this function will not be serialized without setting the :include-default-values to T while when calling map-document. The map-document->walk refactoring was important so that I could map sub documents. I use this in my templating library (TALCL) to map nodes created in a dom document into a buffer of sax events (that can be played out during serialization). These are available from my github account: git pull git://github.com/bobbysmith007/cxml.git patches-on-upstream https://github.com/bobbysmith007/cxml/commits/patches-on-upstream If you need anything further from me, please don't hesitate to ask. Cheers, Russ Tyndall commit 6fe408e85926329de511cc533c3c670b802df641 Author: Russ Tyndall <russ(a)acceleration.net> Date: Mon May 2 16:07:32 2011 -0400 Refactored dom:map-document into methods of dom:walk * dom:map-document now calls dom:walk on the document * walk used to be a labels inside of map-document, now it is a generic function called by map-document commit 0af0575d47379505ee4b89d042d0c7929b6e727e Author: Russ Tyndall <russ(a)acceleration.net> Date: Mon May 2 16:06:44 2011 -0400 make cxml:attribute* helper accept and correctly set specified-p (which defaults to T rather than the previous nil) * specified-p determines whether or not an attribute is considered being a default attribute vs one that has been explicitly set. Since this is a builder function, it seems that this should be T by default.

1 0

[cxml-devel] roundtripping vcard-4.0 RNG schema
by Cyrus Harmon 28 Apr '11

28 Apr '11

So, I've been messing around with vcards a bit and decided that I would try to make sense of the new XML format for VCARD data, described here: http://tools.ietf.org/html/draft-perreault-vcarddav-vcardxml-02#appendix-A So far so good. I cut and paste the RNG spec from the document (attached), and attempt to read in the compact form and write out the longhand XML RNG schema. So, I do: (defparameter *compact* (cxml-rng:parse-compact #p"vcard-4_0.rng")) ;; so far so good (cxml-rng:make-validator *compact*) ;; again, that works (with-open-file (out "foo.xml" :direction :output :element-type '(unsigned-byte 8)) (cxml-rng:serialize-schema *compact* (cxml:make-octet-stream-sink out))) ;; BOOM! I get the following error: There is no applicable method for the generic function #<STANDARD-GENERIC-FUNCTION CXML:UNPARSE-ATTRIBUTE (3)> when called with arguments (#:ANONYMOUS2507). [Condition of type SIMPLE-ERROR] Restarts: 0: [RETRY] Retry calling the generic function. 1: [*ABORT] Return to SLIME's top level. 2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "worker" RUNNING {1002D97001}>) Backtrace: 0: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) #<unused argument> #<unused argument> #<STANDARD-GENERIC-FUNCTION CXML:UNPARSE-ATTRIBUTE (3)> #:ANONYMOUS2507) 1: (SB-PCL::CALL-NO-APPLICABLE-METHOD #<STANDARD-GENERIC-FUNCTION CXML:UNPARSE-ATTRIBUTE (3)> (#:ANONYMOUS2507)) 2: (CXML:ATTRIBUTE* #<unavailable argument> #<unavailable argument> #<unavailable argument> "name") Locals: CXML::LNAME = :<NOT-AVAILABLE> CXML::PREFIX = :<NOT-AVAILABLE> CXML::QNAME = "name" CXML::VALUE = :<NOT-AVAILABLE> 3: ((LAMBDA ())) 4: (CXML::INVOKE-WITH-ELEMENT* #<CLOSURE (LAMBDA #) {1002E13BA9}> #<unavailable argument> #<unavailable argument> "ref") Locals: CXML::FN = #<CLOSURE (LAMBDA ()) {1002E13BA9}> CXML::LNAME = :<NOT-AVAILABLE> CXML::PREFIX = :<NOT-AVAILABLE> CXML::QNAME = "ref" 5: (CXML-RNG::SERIALIZE-PATTERN #<CXML-RNG:REF {1006DB5171}>) 6: ((LAMBDA ())) 7: (CXML::INVOKE-WITH-ELEMENT* #<CLOSURE (LAMBDA #) {1002E13AF9}> #<unavailable argument> #<unavailable argument> "start") Locals: CXML::FN = #<CLOSURE (LAMBDA ()) {1002E13AF9}> CXML::LNAME = :<NOT-AVAILABLE> CXML::PREFIX = :<NOT-AVAILABLE> CXML::QNAME = "start" 8: ((LAMBDA ())) 9: (CXML::INVOKE-WITH-ELEMENT* #<CLOSURE (LAMBDA #) {1002E13A89}> #<unavailable argument> #<unavailable argument> "grammar") Now, normally, I'd just assume that this is something I messed up. But the fact that the schema parses and validates OK makes me wonder. Any suggestions on how to go about debugging this? thanks, Cyrus

1 2

[cxml-devel] ... and now for something completely different ...
by Marco Antoniotti 22 Apr '11

22 Apr '11

... and more interesting. I am past the set-up phase and I got to the point of building a SAX parser (I have not figured out yet what is the practical difference between SAX and StAX). My beef with XML is obviously increasing :) But let's not go there. Here is my thinking. I write SAX:START-ELEMENT methods and let each one of them look like (defmethod sax:start-element ((h smbl-handler) namespace-uri (local-name (equal "model")) qname attributes) ...) (defmethod sax:start-element ((h smbl-handler) namespace-uri (local-name (equal "compartment")) qname attributes) ...) You get the idea... Of course, we don't have EQUAL specializers (and I don't want to depend on "predicate dispatching" extensions :) ) so I did the following trick. (defclass sbml-parser (sax:sax-parser) ; Why do I need this? () ) (defclass sbml-handler (sax:default-handler) ((source :reader sbml-handler-source :initarg :source) (pathname :reader sbml-handler-pathname :initarg :pathname) ) ) (defclass sbml-symbolizer (cxml:sax-proxy) ()) ... and ... (defmethod sax:start-element ((h sbml-symbolizer) namespace-uri local-name qname attributes) (sax:start-element (cxml:proxy-chained-handler h) (intern namespace-uri) (intern local-name) (intern qname) attributes) ) So now I can write (defmethod sax:start-element ((h smbl-handler) (namespace-uri (eql '|http://www.smbl.org/sbml/level2/version4|)) (local-name (eql '|compartment|)) qname attributes) ...) apart from the details... Do you guys think this is a sensible approach? It'd make writing the SAX parser a cinch. Thanks -- Marco Antoniotti

1 0

[cxml-devel] ... and another
by Marco Antoniotti 22 Apr '11

22 Apr '11

Always on LWM. I assume this has to do with UNICODE or not support, but the example in the docs is misleading. CL-USER 37 > (defparameter *source* (cxml:make-source "<example>text</example>")) Error: In = of (#\< 254) arguments should be of type NUMBER. 1 (continue) Return a value to use. 2 Supply a new first argument. 3 (abort) Return to level 0. 4 Return to top loop level 0. Type :b for backtrace or :c <option number> to proceed. Type :bug-form "<subject>" for a bug report template or :? for other options. Cheers -- Marco Antoniotti

2 5

[cxml-devel] Counter-intuitive API for cxml:parse on LWM?
by Marco Antoniotti 21 Apr '11

21 Apr '11

Hi here is the example on LWM: CL-USER 17 > (cxml:parse #P"CellCycle-1991Tys-2.xml" (cxml:make-whitespace-normalizer (make-instance 'sax:default-handler)) :validate nil) NIL CL-USER 18 > (with-open-file (f #P"CellCycle-1991Tys-2.xml" :direction :input) (cxml:parse f (cxml:make-whitespace-normalizer (make-instance 'sax:default-handler)) :validate nil)) Error: Binary operation READ-BYTE attempted on character stream #<STREAM::LATIN-1-FILE-STREAM /Users/marcoxa/Projects/Genomics/biolosyst/sbmlambda/tests/CellCycle-1991Tys-2.xml>. 1 (abort) Return to level 0. 2 Return to top loop level 0. Type :b for backtrace or :c <option number> to proceed. Type :bug-form "<subject>" for a bug report template or :? for other options. CL-USER 19 : 1 > And here is the doc string for cxml:parse "@arg[input]{A string, pathname, octet vector, or stream.} @arg[handler]{A @class{SAX handler}} @arg[validate]{Boolean. Defaults to @code{nil}. If true, parse in validating mode, i.e. assert that the document contains a DOCTYPE declaration and conforms to the DTD declared.} ... @arg[recode]{Boolean. (Ignored on Lisps with Unicode support.) Recode rods to UTF-8 strings. Defaults to true. Make sure to use @fun{utf8-dom:make-dom-builder} if this option is enabled and @fun{rune-dom:make-dom-builder} otherwise.} @return{The value returned by @fun{sax:end-document} on @var{handler}.} Parse an XML document from @var{input}, which can be a string, pathname, octet vector, or stream. ... All SAX parsing functions share the same keyword arguments. Refer to @fun{parse} for details on keyword arguments." It appears that you need a 'xstream', but this is against the doc string content (and IMHO, against common sense). Cheers -- Marco Antoniotti

2 3

[cxml-devel] normalize-text-nodes!
by Andy Chambers 13 Jun '10

13 Jun '10

Hi, Using xuriella to perform the identity transform fails when there are empty elements (e.g. "<BasicDefinitions/>") in the source document. I believe the problem is in normalize-text-nodes! which doesn't check to see if there are any children before trying to iterate over them. This seems to trigger an assertion error in sbcl. XURIELLA> (apply-stylesheet #p"/usr/src/discworld/xslt/identity.xsl" "<root>abc</root>") "<?xml version=\"1.0\" encoding=\"UTF-8\"?> <root>abc</root>" XURIELLA> (apply-stylesheet #p"/usr/src/discworld/xslt/identity.xsl" "<root></root>") The value NIL is not of type VECTOR. [Condition of type TYPE-ERROR] Restarts: 0: [RETRY] Retry SLIME REPL evaluation request. 1: [ABORT] Return to SLIME's top level. 2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "repl-thread" RUNNING {CA2E459}>) Backtrace: 0: (CXML-STP-IMPL::NORMALIZE-TEXT-NODES! #.(CXML-STP:ELEMENT #| :PARENT of type DOCUMENT |# :LOCAL-NAME "root")) 1: ((SB-PCL::FAST-METHOD XPATH-PROTOCOL:CHILD-PIPE-USING-NAVIGATOR ((EQL :DEFAULT-NAVIGATOR) CXML-STP:ELEMENT)) ..) 2: ((LAMBDA (XPATH::NODE XPATH::STARTING-NODE)) #.(CXML-STP:ELEMENT #| :PARENT of type DOCUMENT |# :LOCAL-NAME "root") #.(CXML-STP:ELEMENT #| :PARENT of type DOCUMENT |# :LOCAL-NAME "root")) 3: ((LAMBDA (XPATH:CONTEXT)) #<XPATH:CONTEXT {C750F39}>) Simple fix is attached. Cheers, Andy

2 1