cxml-cvs October 2007

cxml-cvs@common-lisp.net

1 participants
27 discussions

[cxml-cvs] CVS cxml/doc
by dlichteblau 07 Oct '07

07 Oct '07

Update of /project/cxml/cvsroot/cxml/doc In directory clnet:/tmp/cvs-serv20897/doc Modified Files: index.xml installation.xml Log Message: updated documentation for the closure-common split --- /project/cxml/cvsroot/cxml/doc/index.xml 2007/10/03 15:23:40 1.18 +++ /project/cxml/cvsroot/cxml/doc/index.xml 2007/10/07 14:58:26 1.19 @@ -51,6 +51,18 @@ <a name="changes"/> <h3>Recent Changes</h3> + <div style="background-color: #f7f7f7; + width: 60%; + border: solid #9c0000; + margin: 0em 2pt 1em 2em; + padding: 1em"> + Runes have now been moved into + a <b>separate CVS module</b> unter the + name <b>closure-common</b>. Releases will be available + as <b>separate tarballs</b> in the download directory. Please + refer to the <a href="installation.html#download"> + installation instructions</a> for details. + </div> <p class="nomargin"><tt>rel-2007-xx-yy</tt></p> <ul class="nomargin"> <li> --- /project/cxml/cvsroot/cxml/doc/installation.xml 2007/05/26 21:55:58 1.3 +++ /project/cxml/cvsroot/cxml/doc/installation.xml 2007/10/07 14:58:26 1.4 @@ -5,12 +5,14 @@ <h2>Download</h2> <ul> <li> - <div><a href="http://common-lisp.net/project/cxml/download/">tarballs</a></div> + <div>Download <a href="http://common-lisp.net/project/cxml/download/">tarballs</a> for both cxml itself and closure-common.</div> </li> <li> <div> - Anoncvs (<a href="http://common-lisp.net/cgi-bin/viewcvs.cgi/cxml/?cvsroot=cxml">browse</a>): - <pre>cvs -d :pserver:anonymous:anonymous@common-lisp.net:/project/cxml/cvsroot co cxml</pre> + Or use anonymous CVS (<a href="http://common-lisp.net/cgi-bin/viewcvs.cgi/cxml/?cvsroot=cxml">browse</a>): + <pre>export CVSROOT=:pserver:anonymous:anonymous@common-lisp.net:/project/cxml/cvsroot +cvs co cxml +cvs co closure-common</pre> </div> </li> </ul> @@ -39,16 +41,20 @@ <p> <b>Prerequisites.</b> - CXML needs the <a href="http://www.cliki.net/Puri">puri</a> library - as well as <a href="http://www.common-lisp.net/project/cl-plus-ssl/#trivial-gray-streams">trivial-gray-streams</a>. + CXML needs <a href="http://www.cliki.net/Puri">puri</a> and + <a href="http://www.common-lisp.net/project/cl-plus-ssl/#trivial-gray-streams">trivial-gray-streams</a>. + In addition, + <a href="http://www.cliki.net/closure-common">closure-common</a> + is required, which is a separate module in cxml CVS (see above for + check-out instructions). </p> <p> <b>Compiling and loading CXML.</b> Register the .asd file, e.g. by symlinking it: </p> - <pre>$ ln -sf `pwd`/cxml.asd /path/to/your/registry/ -$ ln -sf `pwd`/runes.asd /path/to/your/registry/</pre> + <pre>$ ln -sf `pwd`/cxml/cxml.asd /path/to/your/registry/ +$ ln -sf `pwd`/closure-common/closure-common.asd /path/to/your/registry/</pre> <p>Then compile CXML using:</p> <pre>* (asdf:operate 'asdf:load-op :cxml)</pre>

1 0

[cxml-cvs] CVS cxml
by dlichteblau 07 Oct '07

07 Oct '07

Update of /project/cxml/cvsroot/cxml In directory clnet:/tmp/cvs-serv20169 Modified Files: cxml.asd Log Message: Moved runes/ into its own cvs module under its new name, closure-common. --- /project/cxml/cvsroot/cxml/cxml.asd 2007/07/05 20:58:15 1.20 +++ /project/cxml/cvsroot/cxml/cxml.asd 2007/10/07 14:45:07 1.21 @@ -2,8 +2,13 @@ (:use :asdf :cl)) (in-package :cxml-system) -;; force loading of runes.asd, which installs *features* this file depends on -(find-system :runes) +;; force loading of closure-common.asd, which installs *FEATURES* this +;; file depends on. Use MISSING-DEPENDENCY for asdf-install. +(unless (find-system :closure-common nil) + (error 'missing-dependency + :required-by :cxml + :version nil + :requires :closure-common)) (defclass closure-source-file (cl-source-file) ()) @@ -36,7 +41,7 @@ (:file "space-normalizer" :depends-on ("xml-parse")) (:file "catalog" :depends-on ("xml-parse")) (:file "sax-proxy" :depends-on ("xml-parse"))) - :depends-on (:runes :puri #-scl :trivial-gray-streams)) + :depends-on (:closure-common :puri #-scl :trivial-gray-streams)) (defclass utf8dom-file (closure-source-file) ((of))) @@ -57,7 +62,8 @@ (defmethod perform ((operation compile-op) (c utf8dom-file)) (let ((*features* (cons 'utf8dom-file *features*)) (*readtable* - (symbol-value (find-symbol "*UTF8-RUNES-READTABLE*" :runes-system)))) + (symbol-value (find-symbol "*UTF8-RUNES-READTABLE*" + :closure-common-system)))) (call-next-method))) (asdf:defsystem :cxml-dom

1 0

[cxml-cvs] CVS cxml/runes
by dlichteblau 07 Oct '07

07 Oct '07

Update of /project/cxml/cvsroot/cxml/runes In directory clnet:/tmp/cvs-serv20169/runes Removed Files: characters.lisp definline.lisp encodings-data.lisp encodings.lisp package.lisp runes.lisp stream-scl.lisp syntax.lisp utf8.lisp xstream.lisp ystream.lisp Log Message: Moved runes/ into its own cvs module under its new name, closure-common.

1 0

[cxml-cvs] CVS cxml/doc
by dlichteblau 03 Oct '07

03 Oct '07

Update of /project/cxml/cvsroot/cxml/doc In directory clnet:/tmp/cvs-serv25288/doc Modified Files: index.xml Log Message: web page update --- /project/cxml/cvsroot/cxml/doc/index.xml 2007/08/29 12:11:17 1.17 +++ /project/cxml/cvsroot/cxml/doc/index.xml 2007/10/03 15:23:40 1.18 @@ -53,8 +53,18 @@ <h3>Recent Changes</h3> <p class="nomargin"><tt>rel-2007-xx-yy</tt></p> <ul class="nomargin"> - <li>fixed make-source :buffering nil, thanks to Magnus Henoch for - the report</li> + <li> + fixed make-source :buffering nil, thanks to Magnus Henoch for + the report + </li> + <li> + fixed time and space usage in cases where entity references + follow each other (thanks to Ivan Shvedunov for the report) + </li> + <li> + In the DOM builder, grow a buffer for string normalization + exponentially, fixing a long-standing speed issue. + </li> </ul> <p class="nomargin"><tt>rel-2007-08-05</tt></p> <ul class="nomargin">

1 0

[cxml-cvs] CVS cxml/klacks
by dlichteblau 03 Oct '07

03 Oct '07

Update of /project/cxml/cvsroot/cxml/klacks In directory clnet:/tmp/cvs-serv25228 Modified Files: klacks.lisp package.lisp Log Message: Minor klacks enhancements. klacks/klacks.lisp (CONSUME-CHARACTERS): New function. klacks/package.lisp (KLACKS): Export dribble-handler and consume-characters. --- /project/cxml/cvsroot/cxml/klacks/klacks.lisp 2007/05/01 18:21:41 1.8 +++ /project/cxml/cvsroot/cxml/klacks/klacks.lisp 2007/10/03 15:22:57 1.9 @@ -79,6 +79,12 @@ (check-type key (member :characters)) characters)) +(defun klacks:consume-characters (source) + (with-output-to-string (s) + (while (eq (klacks:peek source) :characters) + (write-string (klacks:current-characters source) s) + (klacks:consume source)))) + (defun klacks:serialize-event (source handler &key (consume t)) (multiple-value-bind (key a b c) (klacks:peek source) (let ((result nil)) --- /project/cxml/cvsroot/cxml/klacks/package.lisp 2007/05/01 18:21:41 1.6 +++ /project/cxml/cvsroot/cxml/klacks/package.lisp 2007/10/03 15:22:57 1.7 @@ -21,8 +21,10 @@ (:export #:source #:close-source #:with-open-source + #:tapping-source #:make-tapping-source + #:dribble-handler #:peek #:peek-value @@ -42,6 +44,7 @@ #:current-lname #:current-qname #:current-characters + #:consume-characters #:current-cdata-section-p #:map-current-namespace-declarations

1 0

[cxml-cvs] CVS cxml/xml
by dlichteblau 03 Oct '07

03 Oct '07

Update of /project/cxml/cvsroot/cxml/xml In directory clnet:/tmp/cvs-serv25157 Modified Files: xml-parse.lisp Log Message: Fixed time and space usage in cases where entity references follow each other (thanks to Ivan Shvedunov for the report). * xml/xml-parse.lisp (P/CONTENT): Removed useless call to append. Use loop instead of tail recursion. --- /project/cxml/cvsroot/cxml/xml/xml-parse.lisp 2007/08/05 11:15:48 1.72 +++ /project/cxml/cvsroot/cxml/xml/xml-parse.lisp 2007/10/03 15:21:56 1.73 @@ -2800,45 +2800,39 @@ (defun p/content (input) ;; [43] content ::= (element | CharData | Reference | CDSect | PI | Comment)* - (multiple-value-bind (cat sem) (peek-token input) - (case cat - ((:stag :ztag) - (p/element input) - (p/content input)) - ((:CDATA) - (process-characters input sem) - (sax:characters (handler *ctx*) sem) - (p/content input)) - ((:ENTITY-REF) - (let ((name sem)) - (consume-token input) - (append - (recurse-on-entity input name :general - (lambda (input) - (prog1 - (etypecase (checked-get-entdef name :general) - (internal-entdef (p/content input)) - (external-entdef (p/ext-parsed-ent input))) - (unless (eq (peek-token input) :eof) - (wf-error input "Trailing garbage. - ~S" - (peek-token input)))))) - (p/content input)))) - ((:<!\[) - (let ((data (process-cdata-section input))) - (sax:start-cdata (handler *ctx*)) - (sax:characters (handler *ctx*) data) - (sax:end-cdata (handler *ctx*))) - (p/content input)) - ((:PI) - (consume-token input) - (sax:processing-instruction (handler *ctx*) (car sem) (cdr sem)) - (p/content input)) - ((:COMMENT) - (consume-token input) - (sax:comment (handler *ctx*) sem) - (p/content input)) - (otherwise - nil)))) + (loop + (multiple-value-bind (cat sem) (peek-token input) + (case cat + ((:stag :ztag) + (p/element input)) + ((:CDATA) + (process-characters input sem) + (sax:characters (handler *ctx*) sem)) + ((:ENTITY-REF) + (let ((name sem)) + (consume-token input) + (recurse-on-entity input name :general + (lambda (input) + (prog1 + (etypecase (checked-get-entdef name :general) + (internal-entdef (p/content input)) + (external-entdef (p/ext-parsed-ent input))) + (unless (eq (peek-token input) :eof) + (wf-error input "Trailing garbage. - ~S" + (peek-token input)))))))) + ((:<!\[) + (let ((data (process-cdata-section input))) + (sax:start-cdata (handler *ctx*)) + (sax:characters (handler *ctx*) data) + (sax:end-cdata (handler *ctx*)))) + ((:PI) + (consume-token input) + (sax:processing-instruction (handler *ctx*) (car sem) (cdr sem))) + ((:COMMENT) + (consume-token input) + (sax:comment (handler *ctx*) sem)) + (otherwise + (return)))))) ;; [78] extParsedEnt ::= TextDecl? contentw ;; [79] extPE ::= TextDecl? extSubsetDecl

1 0

[cxml-cvs] CVS cxml/dom
by dlichteblau 03 Oct '07

03 Oct '07

Update of /project/cxml/cvsroot/cxml/dom In directory clnet:/tmp/cvs-serv23055 Modified Files: dom-builder.lisp dom-impl.lisp Log Message: Grow a buffer for string normalization exponentially. * dom/dom-builder.lisp (DOM-BUILDER): New slot `text-buffer'. (SAX:START-ELEMENT, SAX:END-ELEMENT, SAX:START-CDATA, SAX:END-CDATA, SAX:PROCESSING-INSTRUCTION, SAX:COMMENT): Call flush-characters. (SAX:CHARACTERS): Rewritten. (FLUSH-CHARACTERS): New, based on the old sax:characters. * dom/dom-impl.lisp ((initialize-instance :after entity-reference)): Call flush-characters. --- /project/cxml/cvsroot/cxml/dom/dom-builder.lisp 2007/07/22 19:59:26 1.13 +++ /project/cxml/cvsroot/cxml/dom/dom-builder.lisp 2007/10/03 15:17:08 1.14 @@ -18,7 +18,8 @@ (defclass dom-builder () ((document :initform nil :accessor document) (element-stack :initform '() :accessor element-stack) - (internal-subset :accessor internal-subset))) + (internal-subset :accessor internal-subset) + (text-buffer :initform nil :accessor text-buffer))) (defun make-dom-builder () (make-instance 'dom-builder)) @@ -87,6 +88,7 @@ (defmethod sax:start-element ((handler dom-builder) namespace-uri local-name qname attributes) (check-type qname rod) ;catch recoder/builder mismatch + (flush-characters handler) (with-slots (document element-stack) handler (let* ((nsp sax:*namespace-processing*) (element (make-instance 'element @@ -126,27 +128,45 @@ (defmethod sax:end-element ((handler dom-builder) namespace-uri local-name qname) (declare (ignore namespace-uri local-name qname)) + (flush-characters handler) (pop (element-stack handler))) (defmethod sax:characters ((handler dom-builder) data) - (with-slots (document element-stack) handler - (let* ((parent (car element-stack)) - (last-child (dom:last-child parent))) - (cond - ((eq (dom:node-type parent) :cdata-section) - (setf (dom:data parent) data)) - ((and last-child (eq (dom:node-type last-child) :text)) - ;; um entities herum wird SAX:CHARACTERS mehrfach aufgerufen fuer - ;; den gleichen Textknoten. Hier muessen wir den bestehenden Knoten - ;; erweitern, sonst ist das Dokument nicht normalisiert. - ;; (XXX Oder sollte man besser den Parser entsprechend aendern?) - (dom:append-data last-child data)) - (t - (let ((node (dom:create-text-node document data))) - (setf (slot-value node 'parent) parent) - (fast-push node (slot-value (car element-stack) 'children)))))))) + (with-slots (text-buffer) handler + (cond + ((null text-buffer) + (setf text-buffer data)) + (t + (unless (array-has-fill-pointer-p text-buffer) + (setf text-buffer (make-array (length text-buffer) + :element-type 'rune + :adjustable t + :fill-pointer t + :initial-contents text-buffer))) + (let ((n (length text-buffer)) + (m (length data))) + (adjust-vector-exponentially text-buffer (+ n m) t) + (move data text-buffer 0 n m)))))) + +(defun flush-characters (handler) + (with-slots (document element-stack text-buffer) handler + (let ((data text-buffer)) + (when data + (when (array-has-fill-pointer-p data) + (setf data + (make-array (length data) + :element-type 'rune + :initial-contents data))) + (let ((parent (car element-stack))) + (if (eq (dom:node-type parent) :cdata-section) + (setf (dom:data parent) data) + (let ((node (dom:create-text-node document data))) + (setf (slot-value node 'parent) parent) + (fast-push node (slot-value (car element-stack) 'children))))) + (setf text-buffer nil))))) (defmethod sax:start-cdata ((handler dom-builder)) + (flush-characters handler) (with-slots (document element-stack) handler (let ((node (dom:create-cdata-section document #"")) (parent (car element-stack))) @@ -155,10 +175,12 @@ (push node element-stack)))) (defmethod sax:end-cdata ((handler dom-builder)) + (flush-characters handler) (let ((node (pop (slot-value handler 'element-stack)))) (assert (eq (dom:node-type node) :cdata-section)))) (defmethod sax:processing-instruction ((handler dom-builder) target data) + (flush-characters handler) (with-slots (document element-stack) handler (let ((node (dom:create-processing-instruction document target data)) (parent (car element-stack))) @@ -166,6 +188,7 @@ (fast-push node (slot-value (car element-stack) 'children))))) (defmethod sax:comment ((handler dom-builder) data) + (flush-characters handler) (with-slots (document element-stack) handler (let ((node (dom:create-comment document data)) (parent (car element-stack))) --- /project/cxml/cvsroot/cxml/dom/dom-impl.lisp 2006/09/10 14:52:44 1.42 +++ /project/cxml/cvsroot/cxml/dom/dom-impl.lisp 2007/10/03 15:17:08 1.43 @@ -1247,7 +1247,8 @@ (push instance (element-stack handler)) #+cxml-system::utf8dom-file (setf handler (cxml:make-recoder handler #'cxml:rod-to-utf8-string)) - (funcall resolver (real-rod (dom:name instance)) handler))) + (funcall resolver (real-rod (dom:name instance)) handler) + (flush-characters handler))) (labels ((walk (n) (setf (slot-value n 'read-only-p) t) (when (dom:element-p n)

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

cxml-cvs October 2007