reading s-expressions from a file
The “read” function makes really easy to read a bunch of s-expressions from a file, but how can I keep track of the line number where the expressions were in the file? Any ideia? Alexandre Sent from my iPhone
You cannot. Unless you write your own library or Quicklisp already has something for it. Besides, a S-expression is a S-expression regardless of how many “lines” it occupies. Cheers MA
On Jun 26, 2019, at 12:18 , Alexandre Rademaker <arademaker@gmail.com> wrote:
The “read” function makes really easy to read a bunch of s-expressions from a file, but how can I keep track of the line number where the expressions were in the file?
Any ideia?
Alexandre Sent from my iPhone
-- Marco Antoniotti, Associate Professor tel. +39 - 02 64 48 79 01 DISCo, Università Milano Bicocca U14 2043 http://bimib.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY Please check: http://cdac2019.lakecomoschool.org Please check: http://troncopackage.org Please check: https://www.frontiersin.org/research-topics/7394/network-bioscience Please note that I am not checking my Spam-box anymore. Please do not forward this email without asking me first (cum grano salis).
On Wed, Jun 26, 2019 at 5:19 PM Alexandre Rademaker <arademaker@gmail.com> wrote:
The “read” function makes really easy to read a bunch of s-expressions from a file, but how can I keep track of the line number where the expressions were in the file?
One way to do it is to define a stream that keeps track of the current line. Here's an example of such a stream: <https://github.com/cl-stream/cl-stream/blob/master/line-tracking-stream.lisp>. -- Luís Oliveira http://kerno.org/~luis/
Also I know about (but never used) Eclector https://github.com/robert-strandh/Eclector which I think can do what you're asking for. On Wed, Jun 26, 2019, 18:30 Luís Oliveira <luismbo@gmail.com> wrote:
On Wed, Jun 26, 2019 at 5:19 PM Alexandre Rademaker <arademaker@gmail.com> wrote:
The “read” function makes really easy to read a bunch of s-expressions from a file, but how can I keep track of the line number where the expressions were in the file?
One way to do it is to define a stream that keeps track of the current line. Here's an example of such a stream: < https://github.com/cl-stream/cl-stream/blob/master/line-tracking-stream.lisp
.
-- Luís Oliveira http://kerno.org/~luis/
This looks interesting, but the repo has no examples! Any idea how to use it? Best, Alexandre
One way to do it is to define a stream that keeps track of the current line. Here's an example of such a stream: <https://github.com/cl-stream/cl-stream/blob/master/line-tracking-stream.lisp>.
-- Luís Oliveira http://kerno.org/~luis/
If I understand correctly, this example my help http://www.chiark.greenend.org.uk/doc/sbcl-doc/html/sbcl.html#Gray-Streams-e... Regards, Ala'a On Wed, Jun 26, 2019 at 9:43 PM Alexandre Rademaker <arademaker@gmail.com> wrote:
This looks interesting, but the repo has no examples! Any idea how to use it?
Best, Alexandre
One way to do it is to define a stream that keeps track of the current line. Here's an example of such a stream: <https://github.com/cl-stream/cl-stream/blob/master/line-tracking-stream.lisp>.
-- Luís Oliveira http://kerno.org/~luis/
The first thing to realize is that "you" cannot do this, but you might be able to write CL-conformant code that does it for you. But the READ function won't do it by itself. It's not part of READ's contract, which silently eats Newlines without counting them for you. You might consider some readtable magic, changing Newline to a non-non-breaking macro char that returns nothing but maintains a count somewhere, except such hackery would miss line breaks inside strings, #|comments|#, probably also ;regular comments, and escaped newlines inside symbol names (UGH). Rather than writing your own entire READ function, if the source is a regular static file that can be reopened multiple times, wrtie a simple alternative MY-READ function that calls FILE-POSITION before calling READ and returns both a multiple values. Converting a character FILE-POSITION into a line number, when and if necessary, can be accomplished by reopening the file and reading characters until the desired FILE-POSITION counting line breaks along the way. Taking care to use the same external-format avoids the difficulty of multi-byte characters and the possibility that FILE-POSITION does not necessarily increment by 1 for each char. It is only guaranteed to increase monotonically. On Wed, Jun 26, 2019 at 9:19 AM Alexandre Rademaker <arademaker@gmail.com> wrote:
The “read” function makes really easy to read a bunch of s-expressions from a file, but how can I keep track of the line number where the expressions were in the file?
Any ideia?
Alexandre Sent from my iPhone
Thank you all! Reporting my results so far, suggestions are welcome! ;-) My goal is to read KIF files such as https://github.com/ontologyportal/sumo/blob/master/Merge.kif Alessio's suggestion is probably something to explorer, but I will need some time to read the documentation of https://github.com/robert-strandh/Eclector. The most directly solution seems to be the suggestion from Luís Oliveira, but I didn’t understand how to use https://github.com/cl-stream/cl-stream/blob/master/line-tracking-stream.lisp. The Ala'a Mohammad link (http://www.chiark.greenend.org.uk/doc/sbcl-doc/html/sbcl.html#Gray-Streams-e...) helped me. But if I understood it right, cl-stream implements only the equivalent of read-char from CL in its stream-read function. Moreover, I didn’t yet figure out how to initialise the ’state’ of the class. CL-USER> (with-open-file (input "/Users/ar/workspace/sumo/Merge.kif") (let ((counted-stream (make-instance 'cl-stream::line-tracking-input-stream :stream input :input-line 0))) (dotimes (x 20) (print (multiple-value-list (cl-stream:stream-read counted-stream)))))) (#\; NIL) (#\; NIL) (#\ NIL) (#\= NIL) (#\= NIL) (#\= NIL) ... Regarding Steve message:
On 26 Jun 2019, at 13:49, Steve Haflich <shaflich@gmail.com> wrote:
The first thing to realize is that "you" cannot do this, but you might be able to write CL-conformant code that does it for you. But the READ function won't do it by itself. It's not part of READ's contract, which silently eats Newlines without counting them for you.
Yep, that is what I discovered reading the http://clhs.lisp.se/Body/02_b.htm.
You might consider some readtable magic, changing Newline to a non-non-breaking macro char that returns nothing but maintains a count somewhere, except such hackery would miss line breaks inside strings, #|comments|#, probably also ;regular comments, and escaped newlines inside symbol names (UGH).
I was trying - without success - this approach. Maybe I didn’t understand how to `change Newline to non-non-breaking macro’. On the other hand, as Steve said, it looks like the line breaks in commented lines are ignored. I was trying (defparameter *line* 0) (defun linebreak-reader (stream char) (declare (ignore char)) (incf *line*) (read-preserving-whitespace stream t nil t)) CL-USER> (let ((*readtable* (copy-readtable))) (setf *line* 0) (set-macro-character #\Newline #'linebreak-reader) (with-open-file (in "~/workspace/sumo/Merge.kif") (dotimes (x 10) (format t "~a ~s ~%" *line* (read-preserving-whitespace in nil nil))))) 0 (INSTANCE INSTANCE BINARYPREDICATE) 20 (DOMAIN INSTANCE 1 ENTITY) 21 (DOMAIN INSTANCE 2 SETORCLASS) ...
Rather than writing your own entire READ function, if the source is a regular static file that can be reopened multiple times, wrtie a simple alternative MY-READ function that calls FILE-POSITION before calling READ and returns both a multiple values. Converting a character FILE-POSITION into a line number, when and if necessary, can be accomplished by reopening the file and reading characters until the desired FILE-POSITION counting line breaks along the way. Taking care to use the same external-format avoids the difficulty of multi-byte characters and the possibility that FILE-POSITION does not necessarily increment by 1 for each char. It is only guaranteed to increase monotonically.
I am still have to fix some details like dealing with the first expression, improve performance using a smarter strategy to consume the linebreaks position list etc. But here is a first version. (defstruct expression position line form) (defun my-read (stream) (cons (file-position stream) (read stream nil nil))) (defun pos-line (position linebreaks) (position-if-not (lambda (n) (> position n)) linebreaks)) (defun linebreaks-in-file (file) (with-open-file (kb file) (loop for c = (read-char kb nil 'the-end) while (characterp c) when (equal c #\Newline) collect (file-position kb)))) (defun read-kif (file) (let ((res nil) (linebreaks (linebreaks-in-file file))) (with-open-file (kb file) (do ((st (my-read kb) (my-read kb))) ((null (cdr st)) (reverse res)) (push (make-expression :position (car st) :line (pos-line (car st) linebreaks) :form (cdr st)) res))))) (read-kif "~/workspace/sumo/Merge.kif”) -- Alexandre Rademaker http://arademaker.github.io
On Wed, Jun 26, 2019 at 9:19 AM Alexandre Rademaker <arademaker@gmail.com> wrote:
The “read” function makes really easy to read a bunch of s-expressions from a file, but how can I keep track of the line number where the expressions were in the file?
Any ideia?
Alexandre Sent from my iPhone
participants (6)
-
Ala'a Mohammad -
Alessio Stalla -
Alexandre Rademaker -
Luís Oliveira -
Marco Antoniotti -
Steve Haflich