If a byte stream s has exactly n bytes remaining, then
(read-stream-content-into-byte-vector s :initial-size n) will result in
two unnecessary copies and a total allocation of 4n bytes:
- n for the initial buffer;
- 2n when the buffer is resized and copied since r-s-c-i-b-v doesn't yet
realize that EOF has been reached;
- n when the now oversized buffer is downsized and copied at the end.
Perversely, this means that a size overestimate is more efficient than a
correct estimate. I've attached a possible patch for this below. Sadly
there is no peek-byte, so I had to opt for a rather awkward read-byte
construction; maybe someone knows a better approach.
The patch additionally gets rid of the pitfall where idioms similar to
(r-s-c-i-b-v s :initial-size (file-length s)) fail when (file-length s)
is zero, but I don't know if broadening the type of initial-size would
be considered a breaking change. The internal %length argument also
becomes basically unnecessary, but I'd bet someone out there uses it as
a workaround.
diff --git a/alexandria-1/io.lisp b/alexandria-1/io.lisp
--- a/alexandria-1/io.lisp
+++ b/alexandria-1/io.lisp
@@ -95,29 +95,35 @@ unless it's NIL, which means the system default."
(defun read-stream-content-into-byte-vector (stream &key ((%length length))
(initial-size 4096))
"Return \"content\" of STREAM as freshly allocated (unsigned-byte 8) vector."
- (check-type length (or null non-negative-integer))
- (check-type initial-size positive-integer)
- (do ((buffer (make-array (or length initial-size)
- :element-type '(unsigned-byte 8)))
- (offset 0)
- (offset-wanted 0))
- ((or (/= offset-wanted offset)
- (and length (>= offset length)))
- (if (= offset (length buffer))
- buffer
- (subseq buffer 0 offset)))
- (unless (zerop offset)
- (let ((new-buffer (make-array (* 2 (length buffer))
- :element-type '(unsigned-byte 8))))
- (replace new-buffer buffer)
- (setf buffer new-buffer)))
- (setf offset-wanted (length buffer)
- offset (read-sequence buffer stream :start offset))))
+ (check-type length (or null non-negative-integer)) ; for compatibility
+ (check-type initial-size non-negative-integer)
+ (setf initial-size (or length initial-size))
+ (let ((result (make-array initial-size :element-type '(unsigned-byte 8)))
+ (bytes-read 0))
+ (loop
+ (setf bytes-read (read-sequence result stream :start bytes-read))
+ (when (and length (>= bytes-read length))
+ (return))
+ ;; There is no PEEK-BYTE, so we just try to read a byte.
+ (let ((next-byte (read-byte stream nil nil)))
+ (when (null next-byte)
+ (return))
+ (let ((new-result (make-array (if (zerop (length result))
+ 4096
+ (* 2 (length result)))
+ :element-type '(unsigned-byte 8))))
+ (replace new-result result :end1 bytes-read :end2 bytes-read)
+ (setf (aref new-result bytes-read) next-byte
+ result new-result)
+ (incf bytes-read))))
+ (if (= bytes-read (length result))
+ result
+ (subseq result 0 bytes-read))))
(defun read-file-into-byte-vector (pathname)
"Read PATHNAME into a freshly allocated (unsigned-byte 8) vector."
(with-input-from-file (stream pathname :element-type '(unsigned-byte 8))
- (read-stream-content-into-byte-vector stream '%length (file-length stream))))
+ (read-stream-content-into-byte-vector stream :initial-size (file-length stream))))
(defun write-byte-vector-into-file (bytes pathname &key (if-exists :error)
if-does-not-exist)
