Here's an example of fast reading in ABCL. The implementation of certain array types use java.nio buffers, which can be directly read into via the java.nio functions.

(defun test-read (path)

(let* ((f (new 'RandomAccessFile (namestring (truename path)) "r"))
(channel (#"getChannel" f))
(array (make-array (* 16 1024 1024) :element-type '(unsigned-byte 8)))
(buffer (get-java-field array "elements" t)))
(time (loop for count = (#"read" channel buffer)
until (eql count -1)
sum count
do (#"position" buffer 0)

))))

On my machine, for a 5G file, the SBCL code in an earlier post takes 2.4 seconds. This code takes 1.4 sec. It's fastest if I use 2M buffers - 1.1 seconds. SBCL is also marginally faster with smaller buffer sizes.

In the ABCL source code the files "SimpleArray_*.java" are the implementations of the nio buffer backed array types. See make_array.java where the specific type of underlying array is chosen. There is a global switch for array allocation choosing either direct allocation or nio-buffers, with the default being nio buffers.

(get-java-field 'java$buffers "active" t)

-> #<org.armedbear.lisp.Java$Buffers$AllocationPolicy NIO {432E958E}>

I haven't looked into the :element-type 'character case.

Maybe someone who is familiar with the ABCL stream implementation is interested in writing a fast path for read-sequence that uses the nio calls? If so, shout. Otherwise I'll keep it on my procrastinate-by-hacking-abcl list.

Alan