Here's an example of fast reading in ABCL. The implementation of certain array types use java.nio buffers, which can be directly read into via the java.nio functions.
(defun test-read (path) (let* ((f (new 'RandomAccessFile (namestring (truename path)) "r")) (channel (#"getChannel" f)) (array (make-array (* 16 1024 1024) :element-type '(unsigned-byte 8))) (buffer (get-java-field array "elements" t))) (time (loop for count = (#"read" channel buffer) until (eql count -1) sum count do (#"position" buffer 0) ))))
On my machine, for a 5G file, the SBCL code in an earlier post takes 2.4 seconds. This code takes 1.4 sec. It's fastest if I use 2M buffers - 1.1 seconds. SBCL is also marginally faster with smaller buffer sizes.
In the ABCL source code the files "SimpleArray_*.java" are the implementations of the nio buffer backed array types. See make_array.java where the specific type of underlying array is chosen. There is a global switch for array allocation choosing either direct allocation or nio-buffers, with the default being nio buffers.
(get-java-field 'java$buffers "active" t) -> #<org.armedbear.lisp.Java$Buffers$AllocationPolicy NIO {432E958E}>
I haven't looked into the :element-type 'character case.
Maybe someone who is familiar with the ABCL stream implementation is interested in writing a fast path for read-sequence that uses the nio calls? If so, shout. Otherwise I'll keep it on my procrastinate-by-hacking-abcl list.
Alan