Hi Liam,
I quickly did the following tests on my laptop, to verify where the speed problems I've been having come from. See below for the forms and results. The outcome is that grid:aref is slow (perhaps due to consing?) and :initial-contents in make-foreign-array is sloooooooow.
A quick summary:
Setfing the values of an array inside a loop:
* With a cl-array, grid:aref is an order of magnitude slower than aref. Using grid:aref seems to cons about 4 times as much, and non-GC time is about a factor of 20 slower than just using cl aref.
* Grid:aref is more than two times as slow with a foreign array than with a cl-array. There is about 4 times as much consing as well.
Setting the values of an array using :initial-contents and a list:
* With make-array, this is blazingly fast (no time, no consing); probably because the list values are already stored properly in memory.
* With make-foreign array, this takes 50 minutes and a whole lot of consing (for a 1024x1024 array). Don't try this at home.
Basically, when using antik/grid, don't use :initial-contents. Still, setfing values in a loop is an order of magnitude slower (when no optimizations are done). I assume that if optimizations are applied, where it is known at compile-time you have a cl-array, grid:aref should be as fast as aref, and I will test this next. The thing is, though, for a generic library of data processing functions, you can't optimize for the specific array that you will use. A minor hit in performance would still be acceptable, but an order of magnitude difference for a cl-array when changing from aref to grid:aref is a bit crazy imo.
Sumant
;;;; loop-setf-test.lisp
(defun make-lists-list (rows columns) (loop for row below rows collect (loop for column below columns collect (* row column 1.0d0))))
(defun aref-loop-setf (rows columns) (let ((array (make-array (list rows columns) :element-type 'double-float))) (loop for row below rows do (loop for column below columns do (setf (aref array row column) (* row column 1.0d0)))) array))
(defun grid-aref-loop-setf (rows columns) (let ((array (make-array (list rows columns) :element-type 'double-float))) (loop for row below rows do (loop for column below columns do (setf (grid:aref array row column) (* row column 1.0d0)))) array))
(defun grid-aref-loop-setf-foreign (rows columns) (let ((array (grid:make-foreign-array 'double-float :dimensions (list rows columns)))) (loop for row below rows do (loop for column below columns do (setf (grid:aref array row column) (* row column 1.0d0)))) array))
#|
Results on my laptop are as follows.
CL-USER> (time (progn (aref-loop-setf 1024 10240) (values))) Evaluation took: 0.234 seconds of real time 0.233963 seconds of total run time (0.192970 user, 0.040993 system) [ Run times consist of 0.057 seconds GC time, and 0.177 seconds non-GC time. ] 100.00% CPU 654,633,933 processor cycles 251,658,256 bytes consed
; No value CL-USER> (time (progn (grid-aref-loop-setf 1024 10240) (values))) Evaluation took: 3.094 seconds of real time 3.084532 seconds of total run time (3.054536 user, 0.029996 system) [ Run times consist of 0.103 seconds GC time, and 2.982 seconds non-GC time. ] 99.71% CPU 8,632,811,366 processor cycles 922,738,912 bytes consed
; No value CL-USER> (time (progn (grid-aref-loop-setf-foreign 1024 10240) (values))) Evaluation took: 7.608 seconds of real time 7.591846 seconds of total run time (7.552852 user, 0.038994 system) [ Run times consist of 0.506 seconds GC time, and 7.086 seconds non-GC time. ] 99.79% CPU 21,235,314,846 processor cycles 4,026,564,512 bytes consed
; No value CL-USER> (progn (defvar *initial-contents* (make-lists-list 1024 1024)) (values))
; No value CL-USER> (time (progn (make-array (list 1024 1024) :element-type 'double-float :initial-contents *initial-contents*) (values))) Evaluation took: 0.000 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 100.00% CPU 1,456 processor cycles 0 bytes consed
; No value CL-USER> (progn (defvar *initial-contents* (make-lists-list 1024 1024)) (values))
; No value CL-USER> (time (progn (make-array (list 1024 1024) :element-type 'double-float :initial-contents *initial-contents*) (values))) Evaluation took: 0.000 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 100.00% CPU 1,002 processor cycles 0 bytes consed
; No value CL-USER> (time (progn (grid:make-foreign-array 'double-float :dimensions (list 1024 1024) :initial-contents *initial-contents*) (values))) Evaluation took: 2934.652 seconds of real time 2919.775127 seconds of total run time (2919.301199 user, 0.473928 system) [ Run times consist of 0.199 seconds GC time, and 2919.577 seconds non-GC time. ] 99.49% CPU 8,190,419,850,119 processor cycles 368,726,592 bytes consed
; No value CL-USER>
|#