March 2012 - antik-devel - mailman3.common-lisp.net

[Antik-devel] Setting values in an array using loop and setf, and :initial-contents
by Sumant S. R. Oemrawsingh 28 Mar '12

28 Mar '12

Hi Liam, I quickly did the following tests on my laptop, to verify where the speed problems I've been having come from. See below for the forms and results. The outcome is that grid:aref is slow (perhaps due to consing?) and :initial-contents in make-foreign-array is sloooooooow. A quick summary: Setfing the values of an array inside a loop: * With a cl-array, grid:aref is an order of magnitude slower than aref. Using grid:aref seems to cons about 4 times as much, and non-GC time is about a factor of 20 slower than just using cl aref. * Grid:aref is more than two times as slow with a foreign array than with a cl-array. There is about 4 times as much consing as well. Setting the values of an array using :initial-contents and a list: * With make-array, this is blazingly fast (no time, no consing); probably because the list values are already stored properly in memory. * With make-foreign array, this takes 50 minutes and a whole lot of consing (for a 1024x1024 array). Don't try this at home. Basically, when using antik/grid, don't use :initial-contents. Still, setfing values in a loop is an order of magnitude slower (when no optimizations are done). I assume that if optimizations are applied, where it is known at compile-time you have a cl-array, grid:aref should be as fast as aref, and I will test this next. The thing is, though, for a generic library of data processing functions, you can't optimize for the specific array that you will use. A minor hit in performance would still be acceptable, but an order of magnitude difference for a cl-array when changing from aref to grid:aref is a bit crazy imo. Sumant ;;;; loop-setf-test.lisp (defun make-lists-list (rows columns) (loop for row below rows collect (loop for column below columns collect (* row column 1.0d0)))) (defun aref-loop-setf (rows columns) (let ((array (make-array (list rows columns) :element-type 'double-float))) (loop for row below rows do (loop for column below columns do (setf (aref array row column) (* row column 1.0d0)))) array)) (defun grid-aref-loop-setf (rows columns) (let ((array (make-array (list rows columns) :element-type 'double-float))) (loop for row below rows do (loop for column below columns do (setf (grid:aref array row column) (* row column 1.0d0)))) array)) (defun grid-aref-loop-setf-foreign (rows columns) (let ((array (grid:make-foreign-array 'double-float :dimensions (list rows columns)))) (loop for row below rows do (loop for column below columns do (setf (grid:aref array row column) (* row column 1.0d0)))) array)) #| Results on my laptop are as follows. CL-USER> (time (progn (aref-loop-setf 1024 10240) (values))) Evaluation took: 0.234 seconds of real time 0.233963 seconds of total run time (0.192970 user, 0.040993 system) [ Run times consist of 0.057 seconds GC time, and 0.177 seconds non-GC time. ] 100.00% CPU 654,633,933 processor cycles 251,658,256 bytes consed ; No value CL-USER> (time (progn (grid-aref-loop-setf 1024 10240) (values))) Evaluation took: 3.094 seconds of real time 3.084532 seconds of total run time (3.054536 user, 0.029996 system) [ Run times consist of 0.103 seconds GC time, and 2.982 seconds non-GC time. ] 99.71% CPU 8,632,811,366 processor cycles 922,738,912 bytes consed ; No value CL-USER> (time (progn (grid-aref-loop-setf-foreign 1024 10240) (values))) Evaluation took: 7.608 seconds of real time 7.591846 seconds of total run time (7.552852 user, 0.038994 system) [ Run times consist of 0.506 seconds GC time, and 7.086 seconds non-GC time. ] 99.79% CPU 21,235,314,846 processor cycles 4,026,564,512 bytes consed ; No value CL-USER> (progn (defvar *initial-contents* (make-lists-list 1024 1024)) (values)) ; No value CL-USER> (time (progn (make-array (list 1024 1024) :element-type 'double-float :initial-contents *initial-contents*) (values))) Evaluation took: 0.000 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 100.00% CPU 1,456 processor cycles 0 bytes consed ; No value CL-USER> (progn (defvar *initial-contents* (make-lists-list 1024 1024)) (values)) ; No value CL-USER> (time (progn (make-array (list 1024 1024) :element-type 'double-float :initial-contents *initial-contents*) (values))) Evaluation took: 0.000 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 100.00% CPU 1,002 processor cycles 0 bytes consed ; No value CL-USER> (time (progn (grid:make-foreign-array 'double-float :dimensions (list 1024 1024) :initial-contents *initial-contents*) (values))) Evaluation took: 2934.652 seconds of real time 2919.775127 seconds of total run time (2919.301199 user, 0.473928 system) [ Run times consist of 0.199 seconds GC time, and 2919.577 seconds non-GC time. ] 99.49% CPU 8,190,419,850,119 processor cycles 368,726,592 bytes consed ; No value CL-USER> |# -- Sumant Oemrawsingh

1 0

[Antik-devel] error loading gsll with quicklisp
by harven＠free.fr 22 Mar '12

22 Mar '12

Hi, As far as I understand, gsll is being merged with antik, so I am posting here. I am trying to install gsll using quicklisp and I run into an error, as reported in the attachment below. This seems to be related to cffi-grovel, which is installed. $ ls ~/.quicklisp/dists/quicklisp/software/cffi_0.10.6/grovel/asdf.lisp common.h grovel.lisp invoke.lisp package.lisp CL-USER> (find-package 'cffi-grovel) #<PACKAGE "CFFI-GROVEL"> I am on debian squeeze, using sbcl from the repos. I have installed the following packages related to gsl libgsl0-dev, libgsl0ldbl, libffi-dev, gsl-doc-info, gsl-ref-html I also tried to load gsll with asdf, but I run into the same error CL-USER> (asdf:load-system "gsll") Error while trying to load definition for system gsll from pathname /home/harven/.quicklisp/dists/quicklisp/software/gsll-20111105-git/gsll.asd: The variable #:CFFI-GROVEL is unbound. [Condition of type ASDF:LOAD-SYSTEM-DEFINITION-ERROR] Any suggestion ? Regards, -- Harven

2 1

[Antik-devel] Reader macro for sharp-m returns literal foreign-array (patch included)
by James Wright 06 Mar '12

06 Mar '12

Hi, I have encountered the following problem a couple of times before, although not as reproducibly as the memory corruption issue from my previous message: If I use the #m reader macro in a function somewhere, dump an SBCL core image, load the image, and call the function, I get memory corruption. I have mostly dealt with it up until now by not using the #m() notation in anything that I plan to dump to a binary. I believe that the cause is that the #m macro returns a foreign-array object directly (when foreign-array is the default grid type). If it returned a `make-grid' form instead, it would operate identically on e.g. the REPL. It would have the advantage, however, of being usable in a dumped image, since the array would be created at run-time instead of at read-time, and hence no foreign-pointers would be saved into the image. I've attached a patch that changes the #m reader macro to return a creation form instead of a literal. I hope you'll consider adding it. I've pasted a repl session below that demonstrates the difference. Thanks, James ;;; Before the patch GRID> (read-from-string "#m(1d0 2d0)") #m(1.000000000000000d0 2.000000000000000d0) 11 GRID> #m(1d0 2d0) #m(1.000000000000000d0 2.000000000000000d0) ;;; After the patch GRID> (read-from-string "#m(1d0 2d0)") (MAKE-GRID '((FOREIGN-ARRAY) DOUBLE-FLOAT) :INITIAL-CONTENTS '(1.0 2.0)) 11 GRID> #m(1d0 2d0) #m(1.000000000000000d0 2.000000000000000d0)

2 5

[Antik-devel] Memory corruption when loading an SBCL core containing antik (patch included)
by James Wright 06 Mar '12

06 Mar '12

Hi, When I dump an SBCL core from an image that has antik loaded, I get a memory corruption error when loading the core: *** glibc detected *** sbcl: free(): invalid pointer: 0x0808d588 *** This happens even with a minimal project that does nothing but load antik; I've attached a script to reproduce the problem. I did some investigation, and it looks like the problem is the `*formatting-test-grid*' variable in `format-grid.lisp'. It contains a list of grids, including some foreign arrays. Presumably the foreign pointers in these arrays will be pointing to random/unowned memory when the core is loaded. I've run into similar problems before in my own code (attempting to dump a global `*rng*' variable that contained a GSLL random number generator). I've attached a patch that replaces the problem `defparameter' form with a function instead. I can't find any references to this variable in the source tree, so I assume it's just there for manual testing. Thanks, James

2 4