Hi,
I ran into a problem making foreign arrays. I have a two lists of lists of floats called *data-50* and *data-1024*, which are to be used as the initial contents of a matrix (first is 50x50, second 1024x1024). I do the following:
CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-50*))) Evaluation took: 0.000 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 100.00% CPU 228,096 processor cycles 27,744 bytes consed
(SIMPLE-ARRAY SINGLE-FLOAT (50 50)) CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-50*))) Evaluation took: 0.036 seconds of real time 0.026995 seconds of total run time (0.026995 user, 0.000000 system) 75.00% CPU 79,104,003 processor cycles 294,688 bytes consed
MATRIX-SINGLE-FLOAT
With a 50x50 array, everything works out fine. However, when moving to the 1024x1024 data:
CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-1024*))) Evaluation took: 0.036 seconds of real time 0.035995 seconds of total run time (0.034995 user, 0.001000 system) 100.00% CPU 80,301,650 processor cycles 4,194,320 bytes consed
(SIMPLE-ARRAY SINGLE-FLOAT (1024 1024)) CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-1024*))) Evaluation took: 686.167 seconds of real time 683.878034 seconds of total run time (683.608075 user, 0.269959 system) [ Run times consist of 0.196 seconds GC time, and 683.683 seconds non-GC time. ] 99.67% CPU 3 forms interpreted 1,502,208,488,440 processor cycles 59,838,352 bytes consed
before it was aborted by a non-local transfer of control.
; Evaluation aborted on NIL.
After over 10 minutes of intensive work, I gave up and aborted. When using make-foreign-array directly, instead of make-grid, I get the same problem. I would understand that foreign arrays take a bit more time to make; in the *data-50* example, about an order of magnitude more bytes were consed to make the foreign array (and needing more processor cycles).
While this is not very thorough, I would expect there to be some linear relation between the array size and the time it takes. E.g. 50x50 foreign array takes less than a 0.1 second, so I would expect 1024x1024 to take less than 42.0 seconds. While that is still a long time for a normal-sized array, clearly, this is not the case. I can run some more tests later to find out where the problem lies. Or is this a known bug or limitation for foreign arrays and sbcl? Or am I just doing something wrong here?
Thanks, Sumant
Sumant,
I think this has something to do with the initial contents (time (type-of (make-foreign-array 'double-float :dimensions '(1024 1024)))) Evaluation took: 0.001 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 0.00% CPU 341,884 processor cycles 0 bytes consed
MATRIX-DOUBLE-FLOAT Seems pretty snappy. Generally, I'm not too happy with the idea of setting values from a list, at least for large lists. I tried to find an alternate way to do this, but I also wanted to mimic the arguments to make-array, so that's why it's in there. I presume that your test is contrived and that you wouldn't really make a list of 1 million elements and then load it into a foreign array. Can you try a problem where you're not using :initial-contents? If you don't like to make an array with indeterminate contents, try this:
(time (type-of (map-grid :source '* :source-dims '(1024 1024) :destination-specification '((foreign-array 1024 1024) double-float)))) Evaluation took: 2.487 seconds of real time 2.490000 seconds of total run time (2.490000 user, 0.000000 system) [ Run times consist of 0.220 seconds GC time, and 2.270 seconds non-GC time. ] 100.12% CPU 5,292,435,364 processor cycles 268,462,848 bytes consed
MATRIX-DOUBLE-FLOAT
(By the way, all make-foreign-array does is call make-grid, i.e., it's less "direct" than make-grid.)
Liam
On Wed, Apr 27, 2011 at 9:31 AM, Sumant Oemrawsingh soemraws@xs4all.nlwrote:
Hi,
I ran into a problem making foreign arrays. I have a two lists of lists of floats called *data-50* and *data-1024*, which are to be used as the initial contents of a matrix (first is 50x50, second 1024x1024). I do the following:
CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-50*))) Evaluation took: 0.000 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 100.00% CPU 228,096 processor cycles 27,744 bytes consed
(SIMPLE-ARRAY SINGLE-FLOAT (50 50)) CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-50*))) Evaluation took: 0.036 seconds of real time 0.026995 seconds of total run time (0.026995 user, 0.000000 system) 75.00% CPU 79,104,003 processor cycles 294,688 bytes consed
MATRIX-SINGLE-FLOAT
With a 50x50 array, everything works out fine. However, when moving to the 1024x1024 data:
CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-1024*))) Evaluation took: 0.036 seconds of real time 0.035995 seconds of total run time (0.034995 user, 0.001000 system) 100.00% CPU 80,301,650 processor cycles 4,194,320 bytes consed
(SIMPLE-ARRAY SINGLE-FLOAT (1024 1024)) CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-1024*))) Evaluation took: 686.167 seconds of real time 683.878034 seconds of total run time (683.608075 user, 0.269959 system) [ Run times consist of 0.196 seconds GC time, and 683.683 seconds non-GC time. ] 99.67% CPU 3 forms interpreted 1,502,208,488,440 processor cycles 59,838,352 bytes consed
before it was aborted by a non-local transfer of control.
; Evaluation aborted on NIL.
After over 10 minutes of intensive work, I gave up and aborted. When using make-foreign-array directly, instead of make-grid, I get the same problem. I would understand that foreign arrays take a bit more time to make; in the *data-50* example, about an order of magnitude more bytes were consed to make the foreign array (and needing more processor cycles).
While this is not very thorough, I would expect there to be some linear relation between the array size and the time it takes. E.g. 50x50 foreign array takes less than a 0.1 second, so I would expect 1024x1024 to take less than 42.0 seconds. While that is still a long time for a normal-sized array, clearly, this is not the case. I can run some more tests later to find out where the problem lies. Or is this a known bug or limitation for foreign arrays and sbcl? Or am I just doing something wrong here?
Thanks, Sumant
-- Sumant Oemrawsingh
GSLL-devel mailing list GSLL-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
Hi Liam,
Yes, this example is quite contrived, and I don't use :initial-contents often. I thought it was worth mentioning because making lisp and foreign arrays using that keyword differs by _so_ much time, and I didn't quite understand why.
Thanks, Sumant
On Sat, Apr 30, 2011 at 11:47:54PM -0400, Liam Healy wrote:
Sumant, I think this has something to do with the initial contents (time (type-of (make-foreign-array 'double-float :dimensions '(1024 1024)))) Evaluation took: 0.001 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 0.00% CPU 341,884 processor cycles 0 bytes consed
MATRIX-DOUBLE-FLOAT Seems pretty snappy. Generally, I'm not too happy with the idea of setting values from a list, at least for large lists. I tried to find an alternate way to do this, but I also wanted to mimic the arguments to make-array, so that's why it's in there. I presume that your test is contrived and that you wouldn't really make a list of 1 million elements and then load it into a foreign array. Can you try a problem where you're not using :initial-contents? If you don't like to make an array with indeterminate contents, try this: (time (type-of (map-grid :source '* :source-dims '(1024 1024) :destination-specification '((foreign-array 1024 1024) double-float)))) Evaluation took: 2.487 seconds of real time 2.490000 seconds of total run time (2.490000 user, 0.000000 system) [ Run times consist of 0.220 seconds GC time, and 2.270 seconds non-GC time. ] 100.12% CPU 5,292,435,364 processor cycles 268,462,848 bytes consed
MATRIX-DOUBLE-FLOAT (By the way, all make-foreign-array does is call make-grid, i.e., it's less "direct" than make-grid.) Liam
On Wed, Apr 27, 2011 at 9:31 AM, Sumant Oemrawsingh <[1]soemraws@xs4all.nl> wrote:
Hi, I ran into a problem making foreign arrays. I have a two lists of lists of floats called *data-50* and *data-1024*, which are to be used as the initial contents of a matrix (first is 50x50, second 1024x1024). I do the following: CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-50*))) Evaluation took: 0.000 seconds of real time 0.000000 seconds of total run time (0.000000 user, 0.000000 system) 100.00% CPU 228,096 processor cycles 27,744 bytes consed (SIMPLE-ARRAY SINGLE-FLOAT (50 50)) CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-50*))) Evaluation took: 0.036 seconds of real time 0.026995 seconds of total run time (0.026995 user, 0.000000 system) 75.00% CPU 79,104,003 processor cycles 294,688 bytes consed MATRIX-SINGLE-FLOAT With a 50x50 array, everything works out fine. However, when moving to the 1024x1024 data: CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-1024*))) Evaluation took: 0.036 seconds of real time 0.035995 seconds of total run time (0.034995 user, 0.001000 system) 100.00% CPU 80,301,650 processor cycles 4,194,320 bytes consed (SIMPLE-ARRAY SINGLE-FLOAT (1024 1024)) CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-1024*))) Evaluation took: 686.167 seconds of real time 683.878034 seconds of total run time (683.608075 user, 0.269959 system) [ Run times consist of 0.196 seconds GC time, and 683.683 seconds non-GC time. ] 99.67% CPU 3 forms interpreted 1,502,208,488,440 processor cycles 59,838,352 bytes consed before it was aborted by a non-local transfer of control. ; Evaluation aborted on NIL. After over 10 minutes of intensive work, I gave up and aborted. When using make-foreign-array directly, instead of make-grid, I get the same problem. I would understand that foreign arrays take a bit more time to make; in the *data-50* example, about an order of magnitude more bytes were consed to make the foreign array (and needing more processor cycles). While this is not very thorough, I would expect there to be some linear relation between the array size and the time it takes. E.g. 50x50 foreign array takes less than a 0.1 second, so I would expect 1024x1024 to take less than 42.0 seconds. While that is still a long time for a normal-sized array, clearly, this is not the case. I can run some more tests later to find out where the problem lies. Or is this a known bug or limitation for foreign arrays and sbcl? Or am I just doing something wrong here? Thanks, Sumant -- Sumant Oemrawsingh _______________________________________________ GSLL-devel mailing list [2]GSLL-devel@common-lisp.net [3]http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-develReferences
- mailto:soemraws@xs4all.nl
- mailto:GSLL-devel@common-lisp.net
- http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
GSLL-devel mailing list GSLL-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
-
Liam Healy -
Sumant Oemrawsingh