I'm surprised about the grid:gref result. Please run a profiler and confirm that the time consumed is within grid:gref and its callees, and not something else in the gref-using code. As a side point, it seems like your use of an array is a case of premature optimization: you've concluded it will run faster by allocating an array of length two, without studying the results both ways. I think there would be no measurable difference in speed; perhaps even faster with scalars. And I disagree with you about elegance: scalars are much more understandable to me and therefore elegant than array cruft. But it's your choice and a side point anyway. Liam On Tue, Oct 12, 2010 at 5:12 AM, Sebastian Sturm <Sebastian.Sturm@itp.uni-leipzig.de> wrote:

Thank you very much for your detailed answer. I have chosen to use double-float simple-arrays because I wanted to make sure that SBCL uses unboxed double float arithmetic. I don't know if two double-float scalars might serve the same purpose here, but this seems to be a matter of elegance rather than one of efficiency. Although I would certainly like to use the convenient grid:gref construct, the computational cost is prohibitive in my opinion (unless I have misused it, which may very well be the case). Increasing the dimension 'dim' in the supplied example file to a larger but still modest value of 50, say, the fast aref version takes 3 ms and doesn't cons anything, whereas the grid:gref version takes more than 2 seconds, consing approximately 180 MB in the process. This way, it would probably be more efficient to use Mathematica, so I have to find another way to do it.

Also, I don't think I'm consing new arrays every iteration; I imagined that the arrays should be created only once (upon invocation of unusable-but-fast-force-function) and should then be retained by the lambda function that is given to the solver.

Is there some other way to directly access foreign arrays without paying the consing overhead of grid:gref? Or is it possible to trace the functions invoked by grid:gref along with their time and memory usage? That way I could at least identify the hotspot and maybe find a workaround.

Best regards, Sebastian

On 11.10.2010, at 05:08, Liam Healy wrote:

...

Typically, if you get a nil from cl-array, it means you're either on an implementation not supported by static-vectors (however you're on SBCL, which is supported), or the array was made outside of the grid:make-foreign-array function (or calls equivalent to that).   So, for example, when GSL (or any foreign code) makes an array, it will not have the CL equivalent defined.  That's a limitation of basically all the CL implementations.  The GSL minimization function creates the array to pass to the function needing minimization, so you're out of luck there.

Why not just use foreign arrays and grid:gref (instead of CL arrays and aref) everywhere?  You're not using any of the CL array functions; there's little reason to convert all arrays to CL on every iteration. Actually, now that I look at it, it seems the array you cons every iteration is just of length 2; wouldn't it be easier and faster to just use two scalars?  Though truthfully, I don't think CL arrays should be that slow.  I'm really surprised there's any speed difference.

As a side point, in your unusable-but-fast-force-function, you make two arrays in the let binding of cl-output and cl-zv which you then immediately throw away upon rebinding to the outputs of cl-array. Though this won't work anyway as I've explained above.

Liam

On Fri, Oct 8, 2010 at 11:55 AM, Liam Healy <lhealy@common-lisp.net> wrote:

...

Forwarded on behalf of Sebastian Sturm:

Dear all,

I have been trying to use gsll for some numerical minimization. Allocating the multi-dimensional-root-solver-fdf with scalarsp set to false, I understand that I should provide functions f, df and fdf which accept (pointers to) foreign-arrays (some point x in configuration space, some output vector f(x) and the corresponding Jacobian matrix J(x)) and store the corresponding results within these arrays. Being a complete Lisp newbie, I've taken a ballistic approach to optimization by liberally scattering type declarations and (the double-float )'s throughout my code.

Still, if I use grid:gref to access the given arrays, my functions have to do a lot of consing and, as a result, take a long time to complete. If I replace the grid:gref calls by (aref )'s acting on lisp arrays that have previously been set to (cl-array the-foreign-array), consing is eliminated (using (speed 3) (safety 0) (debug 0)), provided that I explicitly generate the foreign-arrays x, f(x) and J which are then passed on to f and df. Passing these optimized functions to multi-dimensional-root-solver-fdf, however, produces a crash (using speed 3 safety 0 debug 0) or an error message (speed 0 safety 3 debug 3) indicating that the cl-array slot of the function arguments is nil. It seems to me that this is default behaviour for arrays created externally by libgsl. Is this true, and if yes, is there a way to circumvent this? Or am I simply misusing grid:gref? I have attached a fairly minimal example minimization-issue.lisp. Compiling this using sbcl 1.0.43 (on Mac OS X) yields the following output:

"using grid:gref" Evaluation took: 0.003 seconds of real time 0.003419 seconds of total run time (0.003216 user, 0.000203 system) 100.00% CPU 6,943,409 processor cycles 286,496 bytes consed

#m(-0.5397677311665408d0 -1.0671897833207353d0 -1.3289868354749306d0 -1.4592646132527087d0 -1.4992646132527088d0 -1.4592646132527087d0 -1.328986835474931d0 -1.0671897833207358d0 -0.5397677311665405d0) "using aref" Evaluation took: 0.000 seconds of real time 0.000010 seconds of total run time (0.000009 user, 0.000001 system) 100.00% CPU 15,158 processor cycles 0 bytes consed

#m(-0.5397677311665408d0 -1.0671897833207353d0 -1.3289868354749306d0 -1.4592646132527087d0 -1.4992646132527088d0 -1.4592646132527087d0 -1.328986835474931d0 -1.0671897833207358d0 -0.5397677311665405d0)

Best regards, Sebastian Sturm

_______________________________________________ Gsll-devel mailing list Gsll-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel