gsll-devel October 2010

gsll-devel@common-lisp.net

3 participants
9 discussions

[Gsll-devel] Allegro CL 8.2 on 10.5?
by Vipul Periwal 19 Nov '10

19 Nov '10

Has anyone succeeded in building GSLL for Allegro CL 8.2 on 10.5? I'm asking because I've been fixing a bunch of things to get pieces built one by one and it occurs to me that there is probably something wrong with what I'm doing. The last spot was getting through the libgsl- unix.lisp compilation where the appropriate compiler flag (-I/opt/ local/include/) was not being passed to cc. And then the resulting output has lower-case +edom+ etc. gslerrorno's while conditions.lisp is expecting +EDOM+ etc. I'm using mlisp so this last would probably not be a problem with alisp, but I hope someone can tell me of a workaround, or tell me to give up. Thanks. Vipul Periwal

2 1

Re: [GSLL-devel] [Gsll-devel] Efficient access to externally generated double-float arrays?
by Sebastian Sturm 15 Nov '10

15 Nov '10

Here is something similar to what I suggested, although still very unfinished. It's probably not the proper way to write a macro, but I guess other people on the list will know how to do it correctly. Efficient linearization can be added in the same manner; also, the macro should be modified s.t. it accepts several array specifications at once, i.e. with-foreign-array ((array-1 :double) (array-2 :int) (array-3 :double) ...), etc. Of course, if you can figure out a way to incorporate the optimizations into gref without such a clumsy workaround, I'd be all for it. best regards, Sebastian (defun mapcons (fn x) (if (atom x) x (funcall fn (let ((a (mapcons fn (car x))) (d (mapcons fn (cdr x)))) (if (and (eql a (car x)) (eql d (cdr x))) x (cons a d)))))) (defmacro with-fast-access-to-single-foreign-array ((array element-type) &body body) (alexandria:with-unique-names (array-fptr) `(let ((,array-fptr (grid::foreign-pointer ,array))) ,@(mapcons (lambda (expr) (if (and (consp expr) (eq (first expr) 'grid:gref*) (eq (second expr) array)) (list 'cffi:mem-aref array-fptr element-type (elt expr 2)) expr)) body)))) (defun macro-force-function (dim) "Given an integer dim, this constructs a function that, when supplied with a N-dimensional vector Z and some output vector (-> pointer?), yields the corresponding forces" (declare (fixnum dim)) (let ((temp-values (make-array 2 :element-type 'double-float :initial-element 0.0d0))) (lambda (zvector output) (with-fast-access-to-single-foreign-array (output :double) (with-fast-access-to-single-foreign-array (zvector :double) (do ((i 0 (1+ i))) ((= i dim)) (declare (fixnum i)) (setf (aref temp-values 0) 0.0d0) (do ((m 0 (1+ m))) ((> m i)) (declare (fixnum m)) (do ((n i (1+ n))) ((= n dim)) (declare (fixnum n)) (setf (aref temp-values 1) 0.0d0) (do ((k m (1+ k))) ((> k n)) (declare (fixnum k)) (incf (aref temp-values 1) (grid:gref* zvector k))) (incf (aref temp-values 0) (expt (aref temp-values 1) -2)))) (setf (grid:gref* output i) (- (grid:gref* zvector i) (aref temp-values 0)))))))))

2 6

Re: [GSLL-devel] [Gsll-devel] Efficient access to externally generated double-float arrays?
by Sebastian Sturm 28 Oct '10

28 Oct '10

Liam, I have tried this and (without using (the double-float (...)) since this is unnecessary for cffi:mem-aref with hardcoded :double data type), I get the following timing data: using the standard grid:gref routine: Evaluation took: 0.542 seconds of real time 0.536671 seconds of total run time (0.525229 user, 0.011442 system) [ Run times consist of 0.037 seconds GC time, and 0.500 seconds non-GC time. ] 99.08% CPU 1,189,729,211 processor cycles 153,334,416 bytes consed using the standard grid:gref* routine: Evaluation took: 0.067 seconds of real time 0.066006 seconds of total run time (0.063179 user, 0.002827 system) [ Run times consist of 0.008 seconds GC time, and 0.059 seconds non-GC time. ] 98.51% CPU 146,192,222 processor cycles 27,060,416 bytes consed using the modified grid:gref* routine (specialized to grid:vector-double-float) Evaluation took: 0.070 seconds of real time 0.068041 seconds of total run time (0.065151 user, 0.002890 system) [ Run times consist of 0.011 seconds GC time, and 0.058 seconds non-GC time. ] 97.14% CPU 152,642,611 processor cycles 27,061,456 bytes consed Apparently my system didn't notice the difference. Also, SBCL complains about the argument (grid::foreign-pointer object) to cffi:mem-aref being of type NUMBER instead of integer or fixnum. Furthermore, it says it has to do float to pointer coercion to <return-value>. I checked that the redefined method is actually used by doing a second run with a print statement added to the defmethod. using cffi:mem-aref directly: Evaluation took: 0.002 seconds of real time 0.002483 seconds of total run time (0.002482 user, 0.000001 system) 100.00% CPU 5,442,756 processor cycles 0 bytes consed I guess that even using compiler macros or other trickery one would have to remove the allocation of linearized indices and foreign ptr addresses from the inner loops as I have done in my example by using auxiliary variables zvector-fptr and output-fptr. Maybe one can define something like (with-foreign-array (name-of-array :double) ...) that locally redefines (grid:gref name-of-array ...) and (grid:gref* name-of-array ...) as macros evaluating to cffi:mem-aref and storing the respective linearized indices and memory pointers at the level of with-foreign-array? Although not as convenient as some 'self-optimizing' grid:gref, I would consider this a satisfactory solution. Don't know how to do that without getting lost in a forest of commas and backquotes, though. best regards, Sebastian On 27.10.2010, at 05:22, Liam Healy wrote: > Sebastian, > > Can you temporarily define this and find the timing/consing for your > test case: > > (defmethod gref* ((object vector-double-float) linearized-index) > (cffi:mem-aref > (foreign-pointer object) > :double > linearized-index)) > > (I think you don't use any matrices but if you do, define an analogous > function for matrix-double-float.) > > As you can see, it has the literal type declaration, and I'm hopeful > that CFFI will pick that up and make this competitive in speed with > the best that you saw. If that's so, it should be fairly easy for me > to make this generic and incorporate it into GSD. I'm still > interested in making the linearization more efficient if that's still > significant, but let's try this for now to see how much speed we can > squeeze out of gref*. > > Thanks, > > Liam

2 1

Re: [GSLL-devel] [Gsll-devel] Efficient access to externally generated double-float arrays?
by Liam Healy 27 Oct '10

27 Oct '10

Sebastian, Can you temporarily define this and find the timing/consing for your test case: (defmethod gref* ((object vector-double-float) linearized-index) (cffi:mem-aref (foreign-pointer object) :double linearized-index)) (I think you don't use any matrices but if you do, define an analogous function for matrix-double-float.) As you can see, it has the literal type declaration, and I'm hopeful that CFFI will pick that up and make this competitive in speed with the best that you saw. If that's so, it should be fairly easy for me to make this generic and incorporate it into GSD. I'm still interested in making the linearization more efficient if that's still significant, but let's try this for now to see how much speed we can squeeze out of gref*. Thanks, Liam On Tue, Oct 26, 2010 at 10:25 AM, Sebastian Sturm <Sebastian.Sturm(a)itp.uni-leipzig.de> wrote: > It seems that CFFI includes some compiler macros that use type information > supplied at compile time to generate more efficient code (got that from the > cffi mailing > list, http://www.mail-archive.com/cffi-devel@common-lisp.net/msg01154.html). > In my case, I'm using this optimization by supplying :double to > cffi:mem-aref. If I replace this by (cl-cffi (element-type zvector)), as is > done internally by gref, then (again with dim = 50), better-force-function > uses around 1.8 GCycles and conses 80 MB in the process, whereas the :double > version needs ~ 8.6 MCycles, not consing anything. The slow-but-flexible > version of better-force-function reads as follows: > (defun better-force-function (dim) > "Given an integer dim, this constructs a function that, when supplied with > a > N-dimensional vector Z and some output vector (-> pointer?), yields the > corresponding forces" > (declare (fixnum dim)) > (let ((temp-values (make-array 2 :element-type 'double-float > :initial-element 0.0d0))) > (lambda (zvector output) > (let ((zvector-fptr (grid::foreign-pointer zvector)) > (output-fptr (grid::foreign-pointer output)) > ;; this makes it worse > (elt-type (grid:cl-cffi (grid:element-type zvector))) > ) > (macrolet ((quick-ref (the-vector n) > `(cffi:mem-aref > ,(case the-vector > (zvector 'zvector-fptr) > (output 'output-fptr)) > ;; :double > elt-type ;; replace this by :double > ,n))) > (do ((i 0 (1+ i))) ((= i dim)) (declare (fixnum i)) > (setf (aref temp-values 0) 0.0d0) > (do ((m 0 (1+ m))) ((> m i)) (declare (fixnum m)) > (do ((n i (1+ n))) ((= n dim)) (declare (fixnum n)) > (setf (aref temp-values 1) 0.0d0) > (do ((k m (1+ k))) ((> k n)) (declare (fixnum k)) > (incf (aref temp-values 1) (quick-ref zvector k))) ;; generates efficiency > warnings when using elt-type > (incf (aref temp-values 0) (expt (aref temp-values 1) -2)))) > (setf (quick-ref output i) > (- (quick-ref zvector i) > (aref temp-values 0))))))))) > Also, with the variable type left unspecified at compile time, the innermost > loop generates efficiency warnings telling me that generic-+ needs to be > used. Writing (the double-float (quick-ref zvector k)) removes these and > slightly reduces the consing amount of the slow variant to ~ 63 MB. I still > have to try the SLIME profiler though. > thanks, > Sebastian

1 0

Re: [GSLL-devel] [Gsll-devel] Efficient access to externally generated double-float arrays?
by Liam Healy 26 Oct '10

26 Oct '10

On Fri, Oct 22, 2010 at 2:13 PM, Sebastian Sturm <Sebastian.Sturm(a)itp.uni-leipzig.de> wrote: >> I'm surprised about the grid:gref result. Please run a profiler and >> confirm that the time consumed is within grid:gref and its callees, > > I tried using sb-sprof, but as a complete CL newbie I had a hard time making sense of its output and thus ended up ignoring the profiler and using trial-and-error again. It seems that a fair amount of consing is due to the index linearization. Simply replacing grid:gref by grid:gref* for one-dimensional arrays reduces consing by a factor of 3 in my experiments. It seems that the rest can be eliminated by specifying the data type (here it's :double) at compile-time; incorporating this as an option to grid:gref via some macro-trickery would probably uglify the GSLL framework somewhat (I guess?), but I'd be willing to pay that price in return for an order-of-magnitude speedup. Also, I haven't tried if similar problems arise (and if similar workarounds are available) for the case of complex-valued arrays. Interesting. It's plausible to me that linearization is causing the problem; I think that there is room for improvement there. Your example with the mem-aref show that it's got to be that and/or generic function dispatch, because as you can see on line 35 of http://repo.or.cz/w/gsd.git/blob/e537bd9be551b90c9fbfb6020a6119f3df00d650:/…, gref* essentially just calls mem-aref. So maybe if we can add declarations and inlining, we can get the same effect with the current code. If you can't make sense of sb-prof and you use slime, you might try the slime profiler: http://lhealy.livejournal.com/8495.html I find its output much clearer. I'm not sure where you are putting a :double declaration to get that speedup; can you post some example code? It's possible that there is some loss of effiicency in using CLOS (generic function dispatch) that could be remedied by declarations or compiler macros. That's beyond my realm of expertise, but I'm willing to dig into it. Liam

1 0

[Gsll-devel] Fwd: Efficient access to externally generated double-float arrays?
by Liam Healy 22 Oct '10

22 Oct '10

---------- Forwarded message ---------- From: Sebastian Sturm <Sebastian.Sturm(a)itp.uni-leipzig.de> Date: Fri, Oct 22, 2010 at 2:13 PM Subject: Re: [Gsll-devel] Efficient access to externally generated double-float arrays? To: Liam Healy <lhealy(a)common-lisp.net>, gsll-devel(a)common-lisp.net > I'm surprised about the grid:gref result. Please run a profiler and > confirm that the time consumed is within grid:gref and its callees, I tried using sb-sprof, but as a complete CL newbie I had a hard time making sense of its output and thus ended up ignoring the profiler and using trial-and-error again. It seems that a fair amount of consing is due to the index linearization. Simply replacing grid:gref by grid:gref* for one-dimensional arrays reduces consing by a factor of 3 in my experiments. It seems that the rest can be eliminated by specifying the data type (here it's :double) at compile-time; incorporating this as an option to grid:gref via some macro-trickery would probably uglify the GSLL framework somewhat (I guess?), but I'd be willing to pay that price in return for an order-of-magnitude speedup. Also, I haven't tried if similar problems arise (and if similar workarounds are available) for the case of complex-valued arrays. That being said, I'm currently using the following force-function (where caching the foreign-ptrs is irrelevant with respect to consing, but reduces the amount of cycles needed to access array elements by a factor of ~ 5): (defun better-force-function (dim) "Given an integer dim, this constructs a function that, when supplied with a N-dimensional vector Z and some output vector (-> pointer?), yields the corresponding forces" (declare (fixnum dim)) (let ((temp-values (make-array 2 :element-type 'double-float :initial-element 0.0d0))) (lambda (zvector output) (let ((zvector-fptr (grid::foreign-pointer zvector)) (output-fptr (grid::foreign-pointer output))) (macrolet ((quick-ref (the-vector n) `(cffi:mem-aref ,(case the-vector (zvector 'zvector-fptr) (output 'output-fptr)) :double ,n))) (do ((i 0 (1+ i))) ((= i dim)) (declare (fixnum i)) (setf (aref temp-values 0) 0.0d0) (do ((m 0 (1+ m))) ((> m i)) (declare (fixnum m)) (do ((n i (1+ n))) ((= n dim)) (declare (fixnum n)) (setf (aref temp-values 1) 0.0d0) (do ((k m (1+ k))) ((> k n)) (declare (fixnum k)) (incf (aref temp-values 1) (quick-ref zvector k))) (incf (aref temp-values 0) (expt (aref temp-values 1) -2)))) (setf (quick-ref output i) (- (quick-ref zvector i) (aref temp-values 0))))))))) Using the timing code from my first post and dim set to 50, I get the following results: - using plain-vanilla grid:gref: Evaluation took: 2.692 seconds of real time 2.668223 seconds of total run time (2.624302 user, 0.043921 system) [ Run times consist of 0.082 seconds GC time, and 2.587 seconds non-GC time. ] 99.11% CPU 5,906,443,268 processor cycles 189,391,360 bytes consed - using the unpleasant cffi:mem-aref workaround: Evaluation took: 0.004 seconds of real time 0.003107 seconds of total run time (0.003049 user, 0.000058 system) 75.00% CPU 6,997,397 processor cycles 0 bytes consed > As a side point, it seems like your use of an array is a case of > premature optimization: you've concluded it will run faster by > allocating an array of length two, without studying the results both > ways. I think there would be no measurable difference in speed; > perhaps even faster with scalars. And I disagree with you about > elegance: scalars are much more understandable to me and therefore > elegant than array cruft. But it's your choice and a side point > anyway. Concerning the last point, I fully agree with you; probably my previous post was somewhat unclear. Accessing certain unnamed elements of some arbitrary temp-array is probably the most ugly solution one could think of, but it seems to be a simple way to ensure low overhead (I took the idea from a paper by Fateman). Since it doesn't really relate to the problem at hand, I figured that I could just leave it at that until the more pressing issue is resolved. Still, I appreciate very much any advice on how to further improve speed and/or clarity. Thanks alot for your help, Sebastian

1 0

Re: [Gsll-devel] Efficient access to externally generated double-float arrays?
by Liam Healy 21 Oct '10

21 Oct '10

I'm surprised about the grid:gref result. Please run a profiler and confirm that the time consumed is within grid:gref and its callees, and not something else in the gref-using code. As a side point, it seems like your use of an array is a case of premature optimization: you've concluded it will run faster by allocating an array of length two, without studying the results both ways. I think there would be no measurable difference in speed; perhaps even faster with scalars. And I disagree with you about elegance: scalars are much more understandable to me and therefore elegant than array cruft. But it's your choice and a side point anyway. Liam On Tue, Oct 12, 2010 at 5:12 AM, Sebastian Sturm <Sebastian.Sturm(a)itp.uni-leipzig.de> wrote: > Thank you very much for your detailed answer. I have chosen to use double-float simple-arrays because I wanted to make sure that SBCL uses unboxed double float arithmetic. I don't know if two double-float scalars might serve the same purpose here, but this seems to be a matter of elegance rather than one of efficiency. Although I would certainly like to use the convenient grid:gref construct, the computational cost is prohibitive in my opinion (unless I have misused it, which may very well be the case). Increasing the dimension 'dim' in the supplied example file to a larger but still modest value of 50, say, the fast aref version takes 3 ms and doesn't cons anything, whereas the grid:gref version takes more than 2 seconds, consing approximately 180 MB in the process. This way, it would probably be more efficient to use Mathematica, so I have to find another way to do it. > > Also, I don't think I'm consing new arrays every iteration; I imagined that the arrays should be created only once (upon invocation of unusable-but-fast-force-function) and should then be retained by the lambda function that is given to the solver. > > Is there some other way to directly access foreign arrays without paying the consing overhead of grid:gref? Or is it possible to trace the functions invoked by grid:gref along with their time and memory usage? That way I could at least identify the hotspot and maybe find a workaround. > > Best regards, > Sebastian > > On 11.10.2010, at 05:08, Liam Healy wrote: > >> Typically, if you get a nil from cl-array, it means you're either on >> an implementation not supported by static-vectors (however you're on >> SBCL, which is supported), or the array was made outside of the >> grid:make-foreign-array function (or calls equivalent to that). So, >> for example, when GSL (or any foreign code) makes an array, it will >> not have the CL equivalent defined. That's a limitation of basically >> all the CL implementations. The GSL minimization function creates the >> array to pass to the function needing minimization, so you're out of >> luck there. >> >> Why not just use foreign arrays and grid:gref (instead of CL arrays >> and aref) everywhere? You're not using any of the CL array functions; >> there's little reason to convert all arrays to CL on every iteration. >> Actually, now that I look at it, it seems the array you cons every >> iteration is just of length 2; wouldn't it be easier and faster to >> just use two scalars? Though truthfully, I don't think CL arrays >> should be that slow. I'm really surprised there's any speed >> difference. >> >> As a side point, in your unusable-but-fast-force-function, you make >> two arrays in the let binding of cl-output and cl-zv which you then >> immediately throw away upon rebinding to the outputs of cl-array. >> Though this won't work anyway as I've explained above. >> >> Liam >> >> On Fri, Oct 8, 2010 at 11:55 AM, Liam Healy <lhealy(a)common-lisp.net> wrote: >>> Forwarded on behalf of Sebastian Sturm: >>> >>> Dear all, >>> >>> I have been trying to use gsll for some numerical minimization. >>> Allocating the multi-dimensional-root-solver-fdf with scalarsp set to >>> false, I understand that I should provide functions f, df and fdf >>> which accept (pointers to) foreign-arrays (some point x in >>> configuration space, some output vector f(x) and the corresponding >>> Jacobian matrix J(x)) and store the corresponding results within these >>> arrays. Being a complete Lisp newbie, I've taken a ballistic approach >>> to optimization by liberally scattering type declarations and (the >>> double-float )'s throughout my code. >>> >>> Still, if I use grid:gref to access the given arrays, my functions >>> have to do a lot of consing and, as a result, take a long time to >>> complete. If I replace the grid:gref calls by (aref )'s acting on lisp >>> arrays that have previously been set to (cl-array the-foreign-array), >>> consing is eliminated (using (speed 3) (safety 0) (debug 0)), provided >>> that I explicitly generate the foreign-arrays x, f(x) and J which are >>> then passed on to f and df. Passing these optimized functions to >>> multi-dimensional-root-solver-fdf, however, produces a crash (using >>> speed 3 safety 0 debug 0) or an error message (speed 0 safety 3 debug >>> 3) indicating that the cl-array slot of the function arguments is nil. >>> It seems to me that this is default behaviour for arrays created >>> externally by libgsl. Is this true, and if yes, is there a way to >>> circumvent this? Or am I simply misusing grid:gref? I have attached a >>> fairly minimal example minimization-issue.lisp. Compiling this using >>> sbcl 1.0.43 (on Mac OS X) yields the following output: >>> >>> "using grid:gref" >>> Evaluation took: >>> 0.003 seconds of real time >>> 0.003419 seconds of total run time (0.003216 user, 0.000203 system) >>> 100.00% CPU >>> 6,943,409 processor cycles >>> 286,496 bytes consed >>> >>> >>> #m(-0.5397677311665408d0 -1.0671897833207353d0 -1.3289868354749306d0 >>> -1.4592646132527087d0 -1.4992646132527088d0 -1.4592646132527087d0 >>> -1.328986835474931d0 -1.0671897833207358d0 -0.5397677311665405d0) >>> "using aref" >>> Evaluation took: >>> 0.000 seconds of real time >>> 0.000010 seconds of total run time (0.000009 user, 0.000001 system) >>> 100.00% CPU >>> 15,158 processor cycles >>> 0 bytes consed >>> >>> >>> #m(-0.5397677311665408d0 -1.0671897833207353d0 -1.3289868354749306d0 >>> -1.4592646132527087d0 -1.4992646132527088d0 -1.4592646132527087d0 >>> -1.328986835474931d0 -1.0671897833207358d0 -0.5397677311665405d0) >>> >>> Best regards, >>> Sebastian Sturm >>> >> >> _______________________________________________ >> Gsll-devel mailing list >> Gsll-devel(a)common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel > >

1 0

[Gsll-devel] Efficient access to externally generated double-float arrays?
by Liam Healy 11 Oct '10

11 Oct '10

Forwarded on behalf of Sebastian Sturm: Dear all, I have been trying to use gsll for some numerical minimization. Allocating the multi-dimensional-root-solver-fdf with scalarsp set to false, I understand that I should provide functions f, df and fdf which accept (pointers to) foreign-arrays (some point x in configuration space, some output vector f(x) and the corresponding Jacobian matrix J(x)) and store the corresponding results within these arrays. Being a complete Lisp newbie, I've taken a ballistic approach to optimization by liberally scattering type declarations and (the double-float )'s throughout my code. Still, if I use grid:gref to access the given arrays, my functions have to do a lot of consing and, as a result, take a long time to complete. If I replace the grid:gref calls by (aref )'s acting on lisp arrays that have previously been set to (cl-array the-foreign-array), consing is eliminated (using (speed 3) (safety 0) (debug 0)), provided that I explicitly generate the foreign-arrays x, f(x) and J which are then passed on to f and df. Passing these optimized functions to multi-dimensional-root-solver-fdf, however, produces a crash (using speed 3 safety 0 debug 0) or an error message (speed 0 safety 3 debug 3) indicating that the cl-array slot of the function arguments is nil. It seems to me that this is default behaviour for arrays created externally by libgsl. Is this true, and if yes, is there a way to circumvent this? Or am I simply misusing grid:gref? I have attached a fairly minimal example minimization-issue.lisp. Compiling this using sbcl 1.0.43 (on Mac OS X) yields the following output: "using grid:gref" Evaluation took: 0.003 seconds of real time 0.003419 seconds of total run time (0.003216 user, 0.000203 system) 100.00% CPU 6,943,409 processor cycles 286,496 bytes consed #m(-0.5397677311665408d0 -1.0671897833207353d0 -1.3289868354749306d0 -1.4592646132527087d0 -1.4992646132527088d0 -1.4592646132527087d0 -1.328986835474931d0 -1.0671897833207358d0 -0.5397677311665405d0) "using aref" Evaluation took: 0.000 seconds of real time 0.000010 seconds of total run time (0.000009 user, 0.000001 system) 100.00% CPU 15,158 processor cycles 0 bytes consed #m(-0.5397677311665408d0 -1.0671897833207353d0 -1.3289868354749306d0 -1.4592646132527087d0 -1.4992646132527088d0 -1.4592646132527087d0 -1.328986835474931d0 -1.0671897833207358d0 -0.5397677311665405d0) Best regards, Sebastian Sturm

1 1

[Gsll-devel] patch: rank-1-update problem
by James Wright 04 Oct '10

04 Oct '10

Hi everyone, I've fixed an apparent argument-ordering issue in the definition of `rank-1-update'. I also made the corresponding change to `conjugate-rank-1-update'. The patch is attached. I've tested the fix insofar as I can now call `rank-1-update' in my code without it signalling an EBADLEN error; however, I haven't been able to get the corresponding unit test to define itself, so I haven't been able to check whether one of the failing unit tests is fixed by this change. Thanks, James

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

gsll-devel October 2010