January 2010 - gsll-devel - mailman3.common-lisp.net

Re: [Gsll-devel] Introducing "Grid Structured Data"
by Liam Healy 21 Feb '10

21 Feb '10

The timing is unfortunate; I started on GSD a while ago, once I re-discovered Tamas's AFFI, and was wrapping it up when he announced xarray. I have looked over xarray to figure out what we have in common, but not in depth. I think any help from interested/energetic/knowledgeable volunteers who could help figure out a merge strategy would be most welcome in this area. One of the things I did to integrate GSD with GSLL was to pull out the foreign-array access as the system "c-array", with type conversion between languages. This was motivated by the work of a colleague, who essentially tore off pieces of GSLL in order to make a port to a robotics library in C++, and who had requested certain array construction features in GSLL. Now it should be easier to built an interface to a new library, and I hope eventually to smoothly change array objects to make them compatible with different libraries. Unfortunately I haven't had the time I thought I would in January to pursue this, but there is certainly much more work to be done in this area. I pushed to the GSLL master branch because that all worked as before, and GSD are usable. I think all of our goals are basically the same. Per Tamas's points: 1 - Exactly the same, present in GSD "grid" now 2 - Not yet in GSD, but that is the goal as I say 3 - Not there; would be nice to integrate with GSL's views as we have discussed 4 - Present in GSD; see "compose.lisp". Perhaps not all everyone is looking for, but with some thought and knowledge of AFFI, they're not too hard to write. The general function to make most of the array composition functions possible is map-grid and map-n-grids. One thing I would like to have but isn't there is reduction over an axis, as for instance with dot product and matrix multiplication. Also, I would like to be able to treat a scalar as a grid but that isn't possible yet. Liam On Sun, Jan 10, 2010 at 2:39 AM, Tamas Papp <tkpapp(a)gmail.com> wrote: > Hi Mirko, > > I only had a cursory look at Liam's library. Slices seem to be based > on affine indexing, which in theory is faster than xarray's views, but > it is less general. > > At the moment, xarray does four things: > > 1. rank/dimension query and element access. If it is an array-like > object, you can just write methods and access it via a general > interface. Essentially redoing CL's array interface, but with generic > functions. > > 2. object conversion framework. You can use (take 'array object) to > convert your object to a Lisp array. TAKE is a generic function, you > just specialize if you introduce new classes. It takes options (like > element-type for arrays, etc). Think of R's as.* functions. > > 3. views. This is the only part I blogged about, but it may be the > least important. It is just that the rest is still under development. > > 4. convenience functions for generating, mapping, manipulating > array-like objects. If your objects have a basic xarray interface > (basically xdims and xref), xarray can take outer products, map into > other objects, etc. You can specialize methods for speed, too. > > There are a few details I am still ironing out, but basically > everything is functional now. I consider 1. and 2. above the most > important features of xarray. > > My goal is to arrive at a state like R, where I don't have to think > about whether my matrix is R's standard matrix, or implemented by the > Matrix library, if I don't want to. Things should "just work", and I > should only have to think about details when I optimize my code. > > Caveat: xarray is still experimental. 1. above should be stable, > 2. more or less, 3. might change, and 4. too. I want to get it right. > Discussion about people's needs/preferences would be welcome. > > Best, > > Tamas > > PS.: I am on the GSLL list, so no need to cc me. My Princeton e-mail > is deprecated now. > > > On Sat, 09 Jan 2010, Mirko Vukovic wrote: > >> Any comments on how that compares to Tamas Papp's xarray? >> >> I am very excited, happy and grateful with the availability of numerical >> libraries in CL. But I am concerned that we may end up with several >> non-conformant libraries for array accessing. >> >> In an ideal world, we would have a common library for array manipulation, >> and also a common architecture for accessing libraries such as gsl (C), >> netlib (fortran), and others (sundials for example). >> >> Respectfully, >> >> Mirko >> >> On Sun, Dec 27, 2009 at 10:31 PM, Liam Healy <lhealy(a)common-lisp.net> wrote: >> >> > Based on several expressed wishes for the ability to create, compose >> > and extract array-like objects and pieces of such objects, I have >> > introduced the "Grid Structured Data" collection at >> > http://repo.or.cz/w/gsd.git, and rewritten GSLL to be built on top of >> > that. The GSLL interface is the same as before, but now it is >> > possible to do subarrays, concatenation, slices, transpose, etc. on >> > arrays (both CL arrays and GSLL marrays). There is some documentation >> > for gsd in gsd/documentation/grid/index.html which describes how it >> > works. There is more work to be done, but as it is now, it provides >> > functions that people have asked for to create and manipulate marrays. >> > If you don't need that, you can go on as before and everything is the >> > same. >> > >> > Liam >> > >> > _______________________________________________ >> > Gsll-devel mailing list >> > Gsll-devel(a)common-lisp.net >> > http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel >> > >

4 7

[Gsll-devel] Array indexing syntactic sugar (was: Introducing "Grid Structured Data")
by Mirko Vukovic 21 Feb '10

21 Feb '10

This is a follow up on the syntactic sugar for array referencing that I referred to in my wish list. Here is what I had in mind. A few weeks ago, I needed the inverse of a tridiagonal matrix. I found an algorithm on the web. See the first page of the following pdf document<http://www.mat.uc.pt/preprints/ps/p0516.pdf> . I am looking for a cleaner and simpler way to transcribe equations (1.1) and the two below in lisp. Note a few things. In the equation for `theta', the index of `theta' varies from 0...n, while all the other ones are from 1...n. Another problem of interest is the Yee algorithm for the numerical solution of time-dependent Maxwell equations. Take a look at the equations following (4.12) on the following page<http://www.nmr.mgh.harvard.edu/%7Eadunn/papers/dissertation/node32.html>. The indices are, among other things, i+1/2. The notation stems from the fact that the algorithm uses two staggered grids, one for the electric, and the other for the magnetic fields. The 1/2 reflects the grid offset. Of course, in actual implementation, the fields are stored in matrics, and the indices are integers. For both of these cases, one has to do some kind of index transformation, so that E_{i+1/2} becoms (aref E (some-function-of i)). With many equations, for scatterbrains like me, that is a recipe for a missed or miscalculated offset. I came up with a macros that will transform code that contains symbols like E_i+1/2 into an appropriately referenced `aref' or `maref' by using symbol-macrolet. For the theta equations, see the following code snippet<http://paste.lisp.org/display/93525>on paste.lisp.org, where the with-_-indexing1 macro does the transforming. One thing to be aware of is that the macros expand into code of `finite' length. So, inserting this macro as follows (if (...) (with-_-indexing () (if clause) (else clause)) will be inaccurate. One has to move the `with-_-indexing' outside of the `if'. When I clean up my routines, I will post them (somewhere, not yet sure where). And if anyone has a better idea how to name the macros, please let me know. Mirko On Tue, Jan 12, 2010 at 5:20 PM, Mirko Vukovic <mirko.vukovic(a)gmail.com>wrote: > Features that I would like: > > 1. Compatibility with GSLL and LAPACK (and NETLIB for that matter). > When doing numerics, I would use grid (or xarray) and forget about > cl-arrays. > 2. More forgiving interface in array creation: coerce supplied values > to declared type > 3. syntactic sugar: refer to array subscripts using underscores: a_i_j > or a_2:5_* > > On that last point, I have a small utility that does the first example. I > am not sure where to post it for your review. I think github is overkill > to post three files (asd, package, and lisp). > > I am intrigued by xarrays' generic interface, so that xarrays can interface > with `any type of object'. I fail to see its use now, but that is just my > lack of imagination. > > On a `lack of imagination' topic, can someone give me an example of > indexing that xarray has, and that the affine indexing cannot accomplish? > > Thanks, > > Mirko > > > On Sun, Jan 10, 2010 at 3:20 PM, Liam Healy <lhealy(a)common-lisp.net>wrote: > >> I don't think there's any doubt we all want one all-singing >> all-dancing interface that provides array utility for everything in CL >> that needs it. The reason there are two starts is because we both >> started in parallel without the being aware of the other's work. The >> reason that GSD only got as far as it did was because I had addressed >> the complaints on this mailing list and elsewhere (including me) and I >> had no more time. I do not think of it as complete. >> >> So the real work here is going through both sets of code and figuring >> out how to unify them. I think all the help we can get would be >> welcome; I'm certainly willing to work toward the goal. >> >> To get the ball rolling, start with a feature at a user's level that >> you need; I mean by this some very specific thing that you've coded in >> CL already and found to be clumsy. Then see how each package >> implements it or would implement it. Then post to the list your >> findings, together with your example if possible, to start a >> discussion. Anyone can do this; I don't think there's a need to >> restrict to a "third party"; we lack a first party at the moment. By >> picking single feature(s) and working from there, we will >> incrementally get to the goal we all seek. If we try to do everything >> at once or at the most abstract level, we're likely not going to get >> there as quickly. >> >> Liam >> >> On Sun, Jan 10, 2010 at 11:50 AM, A.J. Rossini <blindglobe(a)gmail.com> >> wrote: >> > My cursory look suggests a fair amount of similarity as well, and I >> > find attractive features in both -- but want just one API! >> > >> > A reasonable proposal, from my highly biased perspective, would be a >> > 3rd party merge of the better components of each. >> > >> > (I've spent time with the xarray interface, which is the source of my >> > biases -- it mostly works for what I want to do, despite being a >> > simplification of the lisp-matrix access approach -- which isn't bad, >> > there are some lisp-matrix functionality which is strictly edge-case >> > relevant... >> > >> >> _______________________________________________ >> Gsll-devel mailing list >> Gsll-devel(a)common-lisp.net >> http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel >> > >

2 1

[Gsll-devel] bug in grid/iterate?
by Mirko Vukovic 08 Feb '10

08 Feb '10

Apologies for cross posting, but I am not sure where this problem belongs (iterage or gsll). I use slime (updated a few days ago) on sbcl 1.031 on linux. I have problems compiling the following file: >>> (in-package :grid) (defun foo () (iter:iter (iter:for e :Matrix-row grid::*array-3-4-double-float*) (princ e) (princ " "))) <<< (the code is from the iterate.lisp file of the distribution) Now in slime, I have problems. Evaluation works: I can do C-x C-e to compile `foo' I can also do C-M-x But compilation does not. Both C-c C-c and C-c C-k will flag a compiler error: note: The first argument never returns a value. --> LET* BLOCK TAGBODY PROGN SETQ THE FUNCALL SB-C::%FUNCALL THE ==> (SB-KERNEL:%COERCE-CALLABLE-TO-FUN #<FUNCTION # {10030285A9}>) Same story in raw sbcl. I looked at the iterate.lisp source code and see nothing obvious. But, I am not sure what to make of the following: If I macro-expand the above code (iter:iter ... (iter:for ... :matrix-row ...(...))) I get the following expansion. The puzzling thing is thta the first argument to `funcall' is a single hash character `#': (let* ((#:sequence208 nil) (#:limit209 nil) (e nil) (#:index207 nil)) (block nil (tagbody (progn (setq #:sequence208 *array-3-4-double-float*) (setq #:limit209 (funcall # #:sequence208)) ;; <----- (setq #:index207 -1)) loop-top-nil (progn (setq #:index207 (+ #:index207 1)) (if (>= #:index207 #:limit209) (go loop-end-nil)) (setq e (funcall # #:sequence208 #:index207)) ;; <----- (princ e) (princ " ")) (progn) (go loop-top-nil) loop-end-nil (progn)) nil)) Instead of the single hash mark, I would have expected the :access-fn and :size-fn `lambda's from the (defclause-sequence matrix-row matrix-row-index ...) in iterate.lisp. So, lots of puzzles. Not sure where this bug belongs: iterate or grid? Thanks, Mirko

2 2

[Gsll-devel] GSLL on CCL
by Leo 07 Feb '10

07 Feb '10

Hello, When I run (make-multi-dimensional-root-solver-f +powells-hybrid+ 3 #m(0 0 0)), CCL enters debug mode with the following msg on the terminal it runs in. ? Unhandled exception 10 at 0x0, context->regs at #xb0e30b1c Exception occurred while executing foreign code ? for help [57048] Clozure CL kernel debugger: More debug info are attached. Any idea how to fix this? I am running Clozure Common Lisp Version 1.4-r13357M (DarwinX8632). Thanks. Leo

2 2

[Gsll-devel] Bug? Conflict between grid & gsll: iterate.lisp present in both systems
by Mirko Vukovic 07 Feb '10

07 Feb '10

The latest versions of gsd and gsll both contain a file iterate.lisp gsll's iterate.lisp will cause grid's iterate macros to fail on cl-arrays as it uses c-array:dim0 and c-array:dim1 The iteration on cl-arrays problem is resolved if on recompiles grid's iterate.lisp after loading gsll. Mirko

2 1

[Gsll-devel] grid tranpose & diagonal compatibility with marray's
by Mirko Vukovic 07 Feb '10

07 Feb '10

I am converting some of my code to use grid. The following is on clisp & cygwin (my sbcl & linux is behind a firewall, so it takes a bit for me to transfer stuff to them) It seems that while `transpose' can handle both arrays and marrays, diagonal can handle only arrays. I traced the execution to `make-specification' in specification.lisp: SYSTEM::LIST-ACCESS: NIL is not a pair. GSL> #m(1 2 3 ^ 4 5 6 ^ 7 8 9) #<MATRIX-DOUBLE-FLOAT #2A((1.0d0 2.0d0 3.0d0) (4.0d0 5.0d0 6.0d0) (7.0d0 8.0d0 9.0d0))> GSL> (grid:transpose #m(1 2 3 ^ 4 5 6 ^ 7 8 9)) #<MATRIX-DOUBLE-FLOAT #2A((1.0d0 4.0d0 7.0d0) (2.0d0 5.0d0 8.0d0) (3.0d0 6.0d0 9.0d0))> GSL> (grid:diagonal #m(1 2 3 ^ 4 5 6 ^ 7 8 9)) ; Evaluation aborted. GSL> (grid:transpose (cl-array #m(1 2 3 ^ 4 5 6 ^ 7 8 9))) #2A((1.0d0 4.0d0 7.0d0) (2.0d0 5.0d0 8.0d0) (3.0d0 6.0d0 9.0d0)) GSL> (grid:diagonal (cl-array #m(1 2 3 ^ 4 5 6 ^ 7 8 9))) #(1.0d0 5.0d0 9.0d0) GSL> FWIW, in the test suite (tests/compose.lisp), the diagonal tests are on arrays, and not marrays. Thanks, Mirko

2 2

[Gsll-devel] How to install grid to use the iterate feature?
by Mirko Vukovic 07 Feb '10

07 Feb '10

On my slime+sbcl, I have to explicitly asdf-load `alexandria' and `iterate' before `grid', in order for the `iteration' feature to be included. Is that the intended behavior? Thanks, Mirko

2 1

[Gsll-devel] gsl_cholesky_invert
by Evgeniy Zhemchugov 21 Jan '10

21 Jan '10

The gsl_cholesky_invert function was added in gsl-1.12. The attached patch provides support for it in GSLL.

2 1

[Gsll-devel] Wrong dimension is assumed when an optional parameter is not specified in sv-solve
by Evgeniy Zhemchugov 19 Jan '10

19 Jan '10

SV-decomposition of matrix A (m x n) produces matrices U (m x n) and V (n x n) and a vector s representing the diagonal matrix S (n x n) so that A = U S V'. The solution of the equation Ax = b, x, is of dimension n that is equal to (dim0 V) rather than (dim0 U) as is written in SV-solve for its optional parameter. Hence the patch. Obviously, bug appears only for non-rectanular A, so it passed the tests.

2 1

[Gsll-devel] CCL: Slot C-POINTER is unbound in COMBINATION
by Liam Healy 17 Jan '10

17 Jan '10

On CCL, there is an error in running the combination test. I happened to notice this today, but someone posted to cll about it too: http://paste.lisp.org/display/93531 http://groups.google.com/group/comp.lang.lisp/msg/a812d158fcfcce70 This is mostly a reminder to myself, but if anyone happens to be a CCL user and want to try to debug this, by all means, have at it. Liam

1 1