- antik-devel - mailman3.common-lisp.net

[Antik-devel] FLAME and possible next library ports using grids
by Liam Healy 08 Dec '12

08 Dec '12

On this and other mailing lists, there have been recent discussions of "numerical lisp" which in one aspect or another has been the vision of many of us, some for many years. My approach is to write libraries I think will form a piece of the overall solution, with the hope that others will join in and write compatible libraries that form another piece. So, I initially wrote GSLL and then split the grid part out into Antik (which has other features I adapted from my personal libraries). The grid functions can use enhancement, but they are usable now for developing library ports. I have thought about and had discussions with others about what would be a good candidate library for the next port. These thoughts have revolved mainly around an FFT library (like fftw) or a linear algebra library (like *lapack) because those are widely used and, since GSL already provides some of that capability, offers a good opportunity to compare features and make a unified interface. Recently, I came across the FLAME linear algebra library (http://z.cs.utexas.edu/wiki/flame.wiki/FrontPage) It is designed with an intriguing modularity concept. I think it would be a good library to port to grid/Antik. Unfortunately, I don't have the time to work on this, but if anyone (particularly someone who needs linear algebra capability like this) wants to take it up, I encourage that and will help as much as I can. A big challenge I foresee in any foreign library interface is the allocation of arrays. If it is a "closed box," it makes it hard to use e.g. static-vectors and therefore the arrays have to be copied back and forth between the foreign side and the CL side. GSL has the advantage that there are functions that allow building GSL arrays out of supplied C arrays, so regular CL arrays can be allocated with a C pointer passed to these functions to wrap them with GSL metadata. I'm not sure many other libraries are that flexible; many (e.g. C++ libraries) consider it a virtue to seal off the raw storage (C arrays) from the user, and force you to go through a foreign interface to do the allocation. Since there are almost no CLs that permit construction of CL arrays using foreign data, copying is the only option left. So, if anyone has thoughts on FLAME or any other library, I'll be happy to contribute my ideas if it will help. Liam

2 2

[Antik-devel] test, disregard: does the list daemon accept non-plain-text emails
by Mirko Vukovic 25 Nov '12

25 Nov '12

Test email in gmail's rich-format. Please disregard. I apologize for this test. I could not find information on web whether only plain-text emails were accepted or not. This email is sent from gmail using rich formatting. Mirko

1 1

[Antik-devel] nested vector structure
by Mirko Vukovic 17 Nov '12

17 Nov '12

I have created a bit of code for gsll that uses a vector of vectors. I use this for tabular data (https://github.com/mirkov/data-table) I use raw CL for that, even though the outer vector can store foreign arrays. That way, I can do interpolation and fitting. The code is still a bit raw, and the object model may need cleaning up. A nice thing is that I can query and reshuffle the table, much like the database features in chapter 27 of Seibel's Practical Common Lisp. I would like to make this more grid compatible, but I am just too tired to figure it out. So, is it possible to - create a vector of vectors in grid - can grid use aref to access individual elements, or am I stuck with a nested aref Thanks, Mirko

2 3

[Antik-devel] Running antik on clisp+cygwin
by Mirko Vukovic 04 Nov '12

04 Nov '12

I am summarizing the changes I had to make to various pieces of code to get antik to compile and run on clisp+cygwin. I had to make changes in two libraries: fsbv and antik. They are all rather minor. *FSBV* You will need to install cygwin's libffi4 package Two modifications in libffi-unix.lisp: In the library loading sections, point to cygwin's ``ffi.h'' like this. #+cygwin (include "ffi.h") We comment out a unix64 section in ``cenum abi'' definition like so: (cenum abi ((:default-abi "FFI_DEFAULT_ABI")) ((:sysv "FFI_SYSV")) #-cygwin((:unix64 "FFI_UNIX64"))) (Cygwin is not a 32-bit application) To make sure things are workinig, you will need to modify, compile and load `examples.lisp'. The library section of "examples.lisp" is modified to load the correct cygwin libraries like so: (cffi:load-foreign-library #+(and unix (not cygwin)) "libgslcblas.so" #+cygwin "cyggslcblas-0.dll") ;; note that we load cygblas explicitly. #+cygwin (cffi:load-foreign-library "/usr/lib/lapack/cygblas-0.dll") (cffi:load-foreign-library #+(and unix (not cygwin)) "libgsl.so" #+cygwin (probe-file "/usr/bin/cyggsl-0.dll")) Once you have modified fsbv like that, make sure it compiles. The go to examples.lisp, compile and load. Then execute the few examples listed at the beginning of the file *ANTIK* In init/package.lisp (defpackage antik) I added the following symbols to be shaddowed: #:second #:fifth #:tenth #:rem Now, since CL's `second' is shaddowed, I had to prepend it with `CL:' in a few places. This was necessary in three files in physical-quantities: angle-component, physical-quantities and units.lisp. Here are the instances: >grep "cl:second" *.lisp angle-component.lisp: (cl:/ (first tangent) (cl:second tangent))) angle-component.lisp: (eql (cl:second (tangent angle)) label)) angle-component.lisp: (setf (cl:second (tangent angle)) run) physical-quantities.lisp: (parse-integer (cl:second prod)) physical-quantities.lisp: (all-same (cl:second mat) :test #'equal) ; all same units physical-quantities.lisp: (cl:second mat)) units.lisp: (or (cl:second (print-name unit)) units.lisp: (pux (cl:second expr) expon) units.lisp: (pux (cl:second expr) (cl:- expon))) ; one argument, invert units.lisp: (pux (cl:second expr) (cl:* expon (third expr)))) units.lisp: (if style (cl:second (print-name unit-sexp)) units.lisp: (mkstr (if (eql (cl:second unit-sexp) 1) units.lisp: (make-pq-string (cl:second unit-sexp) style)) units.lisp: (mkstr (make-pq-string (cl:second unit-sexp) style) units.lisp: (make-pq-string (cl:second units) (nf-option style)) With this, antik should compile cleanly on clisp+cygwin. HTH, Mirko

1 0

[Antik-devel] first draft of numerical lisp
by Mirko Vukovic 28 Oct '12

28 Oct '12

I put up my first draft of numerical lisp on https://github.com/mirkov/numerical-lisp You will have to clone it and look at the documentation in the doc/developers/index.html. Comments, feedback, participation welcome. Mirko

4 4

[Antik-devel] ann: numerical-lisp on github.com
by Mirko Vukovic 22 Oct '12

22 Oct '12

Based on a recent discussion<https://groups.google.com/d/topic/lisp-stat/sYBndICy_HA/discussion>on the Common Lisp Statistics group, I decided to start a github project numerical-lisp <https://github.com/mirkov/numerical-lisp>. The purpose of the project is to serve as an ordered (as much as possible) collection of thoughts on what would make a useful and powerful system for numerical programming, analysis and visualization based on common-lisp. I suspect the discussion will be taking place on the common lisp statistics group, antik, and maybe others. I don't think that we should form yet another discussion forum. Coding would start in the next few months. The project would depend a great deal on Antik, gsll, cl-num-utils, and other numerical libraries. The goal is not to start yet another incompatible numerical library. Instead, the goal is to provide a flexible framework into which one can plug in existing libraries with relatively little modification. When it comes to coding, there will be relatively little intense numerical or graphics coding. Instead, we should use as many of already existing resources and libraries already available, and allow enough flexibility so as not to be locked to a particular library. We should also look at other systems (R, numpy, Matlab, mathematica, Macsyma), and incorporate their best features, albeit in a lispy way. The project may boil down to software engineering, and good language design (maybe we should study Fortress for hints) Right now the project is quite empty, and I will add to it over the coming weeks. Once I figure out how github operates for collaborative work, I would welcome contributions from others as well. I hope this to be a community effort. Mirko

4 6

Re: [Antik-devel] antik on clisp - some corrections
by Liam Healy 20 Oct '12

20 Oct '12

Mirko, If you know texinfo you can patch the doc source where the links are bad, otherwise I'll try to fix it. If you can create a git patch to the head of cffi-libffi I will take a look at it. I think I understand the reason for the conflict on 'second (the unit of time vs. the function that produces the second element of a sequence) but I don't know why it's not seen on other implementations. I don't understand the other conflicts off the top of my head. Liam On Sun, Oct 7, 2012 at 12:14 PM, Mirko Vukovic <mirko.vukovic(a)gmail.com>wrote: > Liam, > > I installed antik on clisp on cygwin+windows. (I also have it running on > sbcl & linux). > > To make it go I had to change a few things: > > - minor syntax tweak in the asd file (use :asdf-system-connections instead > of #:asdf-system-connections) > - shadow a few more cl symbols in package.lisp > > #+clisp #:second #+clisp #:fifth #+clisp #:tenth > #+clisp #:rem > > > Also, in a few files I had to specify `cl:second' instead of just `second' > > Now all but two tests are passing. > > Two questions/points: > - Maybe I missed something. Could that be the reason for the symbol > conflicts. > - If you want to accept these changes, what is the best way to submit them? > > Also, there are faulty links on the html documentation. Chapter five > links5 and 5.1 both lead to Section 4.5. > > Thank you for the antik. It is quite something. > > I will now proceed to adapt map-grid-utils to the version of grid that is > built into antik. > > Mirko > > >

2 1

Re: [Antik-devel] First post: common-lisp -> numerical-lisp->CL-statistics
by Liam Healy 14 Oct '12

14 Oct '12

Hi Mirko, Tamas, Tony, I am happy to read of these developments; they align very closely with what I have in mind in building grid and Antik. A little bit of background: I do numerical computation as part of my "day jobs," and have used Lisp for 25 years now, starting with the Symbolics. Over the decades, I developed for personal use only a rather (ahem) hackish set of numerical libraries and utilities. When GSL came out in the late 90s, I saw it as an opportunity to gain some numerical utilities without having to write them myself, but it took until CFFI's release in 2005 to spur me to start GSLL. I am of the mind that any correct library is potentially useful and should be available in CL; while I prefer to code in CL myself, if the library is already written, then I will try to use it. I realize others may differ. My goal in writing GSLL was to make the non-lispness of it as invisible as possible, so that it can be used with all the power of lisp and none of the hassles of C. One of the hacks I had in my old personal library was that of generic arrays. With some GSL functions needing arrays, I was forced to reinvent that. I did a lot of research of existing CL libraries, and settled on Tamas's affine library (which predates xarray, I believe) as a basis, and built on that. Eventually, this was split out from GSLL into the grid package in Antik, which also includes other numerical utilities. I envision that grids could address any kind of array-like structure (including, say SQL tables, structured files like FITS, HDF, etc.) because these are the kinds of things I use in my work. They should be as transparent as possible, so that if for example there is a function in GSL that does the same thing as another library (written in C, or CL, or whatever), I can send the exact same inputs and compare the outputs. There is a lot of work to be done to get Antik and grids to that point, and I welcome ideas, contributions, and patches. Unfortunately, my time on this has been very limited recently, but I intend to move this forward as best as I can. Mirko has sent me emails and I've been slow to respond. I think the antik-devel<http://lists.common-lisp.net/cgi-bin/mailman/listinfo/antik-devel>list is a good place to discuss any ideas, and its email archive may be useful in the future. It is my intent to be inclusive of different approaches as possible, so that even working entirely within CL using arrays and displaced arrays should be supported (currently, displaced arrays are not there). I look at languages like Fortran 90, Python, and Matlab, and I think there is opportunity to borrow idioms and features, and maybe have interfaces to some of those libraries. I don't know R but suspect it too has something to offer. As Mirko noted, I use (shadowed) CL symbols where possible. One special note about the current status: all development for the last several months or so (and there hasn't been much) is on the "cffi-libffi" branches. Over last fall and winter, working with Luís Oliveira, I have replaced FSBV with the system cffi-libffi, currently distributed on the master branch of CFFI. This is a significant improvement on FSBV because calling structures by value require nothing more than CFFI calls and having libffi (also, things like recursive structures should work). While this is in the master branch of CFFI, it is not yet in a released version of CFFI, so I have held off merging my cffi-libffi branches into master because Quicklisp only pulls from released versions of software, where such a concept is defined (it isn't for Antik and GSLL). However, I don't maintain the FSBV-based master branches any more. (I am copying antik-devel so that other Antik users will see.) Liam On Fri, Oct 12, 2012 at 7:45 AM, A.J. Rossini <blindglobe(a)gmail.com> wrote: > Hi Mirko - > > The other possibility is that depending on the complexity of Antik (sounds > like it does a good deal, whether it should all be in the same place is > another matter, which is just a question that I shouldn't ask until I read > the code later this weekend, Sorry for posing it prematureley!), we just > replace xarray with antik. Some of this might be pretty simple. > > > > On Friday, October 12, 2012 2:02:06 AM UTC+2, Mirko Vukovic wrote: >> >> This is a bit long-winded, but it may have some relevance to the >> architecture of CL-Statistics >> >> The other day, I was writing this as part of a sequence generating >> library documentation that I hope to release soon: >> >> The Intoruction to R<http://cran.r-project.org/doc/manuals/R-intro.html#Top> describes >> vectors<http://cran.r-project.org/doc/manuals/R-intro.html#Simple-manipulations-num…>. >> Of interest are also sections on factors<http://cran.r-project.org/doc/manuals/R-intro.html#Factors>, >> arrays and matrices<http://cran.r-project.org/doc/manuals/R-intro.html#Arrays-and-matrices>, >> and lists and data frames<http://cran.r-project.org/doc/manuals/R-intro.html#Lists-and-data-frames>. >> Finally, reading data<http://cran.r-project.org/doc/manuals/R-intro.html#Reading-data-from-files>is of interest. At this point, this is starting to sound like Rossini's >> project common-lisp-stat. However note that this project uses Tamas >> Papp's xarray. >> >> At that point, I went to google what Rossini was up to, and came upon >> this list. It was up for only a day or so when I found it. >> >> The reason for this post is that I am using Liam Healy's gsll and antik >> libraries. These use his grid library for representing vectors, matrices, >> etc. It is unfortunate that at the time Liam was releasing grid, Tamas >> released his xarray. >> >> Personally, I dislike fragmentation (grid vs xarray), but now nothing can >> be done about it. And some competition is good. >> >> The interesting part of Liam's antik is that antik shadows several of >> CL's symobls: many math functions, and also aref and aref*. He >> reimplements these as generic functions, allowing extension for other data >> types (such as grids). In a way, Liam is on his way to creating what I >> would call Numerical Lisp: Extension of CL for numerical computing that may >> be incompatible with CL. But the extension is a relatively thin layer on >> top of CL, leaving all of CL accessible. >> >> Futhermore, Numerical-Lisp (NL) can unify grid and xarray using the >> following three-layered structure: >> - Application layer, such as CL-Statistics >> - Numerical-Lisp layer >> - Raw libraries (grid/xarray) >> >> The numerical lisp layer would consist of Liam's redefinitions of CL >> mathematics related functions via generic functions. The raw libraries >> would hook into this via methods. In addition to CL mathematics functions, >> NL can define interface to higher mathematics functions also using generic >> functions. Then the raw libraries can plug in using packages such as GSL, >> LAPACK, etc. >> >> What needs to be done is >> - creation of the Numerical-Lisp library >> - hooks in grid and xarray >> - write the application layer to use Numerical-Lisp interface. >> >> It seems to me that much of this should be relatively straightforward. I >> would be glad to help in testing (sbcl, clisp) and documenting. >> >> Finally, what does this have to do with CL-statistics? Well, I am >> suggesting that the code be organized in a way to make it easy to >> transition from interfacing to CL & xarray into NL. >> >> Best, >> >> Mirko >> >> PS - and in the end, when all is said and done, someone writes a great >> book: Practical Numerical Lisp >> >

1 0

[Antik-devel] Sharp-m reader macro ignores *read-suppress* (+ patch)
by Martin Schmauder 28 Apr '12

28 Apr '12

Hi, I have encountered a problem with the sharp-m reader macro function. The sharp-m reader macro function (lambda in grid/functions.lisp, l. 202) uses the function cl:read-delimited-list to read the macro arguments. However, when cl:*read-suppress* is true, read-delimited-list always returns nil (See *read-suppress* in CLHS.) That value is then (wrongly) passed to grid:check-initial-contents, which (rightly) signals the error "Initial-contents specified as NIL." The following code can be used to observe the behaviour. No matter what (valid or invalid) code is inside the sharp-m parentheses, the error will be signaled. (let ((*read-suppress* t)) (with-input-from-string (stream "#m(1 2 3)") (read stream))) The solution is to call read-delimited-list and then check if we have to ignore its return value with "(unless *read-suppress* ...)". Patch attached. By the way, how I encountered this issue: When I compile a file with C-c C-k and the compilation yields warnings or notes, the Slime/SBCL combination seems to re-read the file with *read-suppress*. Any sharp-m macro that is then read triggers the error. Try with: #m(1 2 3) (defun foo (bar)) ; unused variable style-warning Comments? Martin

2 1

[Antik-devel] Sharp-m reader macro function ignores *read-suppress* (+ patch)
by Martin Schmauder 25 Apr '12

25 Apr '12

Hi, the sharp-m reader macro function (grid/functions.lisp, l. 202) ignores cl:*read-suppress*. When *read-suppress* is true, the function unconditionally signals the error "Initial-contents specified as NIL." The following code can be used to observe the behaviour. No matter what (valid or invalid) code is inside the sharp-m parentheses, the error will be signaled. (let ((*read-suppress* t)) (with-input-from-string (stream "#m(1 2 3)") (read stream))) I encountered this in Emacs: When I compile a file with C-c C-k and the compilation yields warnings or notes, the Slime/SBCL combination seems to re-read the file with *read-suppress*. Any sharp-m macro that is then read triggers the error. Try with: #m(1 2 3) (defun foo (bar)) ; provoke unused variable style-warning My solution is to put a simple "(unless *read-suppress* ...)" at the appropriate place: if *read-suppress* is true, the macro contents will still be read but then ignored. A patch is attached. The patch did not break any tests for me. Does this seem sane? Comments? Martin

1 0