June 2009 - gsll-devel - mailman3.common-lisp.net

[Gsll-devel] Build error (fsbv)
by Jason Nielsen 04 Jul '09

04 Jul '09

Hello all, I just git pulled all the new goodness in gsll, fsbv etc. but am getting error messages building fsbv: ; /var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/init.fasl written ; compilation finished in 0:00:00.004 ; cc -m64 -fPIC -o /var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix /var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c unhandled SIMPLE-ERROR in thread #<SB-THREAD:THREAD "initial thread" RUNNING {10029DDD71}>: External process exited with code 1. Command was: "cc" "-m64" "-fPIC" "-o" "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix" "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c" Output was: "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 7: cannot find include file: <ffi.h> "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 40: undefined symbol: FFI_OK "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 45: undefined symbol: FFI_BAD_TYPEDEF "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 50: undefined symbol: FFI_BAD_ABI "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 61: undefined symbol: FFI_DEFAULT_ABI "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 66: undefined symbol: FFI_SYSV "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 71: undefined symbol: FFI_UNIX64 "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 79: undefined symbol: ffi_abi "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 126: incomplete struct/union/enum _ffi_type: sizeof() "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 131: undefined struct/union member: size "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 136: undefined struct/union member: alignment "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 141: undefined struct/union member: type "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 146: undefined struct/union member: elements "/var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c", line 397: warning: implicit function declaration: memset cc: acomp failed for /var/cache/common-lisp-controller/1000/sbcl/local/home/jdn/work/lisp/fsbv/libffi-unix.c I have cffi 0.10.5 installed and its test suite passes. I also have libffi-dev installed in the usual place so I'm a bit confused. Cheers, Jason

4 8

Re: [Gsll-devel] Build problems
by Liam Healy 25 Jun '09

25 Jun '09

On Wed, Jun 24, 2009 at 2:52 PM, Zach<elzacho(a)gmail.com> wrote: > > > On Tue, Jun 23, 2009 at 8:05 PM, Liam Healy <lhealy(a)common-lisp.net> wrote: > >> Simply put, CFFI-grovel doesn't "think to look" anywhere, it looks >> where the application author (me, in the case of GSLL) tells it. > > I think that maybe it (CFFI-grovel) should. I'll try to justify this at the > end. > >> >> I >> put the #+darwin conditional in libffi-unix.lisp based on a Mac-using >> colleague's report that he found ffi.h in ffi/ffi.h (sorry I don't >> know the full path). That clause works for him. I understand that >> people can install from Darwin ports, fink, etc., but libffi in >> particular is an Apple standard library so I used the location Apple >> seems to use. > > That is weird. Is this something new in, maybe in 10.5? I want to emphasize: I have utterly no idea. All I know is my colleague claims he didn't install libffi from anywhere, it seemed to be there in his Mac by itself, so I inferred that to mean OS X itself put it there (it is a common package used by many things like python etc., so I figured they automatically include it in OS X). The change I made allowed him to run just fine. So I literally have a sample space of 1. I'd like a Mac expert to take up the reins here so that we'll have a larger sample space. > > smithzv@belial:[0]:~/$ locate ffi/ffi.h > > This is from MacPorts (they changed there name a while back). Yup, I knew that but forgot it. This is proof of my lack of Mac expertise :-). > > smithzv@belial:[0]:~/$ ls /opt/local/include/ffi.h > /opt/local/include/ffi.h > >> >> So I'm going to rely on your Mac expertise here (because I have none) > > Neither do I really, just GNU/Linux experience that I am trying to use on OS > X. Then I think until a Mac expert steps up and enlightens us with the Mac Way to solve this, I think I'll leave my code as it is. > >> >> but can you answer the questions I asked of Jason? (modified suitably >> for OS X). What was the path you gave it? Why isn't ffi.h in the >> Apple-defined place? etc. You say in your other message "...I have >> for a long time had to do this for any system that uses cffi-grovel, >> since I often have libraries in only somewhat standard places under OS >> X." > > >> >> If they are in different places because of the semi-standard >> install (e.g. Darwin ports), arguably we should have that in GSLL. > > I am leaving all of my package managed files in the default directories that > the package managers use. For MacPorts that is /opt/local, and for Fink > that is /sw. I would say that these are absolutely standard places for > people who use MacPorts and Fink. I will say that if you want to build > software outside of these package managers, basically no one looks there by > default. If you want to support builds for people using MacPorts or Fink, > you could have the user push an appropriate symbol onto *FEATURES* and use > something like... > > (cffi-grovel:grovel-file .... #+(or gsll:macports gsll:darwinports > gsll:fink) :cc-flags #+(or macports darwinports)"-I/opt/local/include" > #+fink"-I/sw/include") I'm unclear on who/what put gsll:macports onto *features*. That's not anything I do, nor could I. Therefore I couldn't put it into my source code. > > For FSBV, unfortunately, it seems that sometimes ffi is in a path like > ffi/ffi.h, which means that a simple include flag will not work, instead you > may have to do something like > > #+(and gsll:darwin (not (or gsll:macports gsll:darwinports))) (include > "ffi/ffi.h") > #+(or (not darwin) (and gsll:darwin (or gsll:macports gsll:darwinports))) > (include "ffi.h") > > Not sure about fink here, since I use Macports for ffi. Sorry for typos, I > am typing this straight into the email. Macports and fink typically use > Linux like locations (i.e. they are not going to put a header in an extra > subdirectory since it would break many packages). > >> If on the other hand you have a unique organization, you're going to have >> to make your own modifications to each system. I don't know of a way >> to supply a list of possibilities to search in cffi-grovel, but even >> if that's possible, I'd only make use of it for semi-standard >> locations. > > > Right. So if I am building a C library like GSL, I typically define a build > environment. This includes a location of where header files are found and > where libraries are found. These locations are not carried in the source of > the program because, quite frankly, it isn't the responsibility of the > developer to know what loopy places a user might put his/her files. But > since every build system I know of that handles C like builds (GNU > autotools, CMake, SCons, whatever that thing is you use on OS X) allows you > to specify the location of headers and libraries, I am allowed to build > packages without keeping my own set of patches to someone else's source > tree. Quite often these locations are by default read from environment > variables like CFLAGS and LDFLAGS. > > It makes sense to me that this shouldn't happen at the GSLL level. I think > I am more sure of that now. CFFI deals with libraries, like a linker, and > it offers a special variable CFFI:*FOREIGN-LIBRARY-DIRECTORIES* that acts a > lot like the environment variable LDFLAGS. Since CFFI-grovel deals with > (primarily) header files, should offer such a analog for the environment > variable CFLAGS. There is a variable CFFI-GROVEL:*CC-FLAGS* but any value > it is bound to is thrown away prior to grovelling (it also can only specify > only one token). I could set these in my user init file and be done with > it. > > Doesn't this sound like a better solution for people who install libraries > in their home space, or have a package manager that isn't integrated with > the OS like me? You move that responsibility off of your shoulders and onto > the people who should know what is happening in there system, the users. > > If this sounds good, I will bring it up with Luis, > Zach There are two separate issues here, somewhat related: 1) Can/should CFFI-grovel search a list in much the same way that libraries are searched? 2) What location(s) might the Mac install (official or semi-standard) the header files? As far as 1 goes, I think it would be a good idea and I think you should bring it up on the CFFI mailing list. As far as 2 goes, we need a Mac expert. In fact, the merger of the two issues might be not just that a list can be provided but that if running on a Mac, then *all* the standard and semi-standard locations are searched without explicit reference to the paths. And of course this would be done with all OSes. I think that this is the case with libraries; there is some statement to the effect that system libraries are searched in standard places, so no special system knowledge is required. This could be expanded to headers too. Since this must be a larger issue than FSBV/GSLL, I think it really should be brought up with the CFFI list; there is more likely to be a Mac expert who can step up and provide the search list in general. Liam

2 1

[Gsll-devel] Build problems
by Zach 24 Jun '09

24 Jun '09

I haven't been able to build a fresh clone of GSLL for a while now (maybe never?). This has already floated up on this list before, but GSLL/CFFI-GROVEL don't know where to find header files. There seems to be no way to tell them other than changing the source and passing a special :cc-flags on the groveler line. I also have to do this with FSBV. It would be nice if there was something in CFFI-GROVEL that allows you to say "check here, here, and here for my header files." I'm talking about something like CFFI:*FOREIGN-LIBRARY-DIRECTORIES*. I am bringing it up here first since I might be mistaken about the state of things. Any other people out there have headers in places CFFI-GROVEL doesn't think to look? Do you have a better solution? Zach

2 1

[Gsll-devel] Problems with SVD through gsll on OS X 10.5 with SBCL
by Nimalan Mahendran 24 Jun '09

24 Jun '09

Hello everyone, I'm trying to see whether I could use SBCL (or whatever CL implementation) as an alternative for Matlab in machine learning research. I have gsll running, but I saw that gsll's SVD, (sv-decomposition u s v) is converging to the wrong answers for the identity matrix. >> M = [1 0 0; 0 1 0; 0 0 1] but Matlab's SVD works fine. Here's what happens in gsll/SBCL: (setf M1 (MAKE-MARRAY '(UNSIGNED-BYTE 32) :INITIAL-CONTENTS '((1 0 0) (0 1 0) (0 0 1)))) (setf u (copy m1)) (setf s (make-marray 'vector-unsigned-byte-32 :dimensions 3)) (setf v (make-marray 'matrix-unsigned-byte-32 :dimensions '(3 3))) (sv-decomposition u s v) GSL> u #<MATRIX-UNSIGNED-BYTE-32 #2A((0 0 0) (2147483648 0 3220176896) (0 2147483648 0))> GSL> s #<VECTOR-UNSIGNED-BYTE-32 #(268809831 296241371 268543215)> GSL> v #<MATRIX-UNSIGNED-BYTE-32 #2A((2083159301 2991920483 0) (2147483648 0 1072693248) (0 3220176896 0))> GSL> The answers are definitely not right.. I don't think the right memory addresses are even being accessed. I noticed in the definition of SV-decomposition in svd.lisp, the :outputs parameter is (A S V), but according to the GSL documentation at http://www.gnu.org/software/gsl/manual/html_node/Singular-Value-Decompositi…, maybe it should be (A V S)? I'm not sure how these parameters to defmfun actually work or what they mean (although I am familiar with cffi..). I was getting the same numbers even when I did SVD on this matrix: >> M = [1 0 0; 0 2 0; 0 0 3] so a memory access error seems likely to me. Would someone mind helping me fix this? Thanks! -- Nimalan Mahendran

2 2

Re: [Gsll-devel] gsll:+edom+ has no value
by Liam Healy 19 Jun '09

19 Jun '09

On Tue, Jun 16, 2009 at 11:11 PM, Mirko Vukovic<mirko.vukovic(a)gmail.com> wrote: > On Mon, Jun 15, 2009 at 10:03 PM, Liam Healy<lhealy(a)common-lisp.net> wrote: >> On Mon, Jun 15, 2009 at 10:46 AM, Mirko Vukovic<mirko.vukovic(a)gmail.com> wrote: >>> Liam, >>> >>> My reading of hyperspec is that defconstant is evaluated as a >>> top-level form. Thus the need to use the `eval-when' >> >> Mirko, >> >> Actually the hyperspec is pretty ambiguous but seems to imply that >> it could be evaluated at compile time or not. >> The CLISP manual on the other hand >> http://clisp.cons.org/impnotes/defconstant.html >> is quite definite that it does not evaluate at compile time: >> The initial value is not evaluated at compile time, just like with >> DEFVAR and DEFPARAMETER. Use EVAL-WHEN if you need the value at >> compile time. >> >> I have fixed this and it now works for me in CLISP. Do a fresh pull >> in master and try it. > > You fixed it, but the compilation order needs to change: > In order to get conditions.lisp to compile & load, I first had to > compile & load libgsl-unix.lisp. > It wasn't an order problem, but a dependency declaration problem. Fixed; try again with a fresh pull. >> >> Keep in mind that many tests still fail miserably in CLISP, and in >> fact I can only get through a dozen or so before the test suite >> crashes out of CLISP. If you (or anyone) has patches or ideas about >> how to fix this, I'd be happy to hear about it. > > Well, I did not get that far ;-) Here is where things broke again: > > Illegal keyword/value pair :TYPE, DOUBLE-FLOAT in argument list. > The allowed keywords are #1=(:OPTIONAL > :DOCUMENTATION) > [Condition of type SYSTEM::SIMPLE-KEYWORD-ERROR] You need to get a fresh update to CFFI, version 0.10.5 (released on Tuesday) or later. Liam

2 1

[Gsll-devel] need fft routines
by Mirko Vukovic 18 Jun '09

18 Jun '09

I tried recently to link to gsl's gsl_fft_complex_radix2_forward, but failed miserably: (defmfun fft-c2f (x n) "gsl_fft_complex_radix2_forward" ;; for gsl doc and example see ;; http://www.gnu.org/software/gsl/manual/html_node/Radix_002d2-FFT-routines-f… ;; for stride I use the example from sorting.lisp (((mpointer x) :pointer) (1 sizet) (n :int)) :inputs (x) :definition :function :c-return :success-failure :documentation "Forward FFT for a complex double radix-2 vector") #| ;; test run (let ((arg (make-marray '(complex double-float) :dimensions 4 :initial-element #c(0d0 0d0)))) (setf (aref arg 2) #c(1d0 0d0)) (let* ((dim (length arg))) ;; (double (make-array (* 2 dim) ;; :element-type 'double-float ;; :initial-element 0d0))) ;; repackaging complex as double -- is there a built-in? ; ;; (loop ;; for re-im across arg ;; for i from 0 to (* 2 (1- dim)) by 2 ;; do (progn ;; (setf (aref double i) (realpart re-im)) ;; (setf (aref double (1+ i)) (imagpart re-im)))) (print dim) (fft-c2f arg dim))) |# I see one possible problem, that gsl_complex_packed_array datatypes. I don't see those defined in gsll. Can someone help me with linking to the fft routines? Once I can access gsl's fft's, I will verify the computation, and also generate the unit tests. Thanks, Mirko

1 1

[Gsll-devel] gsll:+edom+ has no value
by Mirko Vukovic 16 Jun '09

16 Jun '09

Hello, on loading a fresh gsll (no fasl, lib, exe) on clisp+cygwin+windows, I get: #<ADDRESS #x44C9D500>: variable GSLL::+EDOM+ has no value [Condition of type SYSTEM::SIMPLE-UNBOUND-VARIABLE] Restarts: 0: [USE-VALUE] You may input a value to be used instead of GSLL::+EDOM+. 1: [STORE-VALUE] You may input a new value for GSLL::+EDOM+. 2: [RETRY] Retry performing #<ASDF:COMPILE-OP NIL #x1018B8A1> on #<ASDF:CL-SOURCE-FILE "conditions" #x10465C61>. 3: [ACCEPT] Continue, treating #<ASDF:COMPILE-OP NIL #x1018B8A1> on #<ASDF:CL-SOURCE-FILE "conditions" #x10465C61> as having been successful. 4: [ABORT] Return to SLIME's top level. 5: [ABORT] Abort --more-- The call stack seems to originate in the `define-gsl-condition' (see attached file) Any ideas? Thanks, Mirko PS BTW, I had to teach cffi-grovel how to access gcc under cygwin. I posted a message to cffi-grovel, so hopefully you will not need to do anything on the gsll side

2 3

[Gsll-devel] cffi-grovel dependency
by Mirko Vukovic 14 Jun '09

14 Jun '09

Hello The cffi-grovel package does not seem to be listed in gsll.asd (pulled down a few minutes ago). I have to load it separately for the installation to proceed. Does anyone else see this problem? Thanks Mirko

2 1

[Gsll-devel] Correct scalarsp form for gsl:make-multi-dimensional-minimizer-fdf?
by James Wright 10 Jun '09

10 Jun '09

Hi everyone, First of all, thanks very much for GSLL. It's impressively comprehensive, and it's making my life a lot easier. I've been using `gsl:make-multi-dimensional-minimizer-f' in one of my projects for a while now, and I'm looking at switching to one of the gradient-based methods instead. However, I can't figure out how SCALARSP works with the f-and-gradient function. As a very simple example, consider the function F(x1 x2) = x1^2 + x2^3: (defun f (x1 x2) (+ (* x1 x1) (* x2 x2 x2))) For the function-and-gradient function, I need to return the value at [x1 x2], plus a vector of 2 partial derivatives. In total that's 3 items, so I had thought that the following function would work: (defun h (x1 x2) (values (+ (* x1 x1) (* x2 x2 x2)) (* 2 x1) (* 3 x2 x2))) However, when I try to use it like so: (gsl:make-multi-dimensional-minimizer-fdf gsl:+vector-bfgs2+ 2 '(f g h) #m(9.0 9.0) 0.1d0 0.1d0 t) I get a TYPE-ERROR: The value NIL is not of type DOUBLE-FLOAT. I'm fairly sure that my definition of H is the problem here, because when I trace F, G, and H, H is the only function that gets called. I've also tried having H return F(X) as the first value and the gradient vector as the second value, but no dice. (SBCL complains that a double-float vector is not a double-float, which is true. :) I'm running SBCL and the most recent version of GSLL (I pulled from the master branch about an hour ago). Could anyone give me a hint as to what I'm doing wrong here? I could always implement using non-scalarsp functions, but if that's the only fix, then it seems pointless for make-multi-dimensional-minimizer-fdf to take a scalarsp argument in the first place. Thanks very much! James

2 2