Hi Liam,

...

I have encountered the following problem a couple of times before, although not as reproducibly as the memory corruption issue from my previous message:  If I use the #m reader macro in a function somewhere, dump an SBCL core image, load the image, and call the function, I get memory corruption.  I have mostly dealt with it up until now by not using the #m() notation in anything that I plan to dump to a binary.

This is odd.  There is a make-load-form for foreign-arrays, so I assumed this would work.  Example?

I'm working on trying to reproduce this, but I haven't had any luck so far. (This is why memory issues are such a joy). However, I don't think that SBCL uses make-load-form when dumping an image; my understanding is that a dumped image is really just a copy of the raw memory bytes. I've attached a script with output that demonstrates that SBCL seems to want a make-load-form method for writing a FASL but not for writing a core image.

My intent is that it be analogous to #, e.g. #(1.0d0 2.0d0) expands to a (CL) array.  I used to have it the way you want but I changed it as of 3e610bd4d5261 to be analogous to the regular array macro

Fair enough. My version does not evaluate its arguments, so in that sense it is the same as #( or #2a(, but on the other hand it does return a form instead of a literal value, so in that sense it is less like #(.

(in fact, #m should make a CL array if grid:*default-grid-type* is set accordingly; I'm still not sure about the wisdom of doing that).

It is nice to be able to switch seamlessly between CL and foreign arrays. One funny situation the current version leads to is that evaluating the string printed by the REPL for a foreign array can give you a CL array back:

ANTIK-USER> (setf *default-grid-type* 'array) ARRAY ANTIK-USER> (make-foreign-array 'double-float :initial-contents '(1d0 2d0)) #m(1.000000000000000d0 2.000000000000000d0) ANTIK-USER> #m(1.000000000000000d0 2.000000000000000d0) #(1d0 2d0)

By the way, do you know about grid:grid?  Maybe that's what you want.  It's a function, so it evaluates its arguments (grid:grid pi pi pi) #m(3.141592653589793d0 3.141592653589793d0 3.141592653589793d0)

I didn't know about that; it looks like it will simplify my code. Thanks!

I think I'd like to figure out why #m can't be saved before I consider changing this back.

I think it's analogous to the problem with foreign pointers in global vars: if you have an actual literal foreign array in your code, then that is what the image will contain when you dump core, so it'll come back as an uninitialized foreign pointer; but if you have a form for creating a foreign array in your code, then the creation form (or rather the code that it compiles down to, of course) is what will get dumped. I'll spend a bit more time next week and see if I can cough up a failing example.

At any rate, it's an easy problem for end users to avoid, so I don't want to come across as badgering. But I do think it's a bit of a stability landmine to be aware of for people that need to dump binaries instead of being able to run from fasls; needing a binary is my typical gsll use case, since I use gsll on a torque cluster.

Thanks! James

On Sun, Dec 18, 2011 at 1:13 AM, James Wright james@chumsley.org wrote:

Hi Liam,

...

...

If I use the #m reader macro in a function somewhere, dump an SBCL core image, load the image, and call the function, I get memory corruption. I have mostly dealt with it up until now by not using the #m() notation in anything that I plan to dump to a binary.

This is odd. There is a make-load-form for foreign-arrays, so I assumed this would work. Example?

I've managed to come up with a script that I believe demonstrates the problem with #m returning a literal foreign-array. I've attached the script to this message, as well as a text file containing the output on my machine.

The script loads a program containing a #m-defined foreign array in a let form in the `main' function, runs the function (which just prints the array), and dumps core. It then loads the core and runs the function again. In the second run, the the array contents are garbage, because the contents of the foreign memory were not dumped or restored, and the foreign pointer of the array, although it has the same numeric value as before, no longer points to memory that it owns.

The script then loads the core again and runs a different function, `main2'. That function also contains a #m array in a let form; it freshly allocates a bunch of foreign memory until it gets back the same pointer that the #m array thinks it owns, and demonstrates that the two values use the same memory.

I haven't managed to force a crash, but it seems clear that creating wild pointers opens up the possibility.

Thanks, James

James,

After researching your previous issue, I think I have some insight on this one too. Unfortunately, contrary to my previous presumption, the save-lisp-and-die function does not in any way use make-load-form. So it's going to save any actual image with a foreign pointer that points to random nonsense. So, what to do about it. Your suggestion to expand to a form instead of the array itself should work. However, I would like to keep it as is; I changed it from form to object because I was having some kind of trouble using it analogously to # and #2A (unfortunately, I can't remember what that was). May I suggest that you use #'grid:grid instead of #m in these contexts so that you don't have problems. It is intended to be analogous to #'list and #'vector, i.e., it is code to build the object rather than the object itself. I wrote this function to be the evaluating equivalent of #m. That should work just fine in your usage.

What do you think?

Liam