On Fri, Jun 20, 2008 at 9:13 AM, Luís Oliveira luismbo@gmail.com wrote:

On Fri, Jun 20, 2008 at 5:21 AM, Hans Hübner hans@huebner.org wrote:

...

is there a good reason why WITH-FOREIGN-STRING allocates the buffer on the heap instead of using WITH-FOREIGN-OBJECT, which will allocate on the stack on platforms that support it?

I believe we were worried about big strings and the possibility of overflowing the stack. I suppose we could add an argument to WITH-FOREIGN-STRING and :STRING to force stack allocation.

Is this really a realistic issue with modern systems? Some sampling:

#include <stdlib.h>

main(int argc, char *argv[]) { int size = 1; while (1) { char* s = alloca(size); memset(s, 0, size); printf("still alive - %d\n", size); size *= 10; } }

netzhansa 16_> uname -a FreeBSD netzhansa.com 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Fri Jan 12 10:40:27 UTC 2007 root@dessler.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 netzhansa 17_> ./alloca still alive - 1 still alive - 10 still alive - 100 still alive - 1000 still alive - 10000 still alive - 100000 still alive - 1000000 still alive - 10000000 Segmentation fault (core dumped)

deng-hhueb 772_$ uname -a Linux deng-hhueb 2.6.22.2-42.asl.2.intel.fc3 #1 SMP Thu Sep 20 14:27:32 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux deng-hhueb 773_$ ./alloca still alive - 1 still alive - 10 still alive - 100 still alive - 1000 still alive - 10000 still alive - 100000 still alive - 1000000 still alive - 10000000 still alive - 100000000 still alive - 1000000000 still alive - 1410065408 still alive - 1215752192 zsh: 14633 segmentation fault ./alloca

So, at least ~10 MB for the 32 bit FreeBSD machine, at least ~1 GB for the amd64 Linux box.

Given this, I'd think that making stack allocation be the default and heap allocation an option would be beneficial. If you really don't like it, making the default a compile-time option would be better than mandating an allocation policy, I think.

Let me know which way you like it.

Thanks! Hans

Hi, cffi-devel

As an addition, I tested your code on LispWorks 5.1.1 (64-bit Linux Edition), seem there's no limit on stack size, though LispWorks has a default value, when it reached, I got a prompt to decide whether increase it (by 50%/300%).

Some outputs when I break the run manually (and LispWorks fine, no error):

[60114] #<Pointer to type :BYTE = #x2AAB276F38C0>: total allocated 3,077,836,800 [60115] #<Pointer to type :BYTE = #x2AAB277000D0>: total allocated 3,077,888,000 [60116] #<Pointer to type :BYTE = #x2AAB2770C8E0>: total allocated 3,077,939,200

P.S. By looking at LispWorks FLI, I cannot find any option in FLI:WITH- FOREIGN-STRING. Don't know how to control a foreign string be allocated in heap or stack...

--binghe

Many Lisps seem to operate with much smaller stacks. Here's the test I used:

(defconstant +size+ (* 50 (expt 2 10))) ; 50 KB

(defun test () (labels ((ek (n) (cffi:with-foreign-pointer (p #.+size+) (loop for i below +size+ do (setf (cffi:mem-ref p :char i) 0)) (format t "[~A] ~A: total allocated ~:D~%" n p (* +size+ n)) (ek (1+ n))))) (ek 1)))

(compile 'test)

SBCL: ~2 MB followed by "the party is over." CCL: ~2 MB followed by a segfault. CLISP: 8 MB followed by a graceful stack overflow. Allegro: ~4 MB followed by a graceful stack overflow. (used 40 KB chunks otherwise Allegro refused to do stack allocation)

This is on linux/amd64 (but I think my copy of Allegro is a 32-bit version). Hopefully my test is not (too) bogus.

...

Given this, I'd think that making stack allocation be the default and heap allocation an option would be beneficial. If you really don't like it, making the default a compile-time option would be better than mandating an allocation policy, I think.

Since the size argument has to be known at compile-time (IIRC), maybe WITH-FOREIGN-POINTER can look at that size and decide if it should fallback to heap allocation.

CCL and Allegro do this. SBCL and CLISP don't. If we did that check ourselves we'd get more consistent behaviour across the various Lisps we support and that might reduce the likelihood of overflowing the stack with big objects, particularly strings. Also, it would be nice if SBCL and CCL could handle stack overflows without crashing.

-- Luís Oliveira http://student.dei.uc.pt/~lmoliv/

---

cffi-devel mailing list cffi-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/cffi-devel