There are many ways to get errno.
With the very cool cffi-wrappers you can use
(include "errno.h")
(defwrapper* "get_errno" :int () "return errno;")
which finesses the point about whether or not errno is a real variable or a thread-local variable or something else.
However, there is no guarantee that between calling your function and getting the errno, the Lisp environment will not call a C function that resets errno.
(progn (open ...) ;; something can happen here (get-errno))
In practice, we have observed this problem.
Each Lisp might have a different way of handing it.
In Allegro CL it is possible to build the wrapper for the foreign function so it collects the errno in a safe way, using def-foreign-call with :ERROR-VALUE :ERRNO.
In SBCL there is a native get-errno function to get the errno.
Is there any plan to add a semi-portable wrapper to this functionality for CFFI?
If not, would you accept a patch for it? I guess the obvious way is to modify (cffi:defcfun ...) to take a :after-collect-value argument, so that it could work not only for errno
On most Lisps I guess it might translate to something like this
(defmacro defcfun-with-collect-value ((name &key after-collect-value) ...) `(without-interrupts (values (call-foreign-function ,name ...) (funcall ,after-collect-value))))
Suggestions for better names and interfaces much appreciated.
On Fri, Jan 9, 2009 at 1:06 AM, John Fremlin jf@msi.co.jp wrote:
However, there is no guarantee that between calling your function and getting the errno, the Lisp environment will not call a C function that resets errno.
I guess this is more of a problem in Lisps with userspace threads. Are there other situations? (Signal handlers shouldn't mess with errno... Maybe GC hooks?) Any idea how Allegro implements this?
In SBCL there is a native get-errno function to get the errno.
Is there any plan to add a semi-portable wrapper to this functionality for CFFI?
Adding a CFFI-SYS:GET-ERRNO function sounds like a good idea. I don't understand the other bit well enough yet to have an opinion.
If not, would you accept a patch for it? I guess the obvious way is to modify (cffi:defcfun ...) to take a :after-collect-value argument, so that it could work not only for errno
Indeed, it'd be useful for GetLastError() as well, etc.
"Luís Oliveira" luismbo@gmail.com writes:
On Fri, Jan 9, 2009 at 1:06 AM, John Fremlin jf@msi.co.jp wrote:
However, there is no guarantee that between calling your function and getting the errno, the Lisp environment will not call a C function that resets errno.
I guess this is more of a problem in Lisps with userspace threads. Are there other situations? (Signal handlers shouldn't mess with errno... Maybe GC hooks?) Any idea how Allegro implements this?
I'm not sure. I asked them and Duane said that wrapping with without-interrupts might be okay
(defmacro with-errno (&body body) `( ,(progn 'progn #+allegro 'excl:without-interrupts) (locally ,@body) (get-errno)))
I asked about whether this might not work in the event of GC and got no answer.
In SBCL there is a native get-errno function to get the errno.
Is there any plan to add a semi-portable wrapper to this functionality for CFFI?
Adding a CFFI-SYS:GET-ERRNO function sounds like a good idea. I don't understand the other bit well enough yet to have an opinion.
It'd be great to be able to use the def-foreign-call Allegro functionality.
ClozureCL has something slightly similar called int-errno-call
(defmacro int-errno-call (form) (let* ((value (gensym))) `(let* ((,value ,form)) (if (< ,value 0) (%get-errno) ,value))))
If not, would you accept a patch for it? I guess the obvious way is to modify (cffi:defcfun ...) to take a :after-collect-value argument, so that it could work not only for errno
Indeed, it'd be useful for GetLastError() as well, etc.
On Fri, Jan 9, 2009 at 5:17 AM, John Fremlin jf@msi.co.jp wrote:
ClozureCL has something slightly similar called int-errno-call
(defmacro int-errno-call (form) (let* ((value (gensym))) `(let* ((,value ,form)) (if (< ,value 0) (%get-errno) ,value))))
This doesn't address your concern that something might happen between FORM and %GET-ERRNO does it? But, again, it seems like the only problem here is tied to user-space threads. You might want to use WITHOUT-SCHEDULING instead, not sure.
BTW, have a look at Osicat: http://common-lisp.net/project/osicat/git/osicat.git/. I believe it has the functionality you're looking for in the ERRNO-WRAPPER type and the DEFSYSCALL macro (both in osicat/posix/early.lisp). It's just missing a WITHOUT-SCHEDULING (or WITHOUT-INTERRUPTS?) for Allegro with user-space threads, IIUC.
"Luís Oliveira" luismbo@gmail.com writes:
On Fri, Jan 9, 2009 at 5:17 AM, John Fremlin jf@msi.co.jp wrote:
ClozureCL has something slightly similar called int-errno-call
(defmacro int-errno-call (form) (let* ((value (gensym))) `(let* ((,value ,form)) (if (< ,value 0) (%get-errno) ,value))))
This doesn't address your concern that something might happen between FORM and %GET-ERRNO does it? But, again, it seems like the only problem here is tied to user-space threads. You might want to use WITHOUT-SCHEDULING instead, not sure.
Yes. On ClozureCL it is quite easy to get errno safely.
My main point is that as errno is not mentioned in the CL standard, the implementation may do anything it wishes with it, and it would be nice if there was a library with a helpful way that allowed one to get errno safely on supported Lisps.
If one were actually using some kinds of rubbishy threading the last thing one wants is to put without-scheduling or without-interrupts around foreign function calls, as I understand it.
BTW, have a look at Osicat: http://common-lisp.net/project/osicat/git/osicat.git/. I believe it has the functionality you're looking for in the ERRNO-WRAPPER type and the DEFSYSCALL macro (both in osicat/posix/early.lisp). It's just missing a WITHOUT-SCHEDULING (or WITHOUT-INTERRUPTS?) for Allegro with user-space threads, IIUC.
I have my own defsyscall I am very happy with, and am very happy with getting the value of errno (and printing out an error message with the symbolic name of the error, e.g. EPERM: Permission denied, if anybody wants this code I may be allowed to share it).
It is just the without-scheduling bit that I need . . .
2009/1/13 John Fremlin jf@msi.co.jp:
It is just the without-scheduling bit that I need . . .
So, perhaps we could start by adding a new CFFI-SYS:WITHOUT-USERSPACE-SCHEDULING macro. (And perhaps CFFI-SYS:ERRNO and (SETF CFFI-SYS:ERRNO) as well.) Would that be a step in the right direction? Not sure if this is the right thing to do.
"Luís Oliveira" luismbo@gmail.com writes:
2009/1/13 John Fremlin jf@msi.co.jp:
It is just the without-scheduling bit that I need . . .
So, perhaps we could start by adding a new CFFI-SYS:WITHOUT-USERSPACE-SCHEDULING macro. (And perhaps CFFI-SYS:ERRNO and (SETF CFFI-SYS:ERRNO) as well.) Would that be a step in the right direction? Not sure if this is the right thing to do.
Ideally there would be a way to use the def-foreign-call errno collection in Allegro CL via CFFI.
For example, if the foreign-function is trace'd then the without-interrupts mechanism (sporadically) fails.
Although it is simple to make a cffi-sys:errno symbol-macro, I think that logically the errno is only valid immediately after a foreign-function call.
The idea that GC must not modify errno must be false on at least a few implementations ;-)
We should not pretend that it is possible to get errno at an arbitrary point after a foreign function call.
On Mon, Jan 19, 2009 at 2:31 AM, John Fremlin jf@msi.co.jp wrote:
Ideally there would be a way to use the def-foreign-call errno collection in Allegro CL via CFFI.
For example, if the foreign-function is trace'd then the without-interrupts mechanism (sporadically) fails.
Although it is simple to make a cffi-sys:errno symbol-macro, I think that logically the errno is only valid immediately after a foreign-function call.
I'm just trying to follow the usual CFFI philosophy: figure out the minimal CFFI-SYS operations required and implement the rest in the CFFI package. So far that'd be something like: GET-ERRNO, SET-ERRNO and WITHOUT-WHATEVER in CFFI-SYS, and an ERRNO symbol macro in CFFI (optional) and something along the lines of Osicat's ERRNO-WRAPPER.
I wonder if you can find out what hoops Allegro has to jump through to ensure errno is right?
The idea that GC must not modify errno must be false on at least a few implementations ;-)
Sounds like we could test that in the test suite.
We should not pretend that it is possible to get errno at an arbitrary point after a foreign function call.
Yeah, but I still haven't fully understood the exact failure cases. Also, it sounds like this sort of thing would be problematic for all sorts of stateful C APIs.
"Luís Oliveira" luismbo@gmail.com writes: [...]
I'm just trying to follow the usual CFFI philosophy: figure out the minimal CFFI-SYS operations required and implement the rest in the CFFI package. So far that'd be something like: GET-ERRNO, SET-ERRNO and WITHOUT-WHATEVER in CFFI-SYS, and an ERRNO symbol macro in CFFI (optional) and something along the lines of Osicat's ERRNO-WRAPPER.
I don't know that there is any need to do set-errno?
In terms of the philosophy, the Linux kernel syscalls actually do not know about errno. If they return a number between -1 and -4096 or so then it is treated as an errno and libc will set the special errno variable. So if we could call the syscalls more directly we would avoid this problem on Linux. (I think on FreeBSD it might be different.)
I wonder if you can find out what hoops Allegro has to jump through to ensure errno is right?
I can pester Duane about it again and see if we get a definite answer about the GC stuff.
The idea that GC must not modify errno must be false on at least a few implementations ;-)
Sounds like we could test that in the test suite.
It is hard to get a definitive answer but a good approximation can be tried for.
We should not pretend that it is possible to get errno at an arbitrary point after a foreign function call.
Yeah, but I still haven't fully understood the exact failure cases. Also, it sounds like this sort of thing would be problematic for all sorts of stateful C APIs.
The problem with errno is that it is modified by many function calls that are necessarily used by the Lisp environment (e.g. IO, memory maps, etc.).
Most C APIs will not have this problem because the Lisp environment does not use them.
I think in summary, most Lisps are quite unsafe with regard to errno.
Suppose we had a (defmacro with-returning-errno (&body body) ...)
What can be permitted inside BODY? Certainly not anything that does IO, because that will almost certainly change errno. That means that no trace'd function can be called, for example. GC is probably not safe.
It is difficult to think of anything useful that can be safely allowed in with-returning-errno. The BODY can only be the call to the defcfun'd function ;-)
In the case of Allegro, Duane points out that when a Lisp thread calls a foreign function, the wrapper may release the heap. If it releases the heap then it has to reacquire it before returning to normal Lisp code. And reacquiring the heap may actually require calling a C library function that resets errno.
This is platform specific, and as I understand does not occur generally on x86/AMD64 Linux.
Duane is having a look to see if there is a way to implement the with-returning-errno after all, but it is unlikely.
-
John Fremlin -
Luís Oliveira