(lisp-implementation-version) --> "1.0.1"
The documentation of extensions:run-program is misleading:
:environment An alist of STRINGs (name . value) describing the new environment. The default is to copy the environment of the current process.
The alist doesn't describe the NEW environment, it is only MERGED into the current environment.
Now of course, I consider the current behavior to be a bug:
(text-stream-contents (extensions:process-output (extensions:run-program "env" '() :wait t :environment 'nil))) --> ("NNTPSERVER=news.individual.net" "ESHELL=/bin/bash" …)
I would expect:
(text-stream-contents (extensions:process-output (extensions:run-program "env" '() :wait t :environment 'nil))) --> ()
and:
(text-stream-contents (extensions:process-output (extensions:run-program "env" '() :wait t :environment '(("LC_CTYPE" . "C"))))) --> ("LC_CTYPE=C"))
Which would match what the documentation says.
Also, in a shell environment, an empty variable is not the same as an inexistant variable, so I cannot just loop over all the existing variables to set them to an empty string. I see no SYSTEM::%PROCESS-BUILDER-ENV-REM function…
On Mar 26, 2012, at 6:12 PM, Pascal J. Bourguignon wrote:
(lisp-implementation-version) --> "1.0.1"
The documentation of extensions:run-program is misleading:
:environment An alist of STRINGs (name . value) describing the new environment. The default is to copy the environment of the current process.
The alist doesn't describe the NEW environment, it is only MERGED into the current environment.
[…]
Which behavior would you prefer?
We certainly have imprecise documentation here. The merged environment behavior what is easiest with the underlying JVM API which is based on the notion of UNIX exec(). In practice, I would venture that that most people would expect the merged behavior, as it is the common pattern for such wrappers around UNIX execv() and friends which have to copy the environment anyways at an operating system level before it replaces it with the new code. (Ok, since every contemporary OS probably has "copy-on-write" semantics here, I'll stop making OS-dependent statements of doubtful utility).
I would advocate tightening the documentation. If Pascal has a further need here, I'll take a stab at a version which would somehow "wipe" the inherited environment.
Since the semantics of our EXT:RUN-PROGRAM implementation were cribbed from SBCL's under the need to implement just as much as ASDF2 required, we should analyze what SBCL does in this situation as well.
Also, in a shell environment, an empty variable is not the same as an inexistant variable, so I cannot just loop over all the existing variables to set them to an empty string. I see no SYSTEM::%PROCESS-BUILDER-ENV-REM function…
This sounds like you really want the ability to constructs a clean NEW environment. If we were able to implement that, would your need to iterate through existing variables disappear?
On 27 March 2012 15:26, Mark Evenson evenson@panix.com wrote:
Also, in a shell environment, an empty variable is not the same as an inexistant variable, so I cannot just loop over all the existing variables to set them to an empty string. I see no SYSTEM::%PROCESS-BUILDER-ENV-REM function…
This sounds like you really want the ability to constructs a clean NEW environment. If we were able to implement that, would your need to iterate through existing variables disappear?
We could just as well evaluate all the options, which are roughly
1) merged environment 2) clean environment 3) allow editing/removing variables from either
On Mar 27, 2012, at 14:26 , Mark Evenson wrote:
On Mar 26, 2012, at 6:12 PM, Pascal J. Bourguignon wrote:
(lisp-implementation-version) --> "1.0.1"
The documentation of extensions:run-program is misleading:
:environment An alist of STRINGs (name . value) describing the new environment. The default is to copy the environment of the current process.
The alist doesn't describe the NEW environment, it is only MERGED into the current environment.
[…]
Which behavior would you prefer?
We certainly have imprecise documentation here. The merged environment behavior what is easiest with the underlying JVM API which is based on the notion of UNIX exec(). In practice, I would venture that that most people would expect the merged behavior, as it is the common pattern for such wrappers around UNIX execv() and friends which have to copy the environment anyways at an operating system level before it replaces it with the new code. (Ok, since every contemporary OS probably has "copy-on-write" semantics here, I'll stop making OS-dependent statements of doubtful utility).
I would advocate tightening the documentation. If Pascal has a further need here, I'll take a stab at a version which would somehow "wipe" the inherited environment.
Since the semantics of our EXT:RUN-PROGRAM implementation were cribbed from SBCL's under the need to implement just as much as ASDF2 required, we should analyze what SBCL does in this situation as well.
I think the most basic unix syscall is execve(2), which takes an explicit, possibly empty environment. exec(3) and its siblings are implemented using execve.
SBCL takes an explicit environment parameter, using the current environment by default but never merging. This seems like a sane approach.
Rudi
Rudi Schlatte rudi@constantly.at writes:
SBCL takes an explicit environment parameter, using the current environment by default but never merging. This seems like a sane approach.
Indeed, I forgot the default case.
run-program without an environment argument would be equivalent to:
(run-program … :environment (getenv))
Mark Evenson evenson@panix.com writes:
On Mar 26, 2012, at 6:12 PM, Pascal J. Bourguignon wrote:
(lisp-implementation-version) --> "1.0.1"
The documentation of extensions:run-program is misleading:
:environment An alist of STRINGs (name . value) describing the new environment. The default is to copy the environment of the current process.
The alist doesn't describe the NEW environment, it is only MERGED into the current environment.
[…]
Which behavior would you prefer?
I agree that for usual programs, merging is useful. On the other hand, if you mess with the environment it's often for security considerations, and there it's better to start with a blank slate.
I certainly expected that :environment '() provide an empty environment, just like execle/execvpe:
char* args[]={"env",0}; char* envv[]={0}; execpe("/bin/env",args,envv);
We certainly have imprecise documentation here. The merged environment behavior what is easiest with the underlying JVM API which is based on the notion of UNIX exec(). In practice, I would venture that that most people would expect the merged behavior, as it is the common pattern for such wrappers around UNIX execv() and friends which have to copy the environment anyways at an operating system level before it replaces it with the new code. (Ok, since every contemporary OS probably has "copy-on-write" semantics here, I'll stop making OS-dependent statements of doubtful utility).
I like the following API, which is implemented in clisp.
(getenv) --> the current environment as an a-list. (("LC_CTYPE" . "C") ("TERM" . "/bin/bash") …)
(getenv "TERM") --> the string value, or NIL if the variable is not present in the environment. "/bin/bash"
(setf (getenv "LC_CTYPE") "en_US.UTF-8")
changes the environment for the current process and its future children.
(setf (getenv "LC_CTYPE") nil)
removes the environment variable if it exists in the environment of the current process and its future children.
Then merging behavior for run-program would be obtained with:
(run-program … :environment (acons "TERM" "linux" (getenv)))
Replacing behavior:
(run-program … :environment '(("TERM" . "linux") ("SHELL" . "/bin/bash")))
and an empty environment can be provided:
(run-program … :environment '())
run-program wouldn't change the environment of the current process.
(let ((env (getenv))) (run-program … :environment other-env) (set-equal env (getenv) :test 'equal)) ;; (there's no guarantee on the order of the environment).
I would advocate tightening the documentation. If Pascal has a further need here, I'll take a stab at a version which would somehow "wipe" the inherited environment.
Since the semantics of our EXT:RUN-PROGRAM implementation were cribbed from SBCL's under the need to implement just as much as ASDF2 required, we should analyze what SBCL does in this situation as well.
Also, in a shell environment, an empty variable is not the same as an inexistant variable, so I cannot just loop over all the existing variables to set them to an empty string. I see no SYSTEM::%PROCESS-BUILDER-ENV-REM function…
This sounds like you really want the ability to constructs a clean NEW environment. If we were able to implement that, would your need to iterate through existing variables disappear?
For the specific purpose of calling run-program, yes. However, I think it's useful to be able to iterate thru existing variables.
The spam filter at tiger.common-lisp.net is wonky.
http://permalink.gmane.org/gmane.lisp.armedbear.devel/2244
"Pascal J. Bourguignon" pjb@informatimago.com writes:
Mark Evenson evenson@panix.com writes:
On Mar 26, 2012, at 6:12 PM, Pascal J. Bourguignon wrote:
(lisp-implementation-version) --> "1.0.1"
The documentation of extensions:run-program is misleading:
:environment An alist of STRINGs (name . value) describing the new environment. The default is to copy the environment of the current process.
The alist doesn't describe the NEW environment, it is only MERGED into the current environment.
[…]
Which behavior would you prefer?
I agree that for usual programs, merging is useful. On the other hand, if you mess with the environment it's often for security considerations, and there it's better to start with a blank slate.
I certainly expected that :environment '() provide an empty environment, just like execle/execvpe:
char* args[]={"env",0}; char* envv[]={0}; execpe("/bin/env",args,envv);We certainly have imprecise documentation here. The merged environment behavior what is easiest with the underlying JVM API which is based on the notion of UNIX exec(). In practice, I would venture that that most people would expect the merged behavior, as it is the common pattern for such wrappers around UNIX execv() and friends which have to copy the environment anyways at an operating system level before it replaces it with the new code. (Ok, since every contemporary OS probably has "copy-on-write" semantics here, I'll stop making OS-dependent statements of doubtful utility).
I like the following API, which is implemented in clisp.
(getenv) --> the current environment as an a-list. (("LC_CTYPE" . "C") ("TERM" . "/bin/bash") …)
(getenv "TERM") --> the string value, or NIL if the variable is not present in the environment. "/bin/bash"
(setf (getenv "LC_CTYPE") "en_US.UTF-8")
changes the environment for the current process and its future children.(setf (getenv "LC_CTYPE") nil)
removes the environment variable if it exists in the environment of the current process and its future children.Then merging behavior for run-program would be obtained with:
(run-program … :environment (acons "TERM" "linux" (getenv)))
Replacing behavior:
(run-program … :environment '(("TERM" . "linux") ("SHELL" . "/bin/bash")))
and an empty environment can be provided:
(run-program … :environment '())
run-program wouldn't change the environment of the current process.
(let ((env (getenv))) (run-program … :environment other-env) (set-equal env (getenv) :test 'equal)) ;; (there's no guarantee on the order of the environment).I would advocate tightening the documentation. If Pascal has a further need here, I'll take a stab at a version which would somehow "wipe" the inherited environment.
Since the semantics of our EXT:RUN-PROGRAM implementation were cribbed from SBCL's under the need to implement just as much as ASDF2 required, we should analyze what SBCL does in this situation as well.
Also, in a shell environment, an empty variable is not the same as an inexistant variable, so I cannot just loop over all the existing variables to set them to an empty string. I see no SYSTEM::%PROCESS-BUILDER-ENV-REM function…
This sounds like you really want the ability to constructs a clean NEW environment. If we were able to implement that, would your need to iterate through existing variables disappear?
For the specific purpose of calling run-program, yes. However, I think it's useful to be able to iterate thru existing variables.
On 27 March 2012 16:42, Pascal J. Bourguignon pjb@informatimago.com wrote:
I like the following API, which is implemented in clisp. (getenv) --> the current environment as an a-list. (("LC_CTYPE" . "C") ("TERM" . "/bin/bash") …) (getenv "TERM") --> the string value, or NIL if the variable is not present in the environment. "/bin/bash" (setf (getenv "LC_CTYPE") "en_US.UTF-8") changes the environment for the current process and its future children. (setf (getenv "LC_CTYPE") nil)
I dcn't think we can get that far. Java doesn't seem to support environment modifications that way, see http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/System.html#getenv%28... and in particular "Returns an unmodifiable string map view of the current system environment." Note the "unmodifiable".
We could allow setting the environment for run-program. Here's what I'd do:
1) fix the documentation of run-program to mention the environment merging 2) add a new function that can completely replace the default environment for the program to run, use an alist or a hash. Also add a function that can return the full environment, for cases where you don't want to rebuild everything.
Comments? Opinions?
On 27 March 2012 19:41, Ville Voutilainen ville.voutilainen@gmail.com wrote:
On 27 March 2012 16:42, Pascal J. Bourguignon pjb@informatimago.com wrote:
I like the following API, which is implemented in clisp. (getenv) --> the current environment as an a-list. (("LC_CTYPE" . "C") ("TERM" . "/bin/bash") …) (getenv "TERM") --> the string value, or NIL if the variable is not present in the environment. "/bin/bash" (setf (getenv "LC_CTYPE") "en_US.UTF-8") changes the environment for the current process and its future children. (setf (getenv "LC_CTYPE") nil)
I dcn't think we can get that far. Java doesn't seem to support environment modifications that way, see http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/System.html#getenv%28... and in particular "Returns an unmodifiable string map view of the current system environment." Note the "unmodifiable".
We could allow setting the environment for run-program. Here's what I'd do:
- fix the documentation of run-program to mention the environment merging
- add a new function that can completely replace the default
environment for the program to run, use an alist or a hash. Also add a function that can return the full environment, for cases where you don't want to rebuild everything.
Comments? Opinions?
Here's a tentative patch. I opted for a new keyword argument, since the new function approach would've led to duplication or to a macro that has the new argument anyway. Untested so far, so this is for general review. Also lacking the docstring changes.
Index: src/org/armedbear/lisp/run-program.lisp =================================================================== --- src/org/armedbear/lisp/run-program.lisp (revision 13893) +++ src/org/armedbear/lisp/run-program.lisp (working copy) @@ -40,11 +40,13 @@ ;;; This implementation uses the JVM facilities for running external ;;; processes. ;;; http://download.oracle.com/javase/6/docs/api/java/lang/ProcessBuilder.html. -(defun run-program (program args &key environment (wait t)) +(defun run-program (program args &key environment (wait t) clear-env) ;;For documentation, see below. (let ((pb (%make-process-builder program args))) (when environment (let ((env-map (%process-builder-environment pb))) + (when clear-env + (%process-builder-env-clear env-map)) (dolist (entry environment) (%process-builder-env-put env-map (princ-to-string (car entry)) @@ -131,6 +133,9 @@ (defun %process-builder-env-put (env-map key value) (java:jcall "put" env-map key value))
+(defun %process-builder-env-clear (env-map) + (java:jcall "clear" env-map)) + (defun %process-builder-start (pb) (java:jcall "start" pb))
Index: src/org/armedbear/lisp/Extensions.java =================================================================== --- src/org/armedbear/lisp/Extensions.java (revision 13893) +++ src/org/armedbear/lisp/Extensions.java (working copy) @@ -37,6 +37,7 @@
import java.io.File; import java.io.IOException; +import java.util.*;
public final class Extensions { @@ -317,4 +318,28 @@ return NIL; } } + + // ### getenv variable => string + private static final Primitive GETENV_ALL = new getenv_all(); + private static class getenv_all extends Primitive + { + getenv_all() + { + super("getenv-all", PACKAGE_EXT, true, "variable", + "Returns all environment variables as an alist containing (name . value)"); + } + @Override + public LispObject execute() + { + Cons result = new Cons(NIL); + Map<String, String> env = System.getenv(); + for (Map.Entry<String, String> entry : env.entrySet()) { + Cons entryPair = new Cons(new SimpleString(entry.getKey()), + new SimpleString(entry.getValue())); + result = new Cons(entryPair, result); + } + return result; + } + } + }
armedbear-devel@common-lisp.net
-
Mark Evenson -
Pascal J. Bourguignon -
Rudi Schlatte -
Ville Voutilainen