[Asdf-devel] Program with metaclass on MKCL.
Hello. Being new to "serious" CL programming I am hitting a brick wall right now in one CLOS project. After some time of infructuous searching I cannot find the source of my problem, which may be of either MKCL or ASDF but I lack the knowledge to debug this. Mr. JCB, who I have seen active on this list, may (hopefully!) be able to shed some light on the issue. The bug I encounter happens when I load the project by the means of ASDF. I can not reproduce it otherwise. It is triggered by a use of a custom metaclass, and it suggests that a file is loaded more times than it should. I have narrowed down the problem to a small program, so here it is: ;;;; hello.asd (asdf:defsystem #:hello :description "Describe hello here" :author "Your Name <your.name@example.com>" :license "Specify license here" :serial t :components ((:file "package") (:file "a")) :depends-on ()) ;;;; package.lisp (defpackage #:hello (:use #:cl)) ;;;; a.lisp (in-package #:hello) (defclass A-class (standard-class) ()) (defclass A (standard-object) () (:metaclass A-class)) * Case 1: load with MKCL only: mkcl -load package.lisp -load a.lisp OK, works as expected. * Case 2: compile with MKCL only and load: mkcl -load package.lisp -compile a.lisp OK, it produces a fas (shared library). I load it from REPL. It works. * Case 3: load or build with ASDF It fails (at the definition of class A). The error message is this: -> When redefining a class, the metaclass can not change.. It acts as if a.lisp was actually loaded twice. Same error as loading a twice in a row, as by this command. mkcl -load package.lisp -load a.lisp -load a.lisp My ASDF command: (asdf:operate 'asdf:load-op :hello) Same problem with compile-op and program-op. However, load-source-op does work without problem. (I always make sure to clear cache between asdf operations) I will happily provide more specific info at request. The bug occured on MKCL 1.1.9 and 1.1.10 git with bundled ASDF.
"D. Martinez" <denis.martinez@live.com> writes:
Hello. Being new to "serious" CL programming I am hitting a brick wall right now in one CLOS project. After some time of infructuous searching I cannot find the source of my problem, which may be of either MKCL or ASDF but I lack the knowledge to debug this. Mr. JCB, who I have seen active on this list, may (hopefully!) be able to shed some light on the issue.
The bug I encounter happens when I load the project by the means of ASDF. I can not reproduce it otherwise. It is triggered by a use of a custom metaclass, and it suggests that a file is loaded more times than it should.
I have narrowed down the problem to a small program, so here it is: ;;;; hello.asd (asdf:defsystem #:hello :description "Describe hello here" :author "Your Name <your.name@example.com>" :license "Specify license here" :serial t :components ((:file "package") (:file "a")) :depends-on ())
;;;; package.lisp (defpackage #:hello (:use #:cl))
;;;; a.lisp (in-package #:hello) (defclass A-class (standard-class) ()) (defclass A (standard-object) () (:metaclass A-class))
AFAIK, you need to define a validate-superclass method when you define a new metaclass: ---- a.lisp ------------------------------------------------------------ (in-package #:hello) (defclass A-class (standard-class) ()) (defmethod validate-superclass ((class A-class) (super standard-class)) t) (defclass A (standard-object) () (:metaclass A-class)) ---- hello.asd --------------------------------------------------------- (asdf:defsystem #:hello :description "Describe hello here" :author "Your Name <your.name@example.com>" :license "Specify license here" :serial t :components ((:file "package") (:file "a")) :depends-on ("closer-mop")) ---- package.lisp ------------------------------------------------------ (defpackage #:hello (:use #:cl) (:import-from #:closer-mop #:validate-superclass)) ------------------------------------------------------------------------
I will happily provide more specific info at request. The bug occured on MKCL 1.1.9 and 1.1.10 git with bundled ASDF.
I tested it with ccl. -- __Pascal Bourguignon__ http://www.informatimago.com/ “The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.” -- Carl Bass CEO Autodesk
Thank you Pascal for pointing this out, but I don't think this relates to my problem. I have looked at MKCL source and it was evident that it does not implement superclass validation at this time. Its MOP is not supported either in Closer-MOP so I assume it is somewhat young. Quoting for clos/standard.lsp: ;; FIXME!!! Here should come the invocation of VALIDATE-SUPERCLASS! As I have played with various CLs I noted that some required VALIDATE-SUPERCLASS and some did not. I have not really checked what is the standard's stance on this. I can trace the error rising from defclass to standard.lsp in mkcl. It is the cond clause from ensure-class-using-class. I display some more information to post here. ((not (eq (class-of class) metaclass)) (format *error-output* "class: ~A~%" class) (format *error-output* "(class-of class): ~A~%" (class-of class)) (format *error-output* "metaclass: ~A~%" metaclass) (error "When redefining a class, the metaclass can not change."))) Which provides this output class: &<The A-CLASS A 140012460512096> (class-of class): &<The STANDARD-CLASS A-CLASS 140012464325568> metaclass: &<The STANDARD-CLASS A-CLASS 140012460537344> Error during command line arguments processing: When redefining a class, the metaclass can not change.. This does show that A-CLASS is intanciated twice, and it is why the EQ test fails. The mystery is why there happen two instanciations when I use ASDF. ----------------------------------------
To: asdf-devel@common-lisp.net From: pjb@informatimago.com Date: Thu, 4 Dec 2014 11:26:50 +0100 Subject: Re: [Asdf-devel] Program with metaclass on MKCL.
"D. Martinez" <denis.martinez@live.com> writes:
Hello. Being new to "serious" CL programming I am hitting a brick wall right now in one CLOS project. After some time of infructuous searching I cannot find the source of my problem, which may be of either MKCL or ASDF but I lack the knowledge to debug this. Mr. JCB, who I have seen active on this list, may (hopefully!) be able to shed some light on the issue.
The bug I encounter happens when I load the project by the means of ASDF. I can not reproduce it otherwise. It is triggered by a use of a custom metaclass, and it suggests that a file is loaded more times than it should.
I have narrowed down the problem to a small program, so here it is:
;;;; hello.asd (asdf:defsystem &:hello :description "Describe hello here" :author "Your Name <your.name@example.com>" :license "Specify license here" :serial t :components ((:file "package") (:file "a")) :depends-on ())
;;;; package.lisp (defpackage &:hello (:use &:cl))
;;;; a.lisp (in-package &:hello) (defclass A-class (standard-class) ()) (defclass A (standard-object) () (:metaclass A-class))
AFAIK, you need to define a validate-superclass method when you define a new metaclass:
---- a.lisp ------------------------------------------------------------ (in-package &:hello) (defclass A-class (standard-class) ()) (defmethod validate-superclass ((class A-class) (super standard-class)) t) (defclass A (standard-object) () (:metaclass A-class)) ---- hello.asd --------------------------------------------------------- (asdf:defsystem &:hello :description "Describe hello here" :author "Your Name <your.name@example.com>" :license "Specify license here" :serial t :components ((:file "package") (:file "a")) :depends-on ("closer-mop")) ---- package.lisp ------------------------------------------------------ (defpackage &:hello (:use &:cl) (:import-from &:closer-mop &:validate-superclass)) ------------------------------------------------------------------------
I will happily provide more specific info at request. The bug occured on MKCL 1.1.9 and 1.1.10 git with bundled ASDF.
I tested it with ccl.
-- __Pascal Bourguignon__ http://www.informatimago.com/ “The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.” -- Carl Bass CEO Autodesk
_______________________________________________ Asdf-devel mailing list Asdf-devel@common-lisp.net http://mailman.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
"D. Martinez" <denis.martinez@live.com> writes:
Thank you Pascal for pointing this out, but I don't think this relates to my problem. I have looked at MKCL source and it was evident that it does not implement superclass validation at this time. Its MOP is not supported either in Closer-MOP so I assume it is somewhat young. Quoting for clos/standard.lsp: ;; FIXME!!! Here should come the invocation of VALIDATE-SUPERCLASS! As I have played with various CLs I noted that some required VALIDATE-SUPERCLASS and some did not. I have not really checked what is the standard's stance on this.
I can trace the error rising from defclass to standard.lsp in mkcl. It is the cond clause from ensure-class-using-class. I display some more information to post here. ((not (eq (class-of class) metaclass)) (format *error-output* "class: ~A~%" class) (format *error-output* "(class-of class): ~A~%" (class-of class)) (format *error-output* "metaclass: ~A~%" metaclass) (error "When redefining a class, the metaclass can not change.")))
Which provides this output
class: &<The A-CLASS A 140012460512096> (class-of class): &<The STANDARD-CLASS A-CLASS 140012464325568> metaclass: &<The STANDARD-CLASS A-CLASS 140012460537344> Error during command line arguments processing: When redefining a class, the metaclass can not change..
This does show that A-CLASS is intanciated twice, and it is why the EQ test fails. The mystery is why there happen two instanciations when I use ASDF.
An explaination could be that MKCL uses the same environment for compilation and loading, and that ASDF does both COMPILE-FILE and LOAD (when the file is not already compiled). A compiler doesn't have to define the class completely when compiling; it only has to do the following: If a defclass form appears as a top level form, the compiler must make the class name be recognized as a valid type name in subsequent declarations (as for deftype) and be recognized as a valid class name for defmethod parameter specializers and for use as the :metaclass option of a subsequent defclass. The compiler must make the class definition available to be returned by find-class when its environment argument is a value received as the environment parameter of a macro. You could test and compare: 1.1- remove the fasl files. 1.2- from a fresh image, (asdf:oos 'asdf:load-source-op :hello) this only LOAD the source file (situation :execute) 2.1- remove the fasl files (if any) 2.2- from a fresh image, (asdf:oos 'asdf:load-op :hello) this uses COMPILE-FILE and LOAD (situations :compile-toplevel and :load-toplevel) 3.1- keep the fasl files from the previous compilation. 3.2- from a fresh image, (asdf:oos 'asdf:load-op :hello) this should only LOAD the fasl file (situation :load-toplevel) -- __Pascal Bourguignon__ http://www.informatimago.com/ “The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.” -- Carl Bass CEO Autodesk
I understand your hypothesis and I guess it makes sense. I have done the testing as you suggest. 1. load-source-op is good 2. load-op fails on the first time. But it produces fasl. 3. load-op succeeds, on the second time. In a larger program, the load-op seems to fail as many times as there are custom metaclasses so I have to repeat the operation that many times (certainly why I haven't had the idea to do that in the first place). This is hardly a satisfactory solution but it is way better than nothing. For lack of a good solution it will do for now. So, should I conclude that MKCL is at fault and redirect my bug reporting there ? ----------------------------------------
To: asdf-devel@common-lisp.net From: pjb@informatimago.com Date: Thu, 4 Dec 2014 13:45:32 +0100 Subject: Re: [Asdf-devel] Program with metaclass on MKCL.
"D. Martinez" <denis.martinez@live.com> writes:
Thank you Pascal for pointing this out, but I don't think this relates to my problem. I have looked at MKCL source and it was evident that it does not implement superclass validation at this time. Its MOP is not supported either in Closer-MOP so I assume it is somewhat young. Quoting for clos/standard.lsp: ;; FIXME!!! Here should come the invocation of VALIDATE-SUPERCLASS! As I have played with various CLs I noted that some required VALIDATE-SUPERCLASS and some did not. I have not really checked what is the standard's stance on this.
I can trace the error rising from defclass to standard.lsp in mkcl. It is the cond clause from ensure-class-using-class. I display some more information to post here. ((not (eq (class-of class) metaclass)) (format *error-output* "class: ~A~%" class) (format *error-output* "(class-of class): ~A~%" (class-of class)) (format *error-output* "metaclass: ~A~%" metaclass) (error "When redefining a class, the metaclass can not change.")))
Which provides this output
class: &<The A-CLASS A 140012460512096> (class-of class): &<The STANDARD-CLASS A-CLASS 140012464325568> metaclass: &<The STANDARD-CLASS A-CLASS 140012460537344> Error during command line arguments processing: When redefining a class, the metaclass can not change..
This does show that A-CLASS is intanciated twice, and it is why the EQ test fails. The mystery is why there happen two instanciations when I use ASDF.
An explaination could be that MKCL uses the same environment for compilation and loading, and that ASDF does both COMPILE-FILE and LOAD (when the file is not already compiled).
A compiler doesn't have to define the class completely when compiling; it only has to do the following:
If a defclass form appears as a top level form, the compiler must make the class name be recognized as a valid type name in subsequent declarations (as for deftype) and be recognized as a valid class name for defmethod parameter specializers and for use as the :metaclass option of a subsequent defclass. The compiler must make the class definition available to be returned by find-class when its environment argument is a value received as the environment parameter of a macro.
You could test and compare:
1.1- remove the fasl files. 1.2- from a fresh image, (asdf:oos 'asdf:load-source-op :hello) this only LOAD the source file (situation :execute)
2.1- remove the fasl files (if any) 2.2- from a fresh image, (asdf:oos 'asdf:load-op :hello) this uses COMPILE-FILE and LOAD (situations :compile-toplevel and :load-toplevel)
3.1- keep the fasl files from the previous compilation. 3.2- from a fresh image, (asdf:oos 'asdf:load-op :hello) this should only LOAD the fasl file (situation :load-toplevel)
-- __Pascal Bourguignon__ http://www.informatimago.com/ “The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.” -- Carl Bass CEO Autodesk
_______________________________________________ Asdf-devel mailing list Asdf-devel@common-lisp.net http://mailman.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
D. Martinez wrote:
I understand your hypothesis and I guess it makes sense.
I have done the testing as you suggest. 1. load-source-op is good 2. load-op fails on the first time. But it produces fasl. 3. load-op succeeds, on the second time. In a larger program, the load-op seems to fail as many times as there are custom metaclasses so I have to repeat the operation that many times (certainly why I haven't had the idea to do that in the first place).
This is hardly a satisfactory solution but it is way better than nothing. For lack of a good solution it will do for now. So, should I conclude that MKCL is at fault and redirect my bug reporting there ?
One thing I notice is that you are defining like this: (defclass A (standard-object) () (:metaclass A-class)) I don't believe that this is in general correct. Looking at the spec: Each superclass-name argument specifies a direct superclass of the new class. If the superclass list is empty, then the superclass defaults depending on the metaclass, with standard-object being the default for standard-class. As I read this, you should drop the specification of STANDARD-OBJECT, because the implementation is licensed to choose a more appropriate default superclass. If it does, probably you want that more appropriate superclass. I don't mean to be troublesome, but if you are new to serious CL programming maybe 1. You should try using a more mature CL implementation. 2. You should avoid the MOP. Unless you are doing something that, with all due respect, someone who's new to serious CL programming shouldn't be doing, you don't need the MOP. CLOS is already so powerful, that unless you are doing something that fundamentally changes the object system, you don't need the MOP. And see my earlier point: if you are new to serious CL programming, you shouldn't be fundamentally changing the object system. Cheers, r
----------------------------------------
To: asdf-devel@common-lisp.net From: pjb@informatimago.com Date: Thu, 4 Dec 2014 13:45:32 +0100 Subject: Re: [Asdf-devel] Program with metaclass on MKCL.
"D. Martinez" <denis.martinez@live.com> writes:
Thank you Pascal for pointing this out, but I don't think this relates to my problem. I have looked at MKCL source and it was evident that it does not implement superclass validation at this time. Its MOP is not supported either in Closer-MOP so I assume it is somewhat young. Quoting for clos/standard.lsp: ;; FIXME!!! Here should come the invocation of VALIDATE-SUPERCLASS! As I have played with various CLs I noted that some required VALIDATE-SUPERCLASS and some did not. I have not really checked what is the standard's stance on this.
I can trace the error rising from defclass to standard.lsp in mkcl. It is the cond clause from ensure-class-using-class. I display some more information to post here. ((not (eq (class-of class) metaclass)) (format *error-output* "class: ~A~%" class) (format *error-output* "(class-of class): ~A~%" (class-of class)) (format *error-output* "metaclass: ~A~%" metaclass) (error "When redefining a class, the metaclass can not change.")))
Which provides this output
class: &<The A-CLASS A 140012460512096> (class-of class): &<The STANDARD-CLASS A-CLASS 140012464325568> metaclass: &<The STANDARD-CLASS A-CLASS 140012460537344> Error during command line arguments processing: When redefining a class, the metaclass can not change..
This does show that A-CLASS is intanciated twice, and it is why the EQ test fails. The mystery is why there happen two instanciations when I use ASDF. An explaination could be that MKCL uses the same environment for compilation and loading, and that ASDF does both COMPILE-FILE and LOAD (when the file is not already compiled).
A compiler doesn't have to define the class completely when compiling; it only has to do the following:
If a defclass form appears as a top level form, the compiler must make the class name be recognized as a valid type name in subsequent declarations (as for deftype) and be recognized as a valid class name for defmethod parameter specializers and for use as the :metaclass option of a subsequent defclass. The compiler must make the class definition available to be returned by find-class when its environment argument is a value received as the environment parameter of a macro.
You could test and compare:
1.1- remove the fasl files. 1.2- from a fresh image, (asdf:oos 'asdf:load-source-op :hello) this only LOAD the source file (situation :execute)
2.1- remove the fasl files (if any) 2.2- from a fresh image, (asdf:oos 'asdf:load-op :hello) this uses COMPILE-FILE and LOAD (situations :compile-toplevel and :load-toplevel)
3.1- keep the fasl files from the previous compilation. 3.2- from a fresh image, (asdf:oos 'asdf:load-op :hello) this should only LOAD the fasl file (situation :load-toplevel)
-- __Pascal Bourguignon__ http://www.informatimago.com/ “The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.” -- Carl Bass CEO Autodesk
_______________________________________________ Asdf-devel mailing list Asdf-devel@common-lisp.net http://mailman.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
_______________________________________________ Asdf-devel mailing list Asdf-devel@common-lisp.net http://mailman.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
On Thu, Dec 4, 2014 at 5:20 AM, D. Martinez <denis.martinez@live.com> wrote:
Hello. Being new to "serious" CL programming I am hitting a brick wall right now in one CLOS project. After some time of infructuous searching I cannot find the source of my problem, which may be of either MKCL or ASDF but I lack the knowledge to debug this. Mr. JCB, who I have seen active on this list, may (hopefully!) be able to shed some light on the issue.
This is definitely a MKCL 1.1.X bug, and one of the "unexpected consequence" variety. ASDF is not concerned by this problem. It may have helped in revealing the issue but nothing more, and it played only an incidental role in the matter. This bug is fixed in MKCL 1.2.0 (currently only in my own private branch) but I do not plan to back port this fix to MKCL 1.1.X, the change-set would just be way too large. Pascal Bourguignon is right in pointing out the fact that CL:COMPILE-FILE on MKCL 1.1.X is persistently polluting the global environment of its running image with by-products of the compilation (structures, classes, symbols, etc.). This has been an hereditary defect of the KCL family from its origins (circa 1984) until now. MKCL 1.2.0 features, among other things, a vastly improved separation between the global environment and the (transient) compilation environment. And the code you provided compiles and loads in it exactly as you would expect, without error, because of that separation. MKCL 1.2.0 has been in the works for far too long (almost two years now) but it is not quite ready for release yet (I need to finish 2 major features and add a new one). I hope to be able to initiate its release process in 2 to 3 months from now. In the meantime there is a workaround for this bug possible in MKCL 1.1.X. It is simple but not without some drawback. All you need to do is to bind special (dynamic) variable clos::*redefine-class-in-place* to T before doing any call (direct or indirect) to cl:load. Something like this would do, I tested it in my current MKCL 1.1.10 git: (let ((clos::*redefine-class-in-place* t)) (asdf::load-system :hello)) With this the global environment pollution of CL::COMPILE-FILE is paved over, and you can merrily attend the rest of your business. The drawback is that by doing so you may wake up cl:update-instance-for-redefined-class and its friend of the class redefinition protocol. And that protocol is definitely thread-unsafe! If you are sure not to share among threads any instance of your classes that would exist through the compilation/redefinition step of the said classes then you are safe. Otherwise you may witness surprising and unexpected results and I waive any responsibility into that since you have been warned. Regards, JCB P.S.: Bug reports of the quality of yours are a treat, thank you so much.
participants (4)
-
D. Martinez -
Jean-Claude Beaudoin -
Pascal J. Bourguignon -
Robert P. Goldman