Anyone know this area of the compiler? It's very frustrating during development. Seems we ought to, at least, be able to blow away all traces of the defstruct, ignoring existing structures and redefine it. My attempt to do that the obvious way - removing the system::structure-definition property from the symbol plist of the struct name - allows the redefinition but sometimes errors out when I use the new structure. Anyways, I'd like to fix this, and any info would give me a head start. Thanks, Alan
Hi Alan, On http://www.lispworks.com/documentation/lw70/CLHS/Body/m_defstr.htm, it says right below named arguments documentation section (or differently put: immediately above the "Predicate" section): The consequences of redefining a *defstruct* <http://www.lispworks.com/documentation/lw70/CLHS/Body/m_defstr.htm#defstruct> structure are undefined. You're running into that, I think. Regards, Erik. On Mon, Jul 11, 2022 at 7:43 PM Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
Anyone know this area of the compiler? It's very frustrating during development. Seems we ought to, at least, be able to blow away all traces of the defstruct, ignoring existing structures and redefine it. My attempt to do that the obvious way - removing the system::structure-definition property from the symbol plist of the struct name - allows the redefinition but sometimes errors out when I use the new structure.
Anyways, I'd like to fix this, and any info would give me a head start.
Thanks, Alan
-- Bye, Erik. http://efficito.com -- Hosted accounting and ERP. Robust and Flexible. No vendor lock-in.
On 11/07/2022 19:41, Alan Ruttenberg wrote:
Anyone know this area of the compiler? It's very frustrating during development.
Interesting. The rationale is performance apparently. CLTL2: 19.2. How to Use Defstruct https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node170.html whose last four paragraphs explain the reason. It concludes with: "The defstruct feature is intended to provide ``the most efficient'' structure class. CLOS classes defined by defclass allow much more flexible structures to be defined and redefined." ---- If you want flexibility, but also don't want to use classes instead of structs everywhere, one solution may be to define your own defstruct macro in some package that produces a class and functions/methods. Import and use that defstruct during development, but switch over to the real one when development ends (assuming you want its performance).
Seems we ought to, at least, be able to blow away all traces of the defstruct, ignoring existing structures and redefine it.
CLTL2 above agrees in its last para, "Programming environments are allowed and encouraged to permit defstruct redefinition, [...]" so it sounds like you'll have made ABCL better once your structure-definition approach succeeds. Others on this list will know more about this area and the errors you're seeing. Vibhu
This is what I came up with: https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc... Depending on the value of switch switch sys::*allow-defstruct-redefinition*. In order to allow the structure to be redefined, we delete the structure class, if there is already one. The undefined behavior is now 1. use of an existing struct from before the redefinition. 2. creation of functions with the same name as a structure element that has been removed. 3. running existing compiled code that uses an accessor for a slot that has changed relative position in the structure. #2 can be fixed by removing the source transformation for the accessor. (sys::%set-function-info accessor nil). It's not hard - involves iterating through the accessors just before the defstruct is redefined. I don't think I'm going to bother fixing this at the moment. #3 can be avoided by (declare (notinline accessor)) in the function being defined. Arguably this is what should be done if (declare (optimize (debug 3))). I could also have sys::not-inline-p return true if debug is 3. I may try to do this, since it will be easy to forget to recompile. We could at least provide warnings for such functions if we recorded that the source transform was applied, during compilation BTW, if you have an existing (regular) class and create a defstruct with the same name, it blows away the previous class. That probably deserves a warning. Comments welcome. Alan On Tue, Jul 12, 2022 at 3:44 AM Vibhu Mohindra <vibhu.mohindra@gmail.com> wrote:
On 11/07/2022 19:41, Alan Ruttenberg wrote:
Anyone know this area of the compiler? It's very frustrating during development.
Interesting. The rationale is performance apparently. CLTL2: 19.2. How to Use Defstruct https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node170.html whose last four paragraphs explain the reason. It concludes with: "The defstruct feature is intended to provide ``the most efficient'' structure class. CLOS classes defined by defclass allow much more flexible structures to be defined and redefined."
---- If you want flexibility, but also don't want to use classes instead of structs everywhere, one solution may be to define your own defstruct macro in some package that produces a class and functions/methods. Import and use that defstruct during development, but switch over to the real one when development ends (assuming you want its performance).
Seems we ought to, at least, be able to blow away all traces of the defstruct, ignoring existing structures and redefine it.
CLTL2 above agrees in its last para, "Programming environments are allowed and encouraged to permit defstruct redefinition, [...]" so it sounds like you'll have made ABCL better once your structure-definition approach succeeds. Others on this list will know more about this area and the errors you're seeing.
Vibhu
Instead of sys::*allow-defstruct-redefinition* it could be a restart. On Thu, Jul 14, 2022 at 12:50 AM Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
This is what I came up with:
https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc...
Depending on the value of switch switch sys::*allow-defstruct-redefinition*. In order to allow the structure to be redefined, we delete the structure class, if there is already one.
The undefined behavior is now 1. use of an existing struct from before the redefinition. 2. creation of functions with the same name as a structure element that has been removed. 3. running existing compiled code that uses an accessor for a slot that has changed relative position in the structure.
#2 can be fixed by removing the source transformation for the accessor. (sys::%set-function-info accessor nil). It's not hard - involves iterating through the accessors just before the defstruct is redefined. I don't think I'm going to bother fixing this at the moment.
#3 can be avoided by (declare (notinline accessor)) in the function being defined. Arguably this is what should be done if (declare (optimize (debug 3))). I could also have sys::not-inline-p return true if debug is 3. I may try to do this, since it will be easy to forget to recompile. We could at least provide warnings for such functions if we recorded that the source transform was applied, during compilation
BTW, if you have an existing (regular) class and create a defstruct with the same name, it blows away the previous class. That probably deserves a warning.
Comments welcome.
Alan
On Tue, Jul 12, 2022 at 3:44 AM Vibhu Mohindra <vibhu.mohindra@gmail.com> wrote:
On 11/07/2022 19:41, Alan Ruttenberg wrote:
Anyone know this area of the compiler? It's very frustrating during development.
Interesting. The rationale is performance apparently. CLTL2: 19.2. How to Use Defstruct https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node170.html whose last four paragraphs explain the reason. It concludes with: "The defstruct feature is intended to provide ``the most efficient'' structure class. CLOS classes defined by defclass allow much more flexible structures to be defined and redefined."
---- If you want flexibility, but also don't want to use classes instead of structs everywhere, one solution may be to define your own defstruct macro in some package that produces a class and functions/methods. Import and use that defstruct during development, but switch over to the real one when development ends (assuming you want its performance).
Seems we ought to, at least, be able to blow away all traces of the defstruct, ignoring existing structures and redefine it.
CLTL2 above agrees in its last para, "Programming environments are allowed and encouraged to permit defstruct redefinition, [...]" so it sounds like you'll have made ABCL better once your structure-definition approach succeeds. Others on this list will know more about this area and the errors you're seeing.
Vibhu
On Thu, 14 Jul 2022 at 00:50, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
This is what I came up with:
https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc...
Depending on the value of switch switch sys::*allow-defstruct-redefinition*. In order to allow the structure to be redefined, we delete the structure class, if there is already one.
The undefined behavior is now 1. use of an existing struct from before the redefinition. 2. creation of functions with the same name as a structure element that has been removed. 3. running existing compiled code that uses an accessor for a slot that has changed relative position in the structure.
#2 can be fixed by removing the source transformation for the accessor. (sys::%set-function-info accessor nil). It's not hard - involves iterating through the accessors just before the defstruct is redefined. I don't think I'm going to bother fixing this at the moment.
#3 can be avoided by (declare (notinline accessor)) in the function being defined. Arguably this is what should be done if (declare (optimize (debug 3))). I could also have sys::not-inline-p return true if debug is 3. I may try to do this, since it will be easy to forget to recompile. We could at least provide warnings for such functions if we recorded that the source transform was applied, during compilation
BTW, if you have an existing (regular) class and create a defstruct with the same name, it blows away the previous class. That probably deserves a warning.
Comments welcome.
Greetings from the (for the two decades of it) other side of the fence, where compilations and one-definition-rules are rather more static than here. :) Sure, this looks plausible, and it probably works in many cases. But if you COMPILE something with one definition of a defstruct, then defstruct again, what happens if you try to call the thing you compiled before? I don't claim to claim it "can't work". But I have a vague understanding why there might be a reason for "this might not work". :P As an unsubstantiated rumination, it might be *more* difficult to make this work in a language that can do dynamic compilation at any point in a program than it is in a language that is more static as far as struct definitions and their compilations are concerned. My architecture-brain can't tell how you could possibly know where all the 'references' to the old defstruct could possibly be, considering that compilations with the old one and uses of those can be so dynamic, and where the uses that would expect the newer redefinitions might be, and how you'd track that. Again, I'm not saying this can't work. I just find it daunting to even ponder what sort of funny situations where your program manages to confuse itself about which struct is which you can end up with. Maybe that's a theoretical problem, but it hurts my brain. :D
Given that structure accessors can be open coded you need a solution that can force recompilation. You could get that with a build system like asdf. You’d have to remember to rebuild the system instead of just recompiling the defstruct form, but that would work -- Robert P. Goldman On July 13, 2022 at 20:40:33, Ville Voutilainen (ville.voutilainen@gmail.com(mailto:ville.voutilainen@gmail.com)) wrote:
On Thu, 14 Jul 2022 at 00:50, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
This is what I came up with:
https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc...
Depending on the value of switch switch sys::*allow-defstruct-redefinition*. In order to allow the structure to be redefined, we delete the structure class, if there is already one.
The undefined behavior is now 1. use of an existing struct from before the redefinition. 2. creation of functions with the same name as a structure element that has been removed. 3. running existing compiled code that uses an accessor for a slot that has changed relative position in the structure.
#2 can be fixed by removing the source transformation for the accessor. (sys::%set-function-info accessor nil). It's not hard - involves iterating through the accessors just before the defstruct is redefined. I don't think I'm going to bother fixing this at the moment.
#3 can be avoided by (declare (notinline accessor)) in the function being defined. Arguably this is what should be done if (declare (optimize (debug 3))). I could also have sys::not-inline-p return true if debug is 3. I may try to do this, since it will be easy to forget to recompile. We could at least provide warnings for such functions if we recorded that the source transform was applied, during compilation
BTW, if you have an existing (regular) class and create a defstruct with the same name, it blows away the previous class. That probably deserves a warning.
Comments welcome.
Greetings from the (for the two decades of it) other side of the fence, where compilations and one-definition-rules are rather more static than here. :)
Sure, this looks plausible, and it probably works in many cases. But if you COMPILE something with one definition of a defstruct, then defstruct again, what happens if you try to call the thing you compiled before?
I don't claim to claim it "can't work". But I have a vague understanding why there might be a reason for "this might not work". :P
As an unsubstantiated rumination, it might be *more* difficult to make this work in a language that can do dynamic compilation at any point in a program than it is in a language that is more static as far as struct definitions and their compilations are concerned. My architecture-brain can't tell how you could possibly know where all the 'references' to the old defstruct could possibly be, considering that compilations with the old one and uses of those can be so dynamic, and where the uses that would expect the newer redefinitions might be, and how you'd track that.
Again, I'm not saying this can't work. I just find it daunting to even ponder what sort of funny situations where your program manages to confuse itself about which struct is which you can end up with. Maybe that's a theoretical problem, but it hurts my brain. :D
On Wed, Jul 13, 2022 at 11:40 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
Given that structure accessors can be open coded you need a solution that can force recompilation. You could get that with a build system like asdf. You’d have to remember to rebuild the system instead of just recompiling the defstruct form, but that would work
Thanks. Yes. I use asdf and was thinking that if I do get into the losing situation I would just do a forced reload of my system. It needs to be forced because the files that need to be recompiled haven't necessarily changed. Alan
-- Robert P. Goldman
On July 13, 2022 at 20:40:33, Ville Voutilainen ( ville.voutilainen@gmail.com) wrote:
On Thu, 14 Jul 2022 at 00:50, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
This is what I came up with:
https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc...
Depending on the value of switch switch sys::*allow-defstruct-redefinition*. In order to allow the structure to be redefined, we delete the structure class, if there is already one.
The undefined behavior is now 1. use of an existing struct from before the redefinition. 2. creation of functions with the same name as a structure element that has been removed. 3. running existing compiled code that uses an accessor for a slot that has changed relative position in the structure.
#2 can be fixed by removing the source transformation for the accessor. (sys::%set-function-info accessor nil). It's not hard - involves iterating through the accessors just before the defstruct is redefined. I don't think I'm going to bother fixing this at the moment.
#3 can be avoided by (declare (notinline accessor)) in the function being defined. Arguably this is what should be done if (declare (optimize (debug 3))). I could also have sys::not-inline-p return true if debug is 3. I may try to do this, since it will be easy to forget to recompile. We could at least provide warnings for such functions if we recorded that the source transform was applied, during compilation
BTW, if you have an existing (regular) class and create a defstruct with the same name, it blows away the previous class. That probably deserves a warning.
Comments welcome.
Greetings from the (for the two decades of it) other side of the fence, where compilations and one-definition-rules are rather more static than here. :)
Sure, this looks plausible, and it probably works in many cases. But if you COMPILE something with one definition of a defstruct, then defstruct again, what happens if you try to call the thing you compiled before?
I don't claim to claim it "can't work". But I have a vague understanding why there might be a reason for "this might not work". :P
As an unsubstantiated rumination, it might be *more* difficult to make this work in a language that can do dynamic compilation at any point in a program than it is in a language that is more static as far as struct definitions and their compilations are concerned. My architecture-brain can't tell how you could possibly know where all the 'references' to the old defstruct could possibly be, considering that compilations with the old one and uses of those can be so dynamic, and where the uses that would expect the newer redefinitions might be, and how you'd track that.
Again, I'm not saying this can't work. I just find it daunting to even ponder what sort of funny situations where your program manages to confuse itself about which struct is which you can end up with. Maybe that's a theoretical problem, but it hurts my brain. :D
Comments inline On Wed, Jul 13, 2022 at 9:40 PM Ville Voutilainen < ville.voutilainen@gmail.com> wrote:
On Thu, 14 Jul 2022 at 00:50, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
This is what I came up with:
https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc...
Depending on the value of switch switch
sys::*allow-defstruct-redefinition*. In order to allow the structure to be redefined, we delete the structure class, if there is already one.
The undefined behavior is now 1. use of an existing struct from before the redefinition. 2. creation of functions with the same name as a structure element that
3. running existing compiled code that uses an accessor for a slot that has changed relative position in the structure.
#2 can be fixed by removing the source transformation for the accessor. (sys::%set-function-info accessor nil). It's not hard - involves iterating through the accessors just before the defstruct is redefined. I don't think I'm going to bother fixing this at the moment.
#3 can be avoided by (declare (notinline accessor)) in the function being defined. Arguably this is what should be done if (declare (optimize (debug 3))). I could also have sys::not-inline-p return true if debug is 3. I may try to do this, since it will be easy to forget to recompile. We could at least provide warnings for such functions if we recorded
has been removed. that the source transform was applied, during compilation
BTW, if you have an existing (regular) class and create a defstruct with
the same name, it blows away the previous class.
That probably deserves a warning.
Comments welcome.
Greetings from the (for the two decades of it) other side of the fence, where compilations and one-definition-rules are rather more static than here. :)
Sure, this looks plausible, and it probably works in many cases. But if you COMPILE something with one definition of a defstruct, then defstruct again, what happens if you try to call the thing you compiled before?
Seems to work, as long as you don't allow the source transform. Currently I'm experimenting with a new option to defstruct. So I compile a file with (defstruct (test2 (:optimize nil)) a b) (defun foo (a) (test2-a a)) Then load it. Then (foo (make-test2 :a 10)) -> 10 Then eval (defstruct (test2 (:optimize nil)) b a) ;; swap order of slots (foo (make-test2 :a 10)) --> 10 ;; still the correct answer The :optimize nil option prevents the source transforms from being created. The source transforms are what put in positional accessors the generated code like: lispObject.getSlotValue_0 (in the disassembly). Without the source transformation, it compiles to a call to test-2-a. If it didn't then when defstruct was redefined the fixed position accessor could get the wrong slot. But, when you redefine the defstruct test2-a is also redefined. The new function will get the wrong answer if you give it a struct that was created *before* defstruct is redefined. But that's what I've noted in my case 1 above as undefined behavior.
I don't claim to claim it "can't work". But I have a vague understanding why there might be a reason for "this might not work". :P
As an unsubstantiated rumination, it might be *more* difficult to make this work in a language that can do dynamic compilation at any point in a program than it is in a language that is more static as far as struct definitions and their compilations are concerned. My architecture-brain can't tell how you could possibly know where all the 'references' to the old defstruct could possibly be, considering that compilations with the old one and uses of those can be so dynamic, and where the uses that would expect the newer redefinitions might be, and how you'd track that.
It would be possible if there was an annotation on a function that noted the use of a structure accessor. Looking for that annotation across all functions will identify all uses of the accessors, which are *potential* problems. But you would have to sort new from old. However, if the warnings are given just before the new defstruct is created then only functions that were defined before would get warned about. It would be your job to recompile those. But we would only have to do that if the positional assessors were inlined. The (:optimize nil) forces that not to happen. There's still an issue if you first defined a defstruct without using (:optimize nil), and then redefined it with (:optimize nil). In that case you may lose if you delete or reorder the slots. Not a problem if your redefinition is just adding slots to the end. Anyways, Don't Do That. Otherwise you will get bugs unless I implement something like what's described in the previous paragraph. In my use case it's fine because I'm using the option in the first place. Again, I'm not saying this can't work. I just find it daunting to even
ponder what sort of funny situations where your program manages to confuse itself about which struct is which you can end up with. Maybe that's a theoretical problem, but it hurts my brain. :D
There's definitely a screw case, as I explained above. It's a compromise. You get the ability to redefine defstruct, but if you add :optimize nil in the redefinition without it being in the original, then you might have stale compiled code with fixed position accessors. I'm thinking that's a lot better than the current situation, where you can't redefine defstruct at all. I'm also only planning on doing this while developing the code. I'm leaning towards using the defstruct :optimize option vs the (declare (optimize (debug 3))) , since the latter hurts performance of everything. The defstruct option is more targeted. Alan
On Thu, 14 Jul 2022 at 05:33, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
Comments inline Seems to work, as long as you don't allow the source transform. Currently I'm experimenting with a new option to defstruct. So
I compile a file with (defstruct (test2 (:optimize nil)) a b)
I'm thinking I'm stupid, since I'm not sure what you mean by "source transform", but I can with fair confidence say that if you can ensure that all defstructs use the option you add as a custom-thingy, then the rest of the concerns I spoke of are moot. :P You're basically saying "this is not a regular defstruct, it's a special one, don't do the usual stuff, and then I can make it mean whatever I want it to mean, side-stepping the problems". If that works for your use cases, sure, seems plausible in general. Or have I misunderstood your approach?
On Wed, Jul 13, 2022 at 10:53 PM Ville Voutilainen < ville.voutilainen@gmail.com> wrote:
On Thu, 14 Jul 2022 at 05:33, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
Comments inline Seems to work, as long as you don't allow the source transform.
Currently I'm experimenting with a new option to defstruct. So
I compile a file with (defstruct (test2 (:optimize nil)) a b)
I'm thinking I'm stupid, since I'm not sure what you mean by "source transform",
ABCL has a mechanism for source transformations, which effectively inline some code instead of calling a function. You can get a source transformation with sys:function-info. For example: (sys::function-info my-accessor) ((:source-transform . #<anonymous-function abcl_b09e0b54_a7c4_4513_86e6_fd9effb1fb25 {27C03870}>)) These are applied in the compiler and aren't affected, currently, by the (declare (optimize ..)) settings. They are used for defstruct in define-reader and define-writer. The implementation I discuss is here: https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc... but I can with fair confidence say that if
you can ensure that all defstructs use the option you add as a custom-thingy, then the rest of the concerns I spoke of are moot. :P You're basically saying "this is not a regular defstruct, it's a special one, don't do the usual stuff, and then I can make it mean whatever I want it to mean, side-stepping the problems". If that works for your use cases, sure, seems plausible in general. Or have I misunderstood your approach?
Sort of. There's no usual stuff around redefining defstructs, so it's more of an added feature. It is implementation dependent, but then so is JSS which I understand many people find useful. I'm hoping it would be useful for anyone who needs to develop defstructs during initial code writing and testing. It doesn't change any behavior if you don't use the option, so it's safe. It's not a perfect solution, but the perfect is the enemy of the good. I will keep thinking about whether there's a way to remove the potential gotchas. FWIW, my use case is developing some code using lisp-binary <https://github.com/j3pic/lisp-binary>(super useful for parsing binary files). It generates defstructs with some extra code around them. But since I don't know it well enough I'm still making mistakes and needing to redefine things. Before this change the only option was to restart lisp if I made a mistake, which gets old quickly. If I had my druthers, lisp-binary would have generated classes instead of defstructs, since there are no problems with redefinitions there. I may try to modify lisp-binary to do that at some point... There's a more conservative approach which fits better with "regular" defstruct usage, which is to allow adding slots to defstruct, but not reordering or deleting. Even that can't be done now. I started to implement that, which wouldn't need as much care to use it. But it turned out that just wasn't enough for the development I was doing now. Not having even that capability has burned me in the past. YMMV. Alan
Here's the patch for (:optimize nil) option. https://github.com/alanruttenberg/abcl/commit/81d3a202544f2071aa6acbbeeb893f... Thanks for the comments, keep them coming. On Wed, Jul 13, 2022 at 10:33 PM Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
Comments inline
On Wed, Jul 13, 2022 at 9:40 PM Ville Voutilainen < ville.voutilainen@gmail.com> wrote:
On Thu, 14 Jul 2022 at 00:50, Alan Ruttenberg <alanruttenberg@gmail.com> wrote:
This is what I came up with:
https://github.com/alanruttenberg/abcl/commit/a9c5541d372012d24c0daa704a22fc...
Depending on the value of switch switch
sys::*allow-defstruct-redefinition*. In order to allow the structure to be redefined, we delete the structure class, if there is already one.
The undefined behavior is now 1. use of an existing struct from before the redefinition. 2. creation of functions with the same name as a structure element
3. running existing compiled code that uses an accessor for a slot
that has been removed. that has changed relative position in the structure.
#2 can be fixed by removing the source transformation for the accessor. (sys::%set-function-info accessor nil). It's not hard - involves
I don't think I'm going to bother fixing this at the moment.
#3 can be avoided by (declare (notinline accessor)) in the function being defined. Arguably this is what should be done if (declare (optimize (debug 3))). I could also have sys::not-inline-p return true if debug is 3. I may
We could at least provide warnings for such functions if we recorded
iterating through the accessors just before the defstruct is redefined. try to do this, since it will be easy to forget to recompile. that the source transform was applied, during compilation
BTW, if you have an existing (regular) class and create a defstruct
with the same name, it blows away the previous class.
That probably deserves a warning.
Comments welcome.
Greetings from the (for the two decades of it) other side of the fence, where compilations and one-definition-rules are rather more static than here. :)
Sure, this looks plausible, and it probably works in many cases. But if you COMPILE something with one definition of a defstruct, then defstruct again, what happens if you try to call the thing you compiled before?
Seems to work, as long as you don't allow the source transform. Currently I'm experimenting with a new option to defstruct. So
I compile a file with (defstruct (test2 (:optimize nil)) a b) (defun foo (a) (test2-a a)) Then load it. Then (foo (make-test2 :a 10)) -> 10 Then eval (defstruct (test2 (:optimize nil)) b a) ;; swap order of slots (foo (make-test2 :a 10)) --> 10 ;; still the correct answer
The :optimize nil option prevents the source transforms from being created. The source transforms are what put in positional accessors the generated code like: lispObject.getSlotValue_0 (in the disassembly). Without the source transformation, it compiles to a call to test-2-a. If it didn't then when defstruct was redefined the fixed position accessor could get the wrong slot. But, when you redefine the defstruct test2-a is also redefined. The new function will get the wrong answer if you give it a struct that was created *before* defstruct is redefined. But that's what I've noted in my case 1 above as undefined behavior.
I don't claim to claim it "can't work". But I have a vague understanding why there might be a reason for "this might not work". :P
As an unsubstantiated rumination, it might be *more* difficult to make this work in a language that can do dynamic compilation at any point in a program than it is in a language that is more static as far as struct definitions and their compilations are concerned. My architecture-brain can't tell how you could possibly know where all the 'references' to the old defstruct could possibly be, considering that compilations with the old one and uses of those can be so dynamic, and where the uses that would expect the newer redefinitions might be, and how you'd track that.
It would be possible if there was an annotation on a function that noted the use of a structure accessor. Looking for that annotation across all functions will identify all uses of the accessors, which are *potential* problems. But you would have to sort new from old. However, if the warnings are given just before the new defstruct is created then only functions that were defined before would get warned about. It would be your job to recompile those. But we would only have to do that if the positional assessors were inlined. The (:optimize nil) forces that not to happen.
There's still an issue if you first defined a defstruct without using (:optimize nil), and then redefined it with (:optimize nil). In that case you may lose if you delete or reorder the slots. Not a problem if your redefinition is just adding slots to the end. Anyways, Don't Do That. Otherwise you will get bugs unless I implement something like what's described in the previous paragraph.
In my use case it's fine because I'm using the option in the first place.
Again, I'm not saying this can't work. I just find it daunting to even
ponder what sort of funny situations where your program manages to confuse itself about which struct is which you can end up with. Maybe that's a theoretical problem, but it hurts my brain. :D
There's definitely a screw case, as I explained above. It's a compromise. You get the ability to redefine defstruct, but if you add :optimize nil in the redefinition without it being in the original, then you might have stale compiled code with fixed position accessors. I'm thinking that's a lot better than the current situation, where you can't redefine defstruct at all. I'm also only planning on doing this while developing the code.
I'm leaning towards using the defstruct :optimize option vs the (declare (optimize (debug 3))) , since the latter hurts performance of everything. The defstruct option is more targeted.
Alan
participants (6)
-
Alan Ruttenberg -
Erik Huelsmann -
Robert P. Goldman -
Stas Boukarev -
Vibhu Mohindra -
Ville Voutilainen