Using ASDF as part of an interactive app framework - storing components in memory?
I'm interested in using ASDF to build interactive applications. Here's an example of the kind of thing I'd like to do. Imagine creating a Web-based multiple choice poll. When someone submits an entry to the poll, a data file containing all the responses is updated with the new entry. Two SVG graphics, a bar chart and a pie chart of the answers, are also updated according to the updated data set. If I wanted to implement something like this in Common Lisp, ASDF seems like a good foundation to build on, since it allows me to create a complex dependency graph of code and data files. In the example above, the SVG visualizations are dependent on the answers data set, and they get updated when the answer data is updated. However, I don't know how well this approach would scale because in order to use ASDF this way, a file needs to be written to disk and other files must be loaded from disk and compiled each time an entry is made to the poll. Would it make sense to create a new component type that represents a quoted piece of code stored as a variable in a running program rather than a file? If the poll is implemented with a persistent Web server, it could keep the answer data and the graphics in its memory instead of on disk, and use some means to periodically back up the contents of those variables to disk. I'm still new to ASDF so I'm not sure if there's a simple way I could add in this functionality or if it's a good idea. I know that the Web poll example I gave above would work better if the graphics generation and answer storage was just written into the Web server instead of using ASDF, but the poll is just an example; my goal is to develop a system that can handle much more complex code/data graphs. Any feedback appreciated. Thanks, Andrew
If you are indeed using files for all your dependency nodes, I suppose you could use ASDF for this kind of things. In that case, package-inferred-system would be a good template to start from, in terms of letting ASDF dynamically build a model from your files. However, if any of your nodes does not correspond to files, then maybe you should consider cells, or computed-class, or some reactive functional programming system. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org To stay young requires unceasing cultivation of the ability to unlearn old falsehoods. — Robert Heinlein, "Time Enough For Love" On Tue, Jul 19, 2016 at 5:38 AM, Andrew Sengul <ml@imagegardenphoto.com> wrote:
I'm interested in using ASDF to build interactive applications. Here's an example of the kind of thing I'd like to do. Imagine creating a Web-based multiple choice poll. When someone submits an entry to the poll, a data file containing all the responses is updated with the new entry. Two SVG graphics, a bar chart and a pie chart of the answers, are also updated according to the updated data set.
If I wanted to implement something like this in Common Lisp, ASDF seems like a good foundation to build on, since it allows me to create a complex dependency graph of code and data files. In the example above, the SVG visualizations are dependent on the answers data set, and they get updated when the answer data is updated.
However, I don't know how well this approach would scale because in order to use ASDF this way, a file needs to be written to disk and other files must be loaded from disk and compiled each time an entry is made to the poll. Would it make sense to create a new component type that represents a quoted piece of code stored as a variable in a running program rather than a file? If the poll is implemented with a persistent Web server, it could keep the answer data and the graphics in its memory instead of on disk, and use some means to periodically back up the contents of those variables to disk.
I'm still new to ASDF so I'm not sure if there's a simple way I could add in this functionality or if it's a good idea. I know that the Web poll example I gave above would work better if the graphics generation and answer storage was just written into the Web server instead of using ASDF, but the poll is just an example; my goal is to develop a system that can handle much more complex code/data graphs.
Any feedback appreciated. Thanks,
Andrew
Drew McDermott made a build system called (IIRC) "chunks," that did just what you want: it put dependencies on arbitrary data structures and recomputed them on need. One challenge is that his build system is built on top of his "yale-tools" mega-library, which can be difficult for outsiders.... Here's Drew's paper, A Framework for Maintaining the Coherence of a Running Lisp: http://www.cs.yale.edu/homes/dvm/papers/lisp05.pdf I suggest you read this, and see if you could graft the ideas onto ASDF (with suggestions from asdf-devel). ASDF mostly uses the filesystem to track dependencies (although it has an internal db, as well), so there would be a little effort to extend to a non-file-centric build. Drew's key idea is that his system (and ASDF), unlike build systems like make or ant, have as a primary job maintaining the integrity of the running lisp image. [Faré may disagree with this claim: he comes to Lisp usage with a newer perspective than my old-school view.] Drew's system is less fine-grained than the cells system that Faré suggests. I don't think the cells system ever got mature enough for use. If you want a dependency-driven, lisp-based constraint system per Faré, I think you'd have better luck with the KR system developed a long time ago at CMU. It is primitive in some ways (hand-built on structures, rather than using CLOS), but was used for years in a very high-stress environment, so is much more likely to work than cells. KR is available as part of the Garnet UI system. Drop me a line if you are interested. Best, r
Hi Fare, Robert and Dave, Thanks for your suggestions, I read Drew's paper and his chunks system is close to the kind of thing I'm looking to implement. I also looked at cells, Garnet and Gendl. I downloaded Garnet and have been looking through the KR code; Robert, is there anything else you would suggest if I'm going to try using KR? I'm going to use ASDF as the basis for this project at least to start with. One of my goals is for the systems I'm building to be usable as portable software modules, so implementing them as ASDF packages is a natural choice. At least at the outset, I don't have a problem with ASDF's model being filesystem-based; it's fine for the components of each system to be defined in the form of files. I think requiring that a file be saved whenever a change is made to a component is inefficient, but that can be left for now. My eventual goal will probably be to combine a constraint-solving system with ASDF; perhaps chunks can be implemented as part of an ASDF extension. I've got the chunks code but it doesn't compile for me with SBCL, so more work will be needed on that front. Any thoughts you have on this direction for the project are welcome. Thanks, Andrew On 07/19/2016 07:13 AM, Robert Goldman wrote:
Drew McDermott made a build system called (IIRC) "chunks," that did just what you want: it put dependencies on arbitrary data structures and recomputed them on need. One challenge is that his build system is built on top of his "yale-tools" mega-library, which can be difficult for outsiders....
Here's Drew's paper, A Framework for Maintaining the Coherence of a Running Lisp: http://www.cs.yale.edu/homes/dvm/papers/lisp05.pdf
I suggest you read this, and see if you could graft the ideas onto ASDF (with suggestions from asdf-devel).
ASDF mostly uses the filesystem to track dependencies (although it has an internal db, as well), so there would be a little effort to extend to a non-file-centric build.
Drew's key idea is that his system (and ASDF), unlike build systems like make or ant, have as a primary job maintaining the integrity of the running lisp image. [Faré may disagree with this claim: he comes to Lisp usage with a newer perspective than my old-school view.]
Drew's system is less fine-grained than the cells system that Faré suggests. I don't think the cells system ever got mature enough for use.
If you want a dependency-driven, lisp-based constraint system per Faré, I think you'd have better luck with the KR system developed a long time ago at CMU. It is primitive in some ways (hand-built on structures, rather than using CLOS), but was used for years in a very high-stress environment, so is much more likely to work than cells. KR is available as part of the Garnet UI system. Drop me a line if you are interested.
Best, r
On 7/21/16 Jul 21 -5:47 PM, Andrew Sengul wrote:
Hi Fare, Robert and Dave,
Thanks for your suggestions, I read Drew's paper and his chunks system is close to the kind of thing I'm looking to implement. I also looked at cells, Garnet and Gendl. I downloaded Garnet and have been looking through the KR code; Robert, is there anything else you would suggest if I'm going to try using KR?
Not really. I only ever used it as part of the Garnet system. Garnet, FWIW, is really awesome but.... (1) it's bitrotted (especially its X interaction and its pre-ASDF build system); (2) learning curve; (3) stock look and feel is so nasty that you don't realize how awesome it is (when I was using it, I had the great good fortune of working with a human interface expert who told me how to fix the default color scheme, widget shapes, and font choices, but all that customization was lost in a proprietary corporate filesystem :-() Are there any cells examples lying around? Recoding them might be reasonable. Or it should be trivial to find examples intended for a Truth Maintenance System and translate them into KR. Maybe something from Forbus & deKleer's "Building Problem Solvers" would be suitable.
I'm going to use ASDF as the basis for this project at least to start with. One of my goals is for the systems I'm building to be usable as portable software modules, so implementing them as ASDF packages is a natural choice. At least at the outset, I don't have a problem with ASDF's model being filesystem-based; it's fine for the components of each system to be defined in the form of files. I think requiring that a file be saved whenever a change is made to a component is inefficient, but that can be left for now. My eventual goal will probably be to combine a constraint-solving system with ASDF; perhaps chunks can be implemented as part of an ASDF extension. I've got the chunks code but it doesn't compile for me with SBCL, so more work will be needed on that front.
I tried for a while to get Drew's yale tools to compile, but I never was fully successful. On my "someday I'd like to do this" list is extracting some of that stuff and making it more compatible with ordinary lisp usage.
Any thoughts you have on this direction for the project are welcome. Thanks,
Sorry, I'm afraid those were not too helpful. best, r
Andrew
On 07/19/2016 07:13 AM, Robert Goldman wrote:
Drew McDermott made a build system called (IIRC) "chunks," that did just what you want: it put dependencies on arbitrary data structures and recomputed them on need. One challenge is that his build system is built on top of his "yale-tools" mega-library, which can be difficult for outsiders....
Here's Drew's paper, A Framework for Maintaining the Coherence of a Running Lisp: http://www.cs.yale.edu/homes/dvm/papers/lisp05.pdf
I suggest you read this, and see if you could graft the ideas onto ASDF (with suggestions from asdf-devel).
ASDF mostly uses the filesystem to track dependencies (although it has an internal db, as well), so there would be a little effort to extend to a non-file-centric build.
Drew's key idea is that his system (and ASDF), unlike build systems like make or ant, have as a primary job maintaining the integrity of the running lisp image. [Faré may disagree with this claim: he comes to Lisp usage with a newer perspective than my old-school view.]
Drew's system is less fine-grained than the cells system that Faré suggests. I don't think the cells system ever got mature enough for use.
If you want a dependency-driven, lisp-based constraint system per Faré, I think you'd have better luck with the KR system developed a long time ago at CMU. It is primitive in some ways (hand-built on structures, rather than using CLOS), but was used for years in a very high-stress environment, so is much more likely to work than cells. KR is available as part of the Garnet UI system. Drop me a line if you are interested.
Best, r
participants (3)
-
Andrew Sengul -
Faré -
Robert Goldman