Hi,
Some time ago I could put the point on a function and press Alt-. and slime would take me to the function definition (or a list of possibilities in the case of generics, etc). After one of the updates I did awhile back it stopped working in many circumstances although it contines to work in others.
The case it doesn't work in is the one that effects me on a daily basis. Here's two examples:
1. I have a piece of development code with multiple files. The main file loads the other lisp files wherein functions are defined. The only way I can get the operation to work is if I manually compile each function individually. Simply loading the file at any point will reset slime back to not knowing where they are again.
2. Similar situation, only this time using ASD:
(in-package :cl-user) (defpackage :ftis-v43-asd (:use :cl :asdf)) (in-package :ftis-v43-asd) (defparameter *ftis-v43-directory* (make-pathname :name nil :type nil :version nil :defaults (parse-namestring *load-truename*))) (asdf:defsystem #:ftis-v43 :depends-on (:cl-extra :cl-ppcre :graham :cl-fad :json-pt :html5 :cl-who :cl-gd-ext) :serial t :components ((:file "ftis-v43-package") (:file "common") (:file "vectors") (:file "waveforms") (:file "interface") (:file "model") (:file "post-processing") (:file "octave") (:file "verify") (:file "make-html-pages")))
And the package.lisp file like this:
(defpackage :ftis-v43 (:use :cl :cl-extra :cl-ppcre :cl-fad :graham :cl-gd-ext :json-pt :html5 :cl-who) (:shadowing-import-from :cl-who :str :htm :fmt) (:export #:gen-vhdl-vectors #:gen-1ping-per-setname #:gen-vhdl-mping-vectors #:gen-mping-per-setname #:run-verification #:compare-model-octave #:compare-model-octave-all #:unpack-new-tarball #:process-vhdl-tarball #:process-all-vhdl-tarballs))
The behavior is identical: Alt-. does not work until I manually open one of the source files and compile that function.
Other packages I've written and load through ASD work as they should. I can't see any difference in the way they are constructed.
Is there any remedy for this? Using grep to move around works but is annoyingly cumbersome and time consuming.
Regards,
--Jeff
On Sat, Oct 26 2013, Jeff Cunningham wrote:
[...]
Is there any remedy for this? Using grep to move around works but is annoyingly cumbersome and time consuming.
I don't see any obvious problem with the bits you posted and without a reproducable test case it's hard to say what the actual problem is.
One problem could be that you have some declaration/proclamation somewhere, e.g. (proclaim '(optimize (debug 0))). If you're using SBCL and debug is 0 then SLIME can't find the definition for a name. I have (proclaim '(optimize (debug 2))) in my .sbclrc and that seems to work quite OK.
Another cause could be that you load the source and not the fasl file. M-. does typically not work so well for non-compiled functions.
Yet another problem could be that there is no IN-PACKAGE form in the file so that SLIME searches in the wrong package. (That's unlikey, tho, as compiling an individual function seems to work.)
In general, M-. seems to be used by many people therefore I think we would note regressions on our side pretty quickly.
Helmut
On 10/27/2013 12:06 AM, Helmut Eller wrote:
On Sat, Oct 26 2013, Jeff Cunningham wrote:
[...]
Is there any remedy for this? Using grep to move around works but is annoyingly cumbersome and time consuming.
I don't see any obvious problem with the bits you posted and without a reproducable test case it's hard to say what the actual problem is.
One problem could be that you have some declaration/proclamation somewhere, e.g. (proclaim '(optimize (debug 0))). If you're using SBCL and debug is 0 then SLIME can't find the definition for a name. I have (proclaim '(optimize (debug 2))) in my .sbclrc and that seems to work quite OK.
Another cause could be that you load the source and not the fasl file. M-. does typically not work so well for non-compiled functions.
Yet another problem could be that there is no IN-PACKAGE form in the file so that SLIME searches in the wrong package. (That's unlikey, tho, as compiling an individual function seems to work.)
In general, M-. seems to be used by many people therefore I think we would note regressions on our side pretty quickly.
Helmut
Thanks for the response, Helmut. I was hoping there was something I was missing.
I don't normally change the optimization in development code. But I went ahead and tried the proclaim '(optimize (ebug 2))) in my .sblcrc as you suggested, but that made no difference.
All my source files have (in-package 'ftis-v43), the package defined in the package.lisp file.
I'm guessing it has something to do with not loading local fasl files, because if I open each of the lisp sources and compile fasls for them with Ctrl-c, k I can then navigate around with Alt-. But I still have to compile the files manually. Simply loading via ASD leaves it in a state where it can't navigate.
On the other hand, i can put the point on a function from an inluded package, say something like cl-ppcre:scan and it will go off and find that just fine.
There's something I'm missing.
--Jeff
Jeff Cunningham jeffrey@jkcunningham.com writes:
I don't normally change the optimization in development code. But I went ahead and tried the proclaim '(optimize (ebug 2))) in my .sblcrc as you suggested, but that made no difference.
All my source files have (in-package 'ftis-v43), the package defined in the package.lisp file.
(in-package 'ftis-v43) is not actually legal common lisp. (The argument to IN-PACKAGE is an unevaluated name). Do things start working if you use (in-package "FTIS-V43") or (in-package :ftis-v43)? I know this might be a red herring, since (a) you're talking about fasls and (b) the source finding works for individually-compiled buffers, but it might be worth knowing in any case.
Best,
Christophe
On 10/29/2013 09:36 AM, Christophe Rhodes wrote:
Jeff Cunningham jeffrey@jkcunningham.com writes:
I don't normally change the optimization in development code. But I went ahead and tried the proclaim '(optimize (ebug 2))) in my .sblcrc as you suggested, but that made no difference.
All my source files have (in-package 'ftis-v43), the package defined in the package.lisp file.
(in-package 'ftis-v43) is not actually legal common lisp. (The argument to IN-PACKAGE is an unevaluated name). Do things start working if you use (in-package "FTIS-V43") or (in-package :ftis-v43)? I know this might be a red herring, since (a) you're talking about fasls and (b) the source finding works for individually-compiled buffers, but it might be worth knowing in any case.
That was a typo - all of them are (in-package :ftis-v43), I just checked. Currently, if I start a fresh sbcl under slime the only functions I can navigate to with Alt-. are system functions (format, for example). Any packages I am loading via quicklisp or of my own result in:
Error: end of file on #<SB-IMPL::STRING-INPUT-STREAM {10070EBBB3}>
When I load packages creates and/or loads a fasl under ~/.cache/common-lisp/<sbcl version>/.. I would have thought that would be how it would navigate, but in my case not.
I can open any source file, compile a LOCAL fasl with C-c k and then those functions can be navigated into. It used to work on anything. And I haven't changed my .sbclrc in years (except to add the suggestion yesterday). Here it is:
;; -*-Lisp-*- (require 'asdf) (proclaim '(optimize (debug 2)))
;;; The following lines added by ql:add-to-init-file: #-quicklisp (let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp" (user-homedir-pathname)))) (when (probe-file quicklisp-init) (load quicklisp-init)))
;;; If a fasl was stale, try to recompile and load (once). (defmethod asdf:perform :around ((o asdf:load-op) (c asdf:cl-source-file)) (handler-case (call-next-method o c) ;; If a fasl was stale, try to recompile and load (once). (sb-ext:invalid-fasl () (asdf:perform (make-instance 'asdf:compile-op) c) (call-next-method))))
Thanks, --Jeff
Jeff Cunningham jeffrey@jkcunningham.com writes:
That was a typo - all of them are (in-package :ftis-v43), I just checked. Currently, if I start a fresh sbcl under slime the only functions I can navigate to with Alt-. are system functions (format, for example). Any packages I am loading via quicklisp or of my own result in:
Error: end of file on #<SB-IMPL::STRING-INPUT-STREAM {10070EBBB3}>
When I load packages creates and/or loads a fasl under ~/.cache/common-lisp/<sbcl version>/.. I would have thought that would be how it would navigate, but in my case not.
I can open any source file, compile a LOCAL fasl with C-c k and then those functions can be navigated into. It used to work on anything. And I haven't changed my .sbclrc in years (except to add the suggestion yesterday). Here it is:
Well, I just did a quick test and I don't see this problem, though inevitably my setup does not mirror yours terribly well.
On the assumption that you have cl-ppcre relatively easily to hand, what does (describe 'cl-ppcre:regex-apropos) say after you have loaded cl-ppcre through quicklisp? (If you don't have cl-ppcre, substitute your favourite simple "utility")
Cheers,
Christophe
On 10/30/2013 01:24 AM, Christophe Rhodes wrote:
Jeff Cunningham jeffrey@jkcunningham.com writes:
That was a typo - all of them are (in-package :ftis-v43), I just checked. Currently, if I start a fresh sbcl under slime the only functions I can navigate to with Alt-. are system functions (format, for example). Any packages I am loading via quicklisp or of my own result in:
Error: end of file on #<SB-IMPL::STRING-INPUT-STREAM {10070EBBB3}>
When I load packages creates and/or loads a fasl under ~/.cache/common-lisp/<sbcl version>/.. I would have thought that would be how it would navigate, but in my case not.
I can open any source file, compile a LOCAL fasl with C-c k and then those functions can be navigated into. It used to work on anything. And I haven't changed my .sbclrc in years (except to add the suggestion yesterday). Here it is:
Well, I just did a quick test and I don't see this problem, though inevitably my setup does not mirror yours terribly well.
On the assumption that you have cl-ppcre relatively easily to hand, what does (describe 'cl-ppcre:regex-apropos) say after you have loaded cl-ppcre through quicklisp? (If you don't have cl-ppcre, substitute your favourite simple "utility")
Cheers,
Christophe
Here's what it says:
FTIS-V43> (describe 'cl-ppcre:regex-apropos) CL-PPCRE:REGEX-APROPOS [symbol]
REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. Source file: /home/jcunningham/qigj/test-model.lisp ; No value
Thanks, Jeff
Jeff Cunningham jeffrey@jkcunningham.com writes:
On 10/30/2013 01:24 AM, Christophe Rhodes wrote:
On the assumption that you have cl-ppcre relatively easily to hand, what does (describe 'cl-ppcre:regex-apropos) say after you have loaded cl-ppcre through quicklisp? (If you don't have cl-ppcre, substitute your favourite simple "utility")
Here's what it says:
FTIS-V43> (describe 'cl-ppcre:regex-apropos) [...] Source file: /home/jcunningham/qigj/test-model.lisp
Say, what? regex-apropos is defined in cl-ppcre/api.lisp, at least in my copy. Can you come up with any idea why this might happen? (I do not have any idea what qigj is, nor what might be inside test-model.lisp)
Cheers,
Christophe
New test: I updated all dists with quicklisp which reinstalled cl-ppcre and slime. I started a fresh emacs and Slime (9/2013) and ran the following test1.lisp (using asdf:load-system instead of require).
;; (require "cl-ppcre") (asdf:load-system "cl-ppcre") (describe 'cl-ppcre:regex-apropos)
After compiling the fresh cl-ppcre code it ran and gave me the following:
; compiling file "/home/jcunningham/slime/test1.lisp" (written 30 OCT 2013 01:10:24 PM):
; /home/jcunningham/slime/test1.fasl written ; compilation finished in 0:00:00.003 CL-PPCRE:REGEX-APROPOS [symbol]
REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. *Source file: /home/jcunningham/slime/test1.lisp** *
Now it's switched its "source" for cl-ppcre to the test file.
--Jeff
On 10/30/2013 09:35 AM, Christophe Rhodes wrote:
Jeff Cunningham jeffrey@jkcunningham.com writes:
On 10/30/2013 01:24 AM, Christophe Rhodes wrote:
On the assumption that you have cl-ppcre relatively easily to hand, what does (describe 'cl-ppcre:regex-apropos) say after you have loaded cl-ppcre through quicklisp? (If you don't have cl-ppcre, substitute your favourite simple "utility")
Here's what it says:
FTIS-V43> (describe 'cl-ppcre:regex-apropos) [...] Source file: /home/jcunningham/qigj/test-model.lisp
Say, what? regex-apropos is defined in cl-ppcre/api.lisp, at least in my copy. Can you come up with any idea why this might happen? (I do not have any idea what qigj is, nor what might be inside test-model.lisp)
I've been noticing that for some time. I'm sure it is a related symptom. Slime thinks that's where regex-apropos is defined. In fact, it's in its normal quicklisp location. the source file given is the code I loaded the ftis-v43 package from within. And that package includes cl-ppcre and everything else in its definition.
For example, if put point on a call like this (cl-ppcre:scan re "some text") - on the "scan" I mean - and press C-c,d,d it shows me the docstring as it should, but shows the source location - again - as that same test-model.lisp source file.
On 10/30/2013 12:21 PM, Jeff Cunningham wrote:
(describe 'cl-ppcre:regex-apropos)
This is really strange. As an experiment I just created a dirt-simple source file named "test1.lisp" in an isolated directory.
(require :cl-ppcre) (use-package :cl-ppcre) (describe 'cl-ppcre:regex-apropos)
When I compile and run this I get the following:
; compiling file "/home/jcunningham/slime/test1.lisp" (written 30 OCT 2013 12:50:50 PM):
; /home/jcunningham/slime/test1.fasl written ; compilation finished in 0:00:00.002 CL-PPCRE:REGEX-APROPOS [symbol]
REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. *Source file: /home/jcunningham/quest/robom/plotwave.lisp*
This is a fresh slime session started in a new emacs. The file it is pointing to is something I haven't been working with for more than a week and most definitely does NOT contain regex-apropos by any package name. But it does contain a defpackage, which looks like this:
(require 'cl-fad) (require 'cl-extra) (require 'cl-ppcre) (require 'quaternion) (require 'graham)) (defpackage #:cnc-user (:use :cl :quaternion :cl-extra :graham :cl-fad :cl-ppcre)) (in-package :cnc-user)
It would appear that somehow when I last compiled this piece of code it modified something persistently for slime so that it now thinks - from any session I start - that functions in these packages have source code located in this file. The functions are found by SBCL because they do work. It is Slime that is going wrong here.
I see that I was using the 'cl-ppcre designation for the package that you say is not legal Common Lisp. I have no idea how that would affect things. I will change this in the future, but I am at a loss to see how to get back to the behavior Alt-. once had.
I am running SBCL 1.1.11 on this machine, and SLIME 2013-06-26
Regards, --Jeff
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
On 10/30/2013 12:21 PM, Jeff Cunningham wrote:
(describe 'cl-ppcre:regex-apropos)
This is really strange. As an experiment I just created a dirt-simple source file named "test1.lisp" in an isolated directory.
(require :cl-ppcre)
I suspect that this might be the source of your problems. If you have this kind of thing in your lisp code files, I can sort-of understand why you might be getting different answers, because this isn't an idiomatic way of doing things. Normally, you would require modules your system depends on from the defsystem, not from within the source files. (In fact, your example as you state it should not work at all, at least in a clean image, because the file test1.lisp as you describe it should not be compilable, as it contains read-time references to a package which is only available at load-time.
(use-package :cl-ppcre) (describe 'cl-ppcre:regex-apropos)
Christophe
On 10/30/2013 01:27 PM, Christophe Rhodes wrote:
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
On 10/30/2013 12:21 PM, Jeff Cunningham wrote:
(describe 'cl-ppcre:regex-apropos)
This is really strange. As an experiment I just created a dirt-simple source file named "test1.lisp" in an isolated directory.
(require :cl-ppcre)
I suspect that this might be the source of your problems. If you have this kind of thing in your lisp code files, I can sort-of understand why you might be getting different answers, because this isn't an idiomatic way of doing things. Normally, you would require modules your system depends on from the defsystem, not from within the source files. (In fact, your example as you state it should not work at all, at least in a clean image, because the file test1.lisp as you describe it should not be compilable, as it contains read-time references to a package which is only available at load-time.
According to the ASDF documentation: http://common-lisp.net/project/asdf/asdf.html#Using-ASDF
4 Using ASDF
4.1 Loading a system
The system foo is loaded (and compiled, if necessary) by evaluating the following Lisp form:
(asdf:load-system :foo)
On some implementations (namely recent versions of ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, LispWorks, MKCL, *SBCL* and XCL), ASDF hooks into the |CL:REQUIRE| facility and you can just use:
(require :foo)
Is this not correct?
Regards, --Jeff
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
According to the ASDF documentation: http://common-lisp.net/project/asdf/asdf.html#Using-ASDF
The system foo is loaded (and compiled, if necessary) by evaluating the following Lisp form: (asdf:load-system :foo) On some implementations (namely recent versions of ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, LispWorks, MKCL, *SBCL* and XCL), ASDF hooks into the |CL:REQUIRE| facility and you can just use: (require :foo) Is this not correct?
It is correct. Evaluating (asdf:load-system :foo) or (require :foo) will (compile and) load the :foo system.
However, when you compile a file containing (require :foo) (describe 'foo::bar) you do not evaluate the `(require :foo)' form; you generate code such that, when you later load the file, (require :foo) will be executed. So when the compiler attempts to read the next form, (describe 'foo::bar), the symbol 'foo::bar does not yet exist, because nothing has yet happened to cause the "FOO" package to be created.
Christophe
On 10/30/2013 01:53 PM, Christophe Rhodes wrote:
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
According to the ASDF documentation: http://common-lisp.net/project/asdf/asdf.html#Using-ASDF
The system foo is loaded (and compiled, if necessary) by evaluating the following Lisp form: (asdf:load-system :foo) On some implementations (namely recent versions of ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, LispWorks, MKCL, *SBCL* and XCL), ASDF hooks into the |CL:REQUIRE| facility and you can just use: (require :foo) Is this not correct?
It is correct. Evaluating (asdf:load-system :foo) or (require :foo) will (compile and) load the :foo system.
However, when you compile a file containing (require :foo) (describe 'foo::bar) you do not evaluate the `(require :foo)' form; you generate code such that, when you later load the file, (require :foo) will be executed. So when the compiler attempts to read the next form, (describe 'foo::bar), the symbol 'foo::bar does not yet exist, because nothing has yet happened to cause the "FOO" package to be created.
I get the same behavior if I compile each line within the file individually with C-c . That should be evaluating the form, right?
CL-USER> ; *compiling (ASDF/OPERATE:LOAD-SYSTEM "cl-ppcre")* ; *compiling (DESCRIBE (QUOTE CL-PPCRE:REGEX-APROPOS))* CL-PPCRE:REGEX-APROPOS [symbol]
REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. Source file: /home/jcunningham/slime/test1.lisp
Regards, --Jeff
Christophe Rhodes csr21@cantab.net writes:
However, when you compile a file containing (require :foo) (describe 'foo::bar) you do not evaluate the `(require :foo)' form; you generate code such that, when you later load the file, (require :foo) will be executed. So when the compiler attempts to read the next form, (describe 'foo::bar), the symbol 'foo::bar does not yet exist, because nothing has yet happened to cause the "FOO" package to be created.
Unless you are using CLISP, which has the "interesting" non-ANSI behavior of special treatment of the REQUIRE function during compilation.
http://www.clisp.org/impnotes.html#lib-files
That's not coming into play here, just wanted to note it in case someone encounters it in the future.
Zach
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
On 10/30/2013 12:21 PM, Jeff Cunningham wrote:
(describe 'cl-ppcre:regex-apropos)
This is really strange. As an experiment I just created a dirt-simple source file named "test1.lisp" in an isolated directory.
(require :cl-ppcre) (use-package :cl-ppcre) (describe 'cl-ppcre:regex-apropos)
I think I can reproduce the problem.
Given:
/tmp/bar.asd containing: (asdf:defsystem bar :contents ((:file "baz")))
/tmp/baz.lisp containing: (defun frob (x) (1+ x))
/tmp/foo.lisp containing: (require :bar)
and neither /tmp/foo.fasl nor /tmp/bar.fasl existing, then executing from the toplevel (load "/tmp/bar.asd") and subsequently hitting C-c C-c on the (require :bar) form in foo.lisp will result in (describe 'frob) thinking that it has a source location of foo.lisp, rather than baz.lisp.
As I have said in other mails, this is not the normal way of using the system. Using REQUIRE in source files is unusual; so too is not having the depended-on systems already compiled, which perhaps explains why it's taken this long just to understand your problem. (In general triggering a file compilation from within LOAD or COMPILE-FILE is hazardous; it doesn't completely surprise me that not everything is actually totally correct). You would make your life easier by writing your own defsystem forms which declare the relevant dependencies on other systems.
Christophe
On Wed, Oct 30 2013, Christophe Rhodes wrote:
I think I can reproduce the problem.
Given:
/tmp/bar.asd containing: (asdf:defsystem bar :contents ((:file "baz")))
/tmp/baz.lisp containing: (defun frob (x) (1+ x))
/tmp/foo.lisp containing: (require :bar)
and neither /tmp/foo.fasl nor /tmp/bar.fasl existing, then executing from the toplevel (load "/tmp/bar.asd") and subsequently hitting C-c C-c on the (require :bar) form in foo.lisp will result in (describe 'frob) thinking that it has a source location of foo.lisp, rather than baz.lisp.
Indeed. I can reproduce that too. If we do it without ASDF we can put (load (compile-file "baz.lisp")) in foo.lisp and press C-c C-c on it.
This happens because SWANK-COMPILE-STRING in swank-sbcl.lisp uses with WITH-COMPILATION-UNIT to pass :source-namestring to nested invocations of COMPILE-FILE. Since LOAD is also called inside that WITH-COMPILATION-UNIT the invocation in foo.lisp picks up the wrong value for :source-namestring.
I moved LOAD outside and this problem should no longer occur. A potential downside is that (redefinition) warnings that are emitted during LOAD are no longer picked up by SLIME.
Helmut
On 10/31/2013 01:07 AM, Helmut Eller wrote:
On Wed, Oct 30 2013, Christophe Rhodes wrote:
I think I can reproduce the problem.
Given:
/tmp/bar.asd containing: (asdf:defsystem bar :contents ((:file "baz")))
/tmp/baz.lisp containing: (defun frob (x) (1+ x))
/tmp/foo.lisp containing: (require :bar)
and neither /tmp/foo.fasl nor /tmp/bar.fasl existing, then executing from the toplevel (load "/tmp/bar.asd") and subsequently hitting C-c C-c on the (require :bar) form in foo.lisp will result in (describe 'frob) thinking that it has a source location of foo.lisp, rather than baz.lisp.
Indeed. I can reproduce that too. If we do it without ASDF we can put (load (compile-file "baz.lisp")) in foo.lisp and press C-c C-c on it.
This happens because SWANK-COMPILE-STRING in swank-sbcl.lisp uses with WITH-COMPILATION-UNIT to pass :source-namestring to nested invocations of COMPILE-FILE. Since LOAD is also called inside that WITH-COMPILATION-UNIT the invocation in foo.lisp picks up the wrong value for :source-namestring.
I moved LOAD outside and this problem should no longer occur. A potential downside is that (redefinition) warnings that are emitted during LOAD are no longer picked up by SLIME.
Helmut
I see you made changes in CVS. Does quicklisp pick these up if I update-dist? Or do I need to reinstall Slime from CVS to get this?
I was able to isolate the problem to this very simple example. I''m running SBCL 1.1.11 and Slime 2013-09-29.
First, I deleted ~/.cache/common-lisp/*
Then I created a new project:
(ql:quickload "quickproject") (quickproject:make-project "/home/jcunningham/myproj/" :depends-on '(:cl-ppcre))
I loaded the project by creating /tmp/test.lisp with the following line and compiling it:
(asdf:load-system "myproj")
And executed the following:
(describe 'cl-ppcre:regex-apropos)
; compiling (DESCRIBE (QUOTE CL-PPCRE:REGEX-APROPOS)) CL-PPCRE:REGEX-APROPOS [symbol]
REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. *Source file: /tmp/test.lisp*
--Jeff
Jeff Cunningham jeffrey@jkcunningham.com writes:
I see you made changes in CVS. Does quicklisp pick these up if I update-dist? Or do I need to reinstall Slime from CVS to get this?
Quicklisp will pick the change up automatically, but you will not see the new code until I make a new dist release, which I hope to do on Saturday or Sunday.
I generally like to make a new release on the first weekend of each month.
Zach
On Thu, Oct 31 2013, Jeff Cunningham wrote:
I see you made changes in CVS. Does quicklisp pick these up if I update-dist? Or do I need to reinstall Slime from CVS to get this?
I don't know exactly how/when quicklisp the code in is updated. Installing from CVS is the safe bet.
Thinking about it, if you add (setq swank-backend::*trap-load-time-warnings* nil) in ~/.swank.lisp should also get the behaviour that LOAD is called outside of WITH-COMPILATION-UNIT even without updating SLIME.
You could also C-x C-e instead of C-c C-c when loading code to see if the WITH-COMPILATION-UNIT is the culprit.
Helmut
On 10/31/2013 06:50 AM, Helmut Eller wrote:
On Thu, Oct 31 2013, Jeff Cunningham wrote:
I see you made changes in CVS. Does quicklisp pick these up if I update-dist? Or do I need to reinstall Slime from CVS to get this?
I don't know exactly how/when quicklisp the code in is updated. Installing from CVS is the safe bet.
Thinking about it, if you add (setq swank-backend::*trap-load-time-warnings* nil) in ~/.swank.lisp should also get the behaviour that LOAD is called outside of WITH-COMPILATION-UNIT even without updating SLIME.
You could also C-x C-e instead of C-c C-c when loading code to see if the WITH-COMPILATION-UNIT is the culprit.
Helmut
I just updated Slime from CVS and it works now. Much obliged.
Jeff
If you *really* want to compile self-contained file that requires something then uses it, The solution is to use (1) eval-when and (2) funcall 'require, to not get optimized away by clisp.
(eval-when (:compile-toplevel :load-toplevel :execute) (funcall 'require :cl-ppcre)) ;; in this case, asdf:load-system is cleaner than require. (cl-ppcre:...)
That said, the above is OK for a one-off experiment, but not at all recommended for libraries or even for sets of experiments.
For publishable libraries, I recommend to write a .asd file for your system.
For sets of experiments, I recommend you use the asdf-package-system (itself inspired by quick-build, and soon to be released as part of asdf 3.1.1), whereby you have a single .asd file for all your experiments (say my-experiments.asd), and then you can (asdf:load-system :my-experiments/foo/bar) and it will read foo/bar.lisp under the directory of my-experiments.asd, deduce its dependencies from the defpackage, and compile and load everything. Adding a new system to the hierarchy is simply a matter of creating a file with a defpackage, and only files needed are compiled and loaded.
(Sorry for a late reply, I am not actively following this mailing-list)
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "The direct use of force is such a poor solution to any problem, it is generally employed only by small children and large nations." – David Friedman
On 10/30/2013 02:22 PM, Christophe Rhodes wrote:
As I have said in other mails, this is not the normal way of using the system. Using REQUIRE in source files is unusual; so too is not having the depended-on systems already compiled, which perhaps explains why it's taken this long just to understand your problem. (In general triggering a file compilation from within LOAD or COMPILE-FILE is hazardous; it doesn't completely surprise me that not everything is actually totally correct). You would make your life easier by writing your own defsystem forms which declare the relevant dependencies on other systems. Christophe
Okay, I tried the following in a new source file test2.lisp with a fresh emacs and slime environment.
(asdf:defsystem #:test2 :depends-on (:cl-ppcre))) (asdf:load-system "test2") (describe 'cl-ppcre:regex-apropos)
I compiled each of these lines with C-c C-c. The behavior is the same as before:
Source file: /home/jcunningham/slime/test1.lisp
If I am not implementing this in the "normal" way of using the system, perhaps you could give me some guidance here. What is the minimal code I need to write in a file so I can load, say, cl-ppcre, write a line of code that calls one of its functions, and allows me to navigate to it by Alt-. ?
I thought I was doing it the normal way.
--Jeff
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
On 10/30/2013 02:22 PM, Christophe Rhodes wrote:
As I have said in other mails, this is not the normal way of using the system. Using REQUIRE in source files is unusual; so too is not having the depended-on systems already compiled, which perhaps explains why it's taken this long just to understand your problem. (In general triggering a file compilation from within LOAD or COMPILE-FILE is hazardous; it doesn't completely surprise me that not everything is actually totally correct). You would make your life easier by writing your own defsystem forms which declare the relevant dependencies on other systems. Christophe
Okay, I tried the following in a new source file test2.lisp with a fresh emacs and slime environment.
(asdf:defsystem #:test2 :depends-on (:cl-ppcre))) (asdf:load-system "test2") (describe 'cl-ppcre:regex-apropos)
I compiled each of these lines with C-c C-c. The behavior is the same as before:
Source file: /home/jcunningham/slime/test1.lisp
If I am not implementing this in the "normal" way of using the system, perhaps you could give me some guidance here. What is the minimal code I need to write in a file so I can load, say, cl-ppcre, write a line of code that calls one of its functions, and allows me to navigate to it by Alt-. ?
I thought I was doing it the normal way.
That code should go in test2.asd:
;;;; test2.asd
(asdf:defsystem #:test2 :depends-on (#:cl-ppcre) :serial t :components ((:file "package") (:file "test2")))
Then you have package.lisp:
(defpackage #:test2 (:use #:cl) (:export #:my-great-test))
Then you can put this in test2.lisp:
;;;; test2.lisp
(in-package #:test2)
(defun my-great-test () (describe 'cl-ppcre:regex-apropos))
Put those files somewhere ASDF knows about. I like to put it in something like ~/quicklisp/local-projects/test2/, so you can, from the repl, use this:
(ql:quickload "test2")
Then you should be able to evaluate, in the REPL:
(test2:my-great-test)
Many people automate the process of setting up project directories, system files, and initial sources. My automation of it is the quickproject utility; there are several others.
I don't recommend putting calls to code-loading functions directly in source files, most of the time.
Zach
On 10/30/2013 04:06 PM, Zach Beane wrote:
That code should go in test2.asd:
;;;; test2.asd (asdf:defsystem #:test2 :depends-on (#:cl-ppcre) :serial t :components ((:file "package") (:file "test2")))Then you have package.lisp:
(defpackage #:test2 (:use #:cl) (:export #:my-great-test))Then you can put this in test2.lisp:
;;;; test2.lisp (in-package #:test2) (defun my-great-test () (describe 'cl-ppcre:regex-apropos))Put those files somewhere ASDF knows about. I like to put it in something like ~/quicklisp/local-projects/test2/, so you can, from the repl, use this:
(ql:quickload "test2")Then you should be able to evaluate, in the REPL:
(test2:my-great-test)Many people automate the process of setting up project directories, system files, and initial sources. My automation of it is the quickproject utility; there are several others.
I don't recommend putting calls to code-loading functions directly in source files, most of the time.
Zach
Thank you, Zach.
But what about the very common (for me, at least) case where I want to experiment around with some code that is not intended to go in a package. It's experimental code - say I have some data I'm want to read and process. I'm in the data directory when I start lisp. Its not in an ASDF path and don't really want to modify that search tree everytime I write a one-off piece of code.
I can quickload cl-fad, say, cl-ppcre, and all kinds of packages and write code in the REPL and it works fine. But I don't want to have to retype all this code everytime I revisit the data. I want to be able to type it in a local file, say, test.lisp" and C-c C-c on the lines again tomorrow if I want to run them again. Or maybe C-c, k the whole file and run it that way. So in this context how can I load packages as lines in this file without losing the ability to navigate within their functions? Are you saying that I either have to enter each package load and subsequent commands in the REPL or build a formal package with an .asd and package file, and tell ASD about it?
I've been writing code like this for years - it's only recently that it started behaving differently.
Regards, --Jeff
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
But what about the very common (for me, at least) case where I want to experiment around with some code that is not intended to go in a package.
One option is to learn a different way. I start almost every experiment with the three-file system of foo.asd, package.lisp, and foo.lisp, which makes it and its dependencies trivially quickloadable. And the experiments often grow into something I want to save for easy reuse.
Zach
On 10/30/2013 04:52 PM, Zach Beane wrote:
Jeffrey Cunningham jeffrey@jkcunningham.com writes:
But what about the very common (for me, at least) case where I want to experiment around with some code that is not intended to go in a package.
One option is to learn a different way. I start almost every experiment with the three-file system of foo.asd, package.lisp, and foo.lisp, which makes it and its dependencies trivially quickloadable. And the experiments often grow into something I want to save for easy reuse.
I'm fine with learning a different way. Actually, I experimented with your quickproject utility when you first released it. I like the idea of it. But the single significant drawback for me is that it seems to require project files to be located to suit the needs of the tool rather than the needs of (for me) the work. If I'm contracting for companyX their files all go somewhere within a companyX folder. Their data goes there. Their docs. Everything. I can tarball the whole thing up and send it to them. I can archive it. I can send it to Richard Snowden. Whatever. It seems to me that the requirements of the package methodology you are suggesting requires that I either do my contract work in subdirectories of the quicklisp folder of some other standard ASD visible folder, or be continually modifying the ASD paths so the loader knows where to find them. Maybe I haven't just wrapped my head around it right, but it seems odd to have to build a package just to load some other packages and use them.
I agree that many, many times what started out experimental developed into a package I've reused for other projects. I think if I could figure out how to keep files organized by job rather than by tool I'd be all for your idea.
--Jeff
The quicklisp/local-projects/ directory is just one convenient place to put stuff, nothing dictates that you have to put it there.
You can also put your own .asd systems elsewhere and set up a call to:
(pushnew ".../my-projects/" ql:*local-project-directories* :test #'string-equal) (ql:register-local-projects)
Yes, you have to evaluate these two lines after making a new .asd file. But if you put all your projects under ".../my-projects/" then nothing else has to change so it can be part of a standard thingie you just call after setting up a new experimental project.
On Wed, Oct 30, 2013 at 8:26 PM, Jeffrey Cunningham < jeffrey@jkcunningham.com> wrote:
On 10/30/2013 04:52 PM, Zach Beane wrote:
Jeffrey Cunningham jeffrey@jkcunningham.com jeffrey@jkcunningham.com writes:
But what about the very common (for me, at least) case where I want to experiment around with some code that is not intended to go in a package.
One option is to learn a different way. I start almost every experiment with the three-file system of foo.asd, package.lisp, and foo.lisp, which makes it and its dependencies trivially quickloadable. And the experiments often grow into something I want to save for easy reuse.
I'm fine with learning a different way. Actually, I experimented with your quickproject utility when you first released it. I like the idea of it. But the single significant drawback for me is that it seems to require project files to be located to suit the needs of the tool rather than the needs of (for me) the work. If I'm contracting for companyX their files all go somewhere within a companyX folder. Their data goes there. Their docs. Everything. I can tarball the whole thing up and send it to them. I can archive it. I can send it to Richard Snowden. Whatever. It seems to me that the requirements of the package methodology you are suggesting requires that I either do my contract work in subdirectories of the quicklisp folder of some other standard ASD visible folder, or be continually modifying the ASD paths so the loader knows where to find them. Maybe I haven't just wrapped my head around it right, but it seems odd to have to build a package just to load some other packages and use them.
I agree that many, many times what started out experimental developed into a package I've reused for other projects. I think if I could figure out how to keep files organized by job rather than by tool I'd be all for your idea.
--Jeff
On 10/30/2013 06:13 PM, Dave Cooper wrote:
The quicklisp/local-projects/ directory is just one convenient place to put stuff, nothing dictates that you have to put it there.
You can also put your own .asd systems elsewhere and set up a call to:
(pushnew ".../my-projects/" ql:*local-project-directories* :test #'string-equal) (ql:register-local-projects)
Yes, you have to evaluate these two lines after making a new .asd file. But if you put all your projects under ".../my-projects/" then nothing else has to change so it can be part of a standard thingie you just call after setting up a new experimental project.
Thanks, Dave. I've been reading Zach's documentation and see that now. Misconception on my part.
I just rebuilt a project following suggestions by Zach and yourself. Gave it a new name. It loads via ql:quickload without complaint. And it runs as before. Here's the way I am invoking the project:
(pushnew "/home/jcunningham/quest/jkc/ftisv44/" ql:*local-project-directories* :test #'equalp) (ql:register-local-projects) (ql:quickload "ftisv44")
It loads just fine. The project asd file:
;;;; ftisv44.asd
(asdf:defsystem #:ftisv44 :serial t :description "Integer model for vhdl verification and testing" :author "J.K.Cunningham" :depends-on (#:cl-ppcre #:cl-fad #:cl-gd-ext #:cl-who #:html5 #:json-pt #:graham #:cl-extra) :components ((:file "package") (:file "common") (:file "vectors") (:file "waveforms") (:file "interface") (:file "model") (:file "post-processing") (:file "octave") (:file "vhdl") (:file "verify") (:file "make-html-pages")))
The package file:
(defpackage #:ftisv44 (:use :cl :cl-extra :cl-ppcre :cl-fad :graham :cl-gd-ext :json-pt :html5 :cl-who) (:shadowing-import-from :cl-who :str :htm :fmt) (:export ;; from common.lisp #:delete-files #:clear-errors #:print-errors #:report-error #:make-vec #:subvector #:inner-product #:complexp* #:inv* #:realpart* #:imagpart* #:conjugate* #:abs* #:cos* #:sin** #:cexp* #:truncate* #:floor* #:round* #:magsqr #:scale #:complex* #:add #:sub #:mul #:div #:sum #:sumsqr #:mean #:var* #:max* #:maxabs #:enough-bits #:clip #:symclip #:to-fixeddpoint #:diff #:set-vectors-enable #:add-to-vfiles #:get-vfiles #:clear-vfiles #:write-vector #:read-vec #:read-dat #:write-integers #:make-zip #:make-tarball #:gen-vec #:compute-stats #:blank #:centersym #:autocor #:fft #:fftd ;; #:clear-accums #:get-accums #:accumulate-unique #:print-accums ;; #:clear-dranges #:get-dranges #:bitwidth #:accumulate-drange #:print-dranges ;; cordic #:cordic-sqrt #:cordic-log #:cordic-log-fp ;; #:fir-filter #:find-peaks #:interpolate ;; #:set-plots-enable #:error_plots-enabled-p #:data_plots-enabled-p #:gd-plot #:gd-plot-2
;; from vectors.lisp #:set-verify #:verify-p #:get-verify-src #:get-verify-dir ;; #:with-vector #:has-vector #:svectors-remaining #:next-svector-length #:clear-vector-table #:write-vector-status #:add-vector #:next-vector-length #:vectors-remaining #:lookat-vector #:read-vector #:subvec #:load-vectors #:with-test-vector #:while-test-vector ;; #:translate-fifo #:print-fifo-errorbits ;; #:compare-and-inject ;; #:clear-test-results #:print-test-results #:plot-vector #:plot-overlay-vectors #:compare-vectors #:compare-values #:make-unsigned #:compare-peaks
;; from waveforms.lisp #:*waveforms-dir* #:waveform-setname #:waveform-file #:waveform-iw #:waveform-hb #:waveform-awall #:waveform-state #:waveform-notes #:make-waveform #:waveform-vdir #:get-waveforms #:with-waveform-sets #:generate-ilaunch2
;; from octave.lisp #:max-wall #:octave-model
;; from verify.lisp #:gen-vhdl-vectors #:gen-1ping-per-setname #:gen-vhdl-mping-vectors #:gen-mping-per-setname #:run-verification #:compare-model-octave #:compare-model-octave-all #:unpack-new-tarball #:process-vhdl-tarball #:process-all-vhdl-tarballs
;; from make-html-pages #:make-html-data-site ))
Then at the top of a (non-project source file located in a data directory)
(in-package :ftisv44)
After all that, when I put the point on, say, the function ftisv44:run-verification and press Alt-. I still get
* Error: end of file on #<SB-IMPL::STRING-INPUT-STREAM {1005A81C63}>** *
Regards, Jeff
-
Christophe Rhodes -
Dave Cooper -
Faré -
Helmut Eller -
Jeff Cunningham -
Jeffrey Cunningham
-
Zach Beane