On 10/27/2013 12:06 AM, Helmut Eller wrote:

On Sat, Oct 26 2013, Jeff Cunningham wrote:

[...]

...

Is there any remedy for this? Using grep to move around works but is annoyingly cumbersome and time consuming. I don't see any obvious problem with the bits you posted and without a reproducable test case it's hard to say what the actual problem is.

One problem could be that you have some declaration/proclamation somewhere, e.g. (proclaim '(optimize (debug 0))). If you're using SBCL and debug is 0 then SLIME can't find the definition for a name. I have (proclaim '(optimize (debug 2))) in my .sbclrc and that seems to work quite OK.

Another cause could be that you load the source and not the fasl file. M-. does typically not work so well for non-compiled functions.

Yet another problem could be that there is no IN-PACKAGE form in the file so that SLIME searches in the wrong package. (That's unlikey, tho, as compiling an individual function seems to work.)

In general, M-. seems to be used by many people therefore I think we would note regressions on our side pretty quickly.

Helmut Thanks for the response, Helmut. I was hoping there was something I was missing.

I don't normally change the optimization in development code. But I went ahead and tried the proclaim '(optimize (ebug 2))) in my .sblcrc as you suggested, but that made no difference. All my source files have (in-package 'ftis-v43), the package defined in the package.lisp file. I'm guessing it has something to do with not loading local fasl files, because if I open each of the lisp sources and compile fasls for them with Ctrl-c, k I can then navigate around with Alt-. But I still have to compile the files manually. Simply loading via ASD leaves it in a state where it can't navigate. On the other hand, i can put the point on a function from an inluded package, say something like cl-ppcre:scan and it will go off and find that just fine. There's something I'm missing. --Jeff

On 10/29/2013 09:36 AM, Christophe Rhodes wrote:

Jeff Cunningham <jeffrey@jkcunningham.com> writes:

...

I don't normally change the optimization in development code. But I went ahead and tried the proclaim '(optimize (ebug 2))) in my .sblcrc as you suggested, but that made no difference.

All my source files have (in-package 'ftis-v43), the package defined in the package.lisp file. (in-package 'ftis-v43) is not actually legal common lisp. (The argument to IN-PACKAGE is an unevaluated name). Do things start working if you use (in-package "FTIS-V43") or (in-package :ftis-v43)? I know this might be a red herring, since (a) you're talking about fasls and (b) the source finding works for individually-compiled buffers, but it might be worth knowing in any case.

That was a typo - all of them are (in-package :ftis-v43), I just checked. Currently, if I start a fresh sbcl under slime the only functions I can navigate to with Alt-. are system functions (format, for example). Any packages I am loading via quicklisp or of my own result in: Error: end of file on #<SB-IMPL::STRING-INPUT-STREAM {10070EBBB3}> When I load packages creates and/or loads a fasl under ~/.cache/common-lisp/<sbcl version>/.. I would have thought that would be how it would navigate, but in my case not. I can open any source file, compile a LOCAL fasl with C-c k and then those functions can be navigated into. It used to work on anything. And I haven't changed my .sbclrc in years (except to add the suggestion yesterday). Here it is: ;; -*-Lisp-*- (require 'asdf) (proclaim '(optimize (debug 2))) ;;; The following lines added by ql:add-to-init-file: #-quicklisp (let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp" (user-homedir-pathname)))) (when (probe-file quicklisp-init) (load quicklisp-init))) ;;; If a fasl was stale, try to recompile and load (once). (defmethod asdf:perform :around ((o asdf:load-op) (c asdf:cl-source-file)) (handler-case (call-next-method o c) ;; If a fasl was stale, try to recompile and load (once). (sb-ext:invalid-fasl () (asdf:perform (make-instance 'asdf:compile-op) c) (call-next-method)))) Thanks, --Jeff

On 10/30/2013 01:24 AM, Christophe Rhodes wrote:

Jeff Cunningham <jeffrey@jkcunningham.com> writes:

...

That was a typo - all of them are (in-package :ftis-v43), I just checked. Currently, if I start a fresh sbcl under slime the only functions I can navigate to with Alt-. are system functions (format, for example). Any packages I am loading via quicklisp or of my own result in:

Error: end of file on #<SB-IMPL::STRING-INPUT-STREAM {10070EBBB3}>

When I load packages creates and/or loads a fasl under ~/.cache/common-lisp/<sbcl version>/.. I would have thought that would be how it would navigate, but in my case not.

I can open any source file, compile a LOCAL fasl with C-c k and then those functions can be navigated into. It used to work on anything. And I haven't changed my .sbclrc in years (except to add the suggestion yesterday). Here it is: Well, I just did a quick test and I don't see this problem, though inevitably my setup does not mirror yours terribly well.

On the assumption that you have cl-ppcre relatively easily to hand, what does (describe 'cl-ppcre:regex-apropos) say after you have loaded cl-ppcre through quicklisp? (If you don't have cl-ppcre, substitute your favourite simple "utility")

Cheers,

Christophe Here's what it says:

FTIS-V43> (describe 'cl-ppcre:regex-apropos) CL-PPCRE:REGEX-APROPOS [symbol] REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. Source file: /home/jcunningham/qigj/test-model.lisp ; No value Thanks, Jeff

New test: I updated all dists with quicklisp which reinstalled cl-ppcre and slime. I started a fresh emacs and Slime (9/2013) and ran the following test1.lisp (using asdf:load-system instead of require). ;; (require "cl-ppcre") (asdf:load-system "cl-ppcre") (describe 'cl-ppcre:regex-apropos) After compiling the fresh cl-ppcre code it ran and gave me the following: ; compiling file "/home/jcunningham/slime/test1.lisp" (written 30 OCT 2013 01:10:24 PM): ; /home/jcunningham/slime/test1.fasl written ; compilation finished in 0:00:00.003 CL-PPCRE:REGEX-APROPOS [symbol] REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. *Source file: /home/jcunningham/slime/test1.lisp** * Now it's switched its "source" for cl-ppcre to the test file. --Jeff

On 10/30/2013 12:21 PM, Jeff Cunningham wrote:

(describe 'cl-ppcre:regex-apropos)

This is really strange. As an experiment I just created a dirt-simple source file named "test1.lisp" in an isolated directory. (require :cl-ppcre) (use-package :cl-ppcre) (describe 'cl-ppcre:regex-apropos) When I compile and run this I get the following: ; compiling file "/home/jcunningham/slime/test1.lisp" (written 30 OCT 2013 12:50:50 PM): ; /home/jcunningham/slime/test1.fasl written ; compilation finished in 0:00:00.002 CL-PPCRE:REGEX-APROPOS [symbol] REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. *Source file: /home/jcunningham/quest/robom/plotwave.lisp* This is a fresh slime session started in a new emacs. The file it is pointing to is something I haven't been working with for more than a week and most definitely does NOT contain regex-apropos by any package name. But it does contain a defpackage, which looks like this: (require 'cl-fad) (require 'cl-extra) (require 'cl-ppcre) (require 'quaternion) (require 'graham)) (defpackage #:cnc-user (:use :cl :quaternion :cl-extra :graham :cl-fad :cl-ppcre)) (in-package :cnc-user) It would appear that somehow when I last compiled this piece of code it modified something persistently for slime so that it now thinks - from any session I start - that functions in these packages have source code located in this file. The functions are found by SBCL because they do work. It is Slime that is going wrong here. I see that I was using the 'cl-ppcre designation for the package that you say is not legal Common Lisp. I have no idea how that would affect things. I will change this in the future, but I am at a loss to see how to get back to the behavior Alt-. once had. I am running SBCL 1.1.11 on this machine, and SLIME 2013-06-26 Regards, --Jeff

On 10/30/2013 01:27 PM, Christophe Rhodes wrote:

Jeffrey Cunningham <jeffrey@jkcunningham.com> writes:

...

...

(describe 'cl-ppcre:regex-apropos) This is really strange. As an experiment I just created a dirt-simple

On 10/30/2013 12:21 PM, Jeff Cunningham wrote: source file named "test1.lisp" in an isolated directory.

(require :cl-ppcre) I suspect that this might be the source of your problems. If you have this kind of thing in your lisp code files, I can sort-of understand why you might be getting different answers, because this isn't an idiomatic way of doing things. Normally, you would require modules your system depends on from the defsystem, not from within the source files. (In fact, your example as you state it should not work at all, at least in a clean image, because the file test1.lisp as you describe it should not be compilable, as it contains read-time references to a package which is only available at load-time.

According to the ASDF documentation: http://common-lisp.net/project/asdf/asdf.html#Using-ASDF 4 Using ASDF 4.1 Loading a system The system foo is loaded (and compiled, if necessary) by evaluating the following Lisp form: (asdf:load-system :foo) On some implementations (namely recent versions of ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, LispWorks, MKCL, *SBCL* and XCL), ASDF hooks into the |CL:REQUIRE| facility and you can just use: (require :foo) Is this not correct? Regards, --Jeff

On 10/30/2013 01:53 PM, Christophe Rhodes wrote:

Jeffrey Cunningham <jeffrey@jkcunningham.com> writes:

...

According to the ASDF documentation: http://common-lisp.net/project/asdf/asdf.html#Using-ASDF

The system foo is loaded (and compiled, if necessary) by evaluating the following Lisp form: (asdf:load-system :foo) On some implementations (namely recent versions of ABCL, Allegro CL, Clozure CL, CMUCL, ECL, GNU CLISP, LispWorks, MKCL, *SBCL* and XCL), ASDF hooks into the |CL:REQUIRE| facility and you can just use: (require :foo) Is this not correct? It is correct. Evaluating (asdf:load-system :foo) or (require :foo) will (compile and) load the :foo system.

However, when you compile a file containing (require :foo) (describe 'foo::bar) you do not evaluate the `(require :foo)' form; you generate code such that, when you later load the file, (require :foo) will be executed. So when the compiler attempts to read the next form, (describe 'foo::bar), the symbol 'foo::bar does not yet exist, because nothing has yet happened to cause the "FOO" package to be created.

I get the same behavior if I compile each line within the file individually with C-c . That should be evaluating the form, right? CL-USER> ; *compiling (ASDF/OPERATE:LOAD-SYSTEM "cl-ppcre")* ; *compiling (DESCRIBE (QUOTE CL-PPCRE:REGEX-APROPOS))* CL-PPCRE:REGEX-APROPOS [symbol] REGEX-APROPOS names a compiled function: Lambda-list: (REGEX &OPTIONAL PACKAGES &KEY (CASE-INSENSITIVE T)) Derived type: (FUNCTION (T &OPTIONAL T &KEY (:CASE-INSENSITIVE T)) (VALUES &OPTIONAL)) Documentation: Similar to the standard function APROPOS but returns a list of all symbols which match the regular expression REGEX. If CASE-INSENSITIVE is true and REGEX isn't already a scanner, a case-insensitive scanner is used. Source file: /home/jcunningham/slime/test1.lisp Regards, --Jeff

Jeffrey Cunningham <jeffrey@jkcunningham.com> writes:

On 10/30/2013 12:21 PM, Jeff Cunningham wrote:

...

(describe 'cl-ppcre:regex-apropos)

This is really strange. As an experiment I just created a dirt-simple source file named "test1.lisp" in an isolated directory.

(require :cl-ppcre) (use-package :cl-ppcre) (describe 'cl-ppcre:regex-apropos)

I think I can reproduce the problem. Given: /tmp/bar.asd containing: (asdf:defsystem bar :contents ((:file "baz"))) /tmp/baz.lisp containing: (defun frob (x) (1+ x)) /tmp/foo.lisp containing: (require :bar) and neither /tmp/foo.fasl nor /tmp/bar.fasl existing, then executing from the toplevel (load "/tmp/bar.asd") and subsequently hitting C-c C-c on the (require :bar) form in foo.lisp will result in (describe 'frob) thinking that it has a source location of foo.lisp, rather than baz.lisp. As I have said in other mails, this is not the normal way of using the system. Using REQUIRE in source files is unusual; so too is not having the depended-on systems already compiled, which perhaps explains why it's taken this long just to understand your problem. (In general triggering a file compilation from within LOAD or COMPILE-FILE is hazardous; it doesn't completely surprise me that not everything is actually totally correct). You would make your life easier by writing your own defsystem forms which declare the relevant dependencies on other systems. Christophe

If you *really* want to compile self-contained file that requires something then uses it, The solution is to use (1) eval-when and (2) funcall 'require, to not get optimized away by clisp. (eval-when (:compile-toplevel :load-toplevel :execute) (funcall 'require :cl-ppcre)) ;; in this case, asdf:load-system is cleaner than require. (cl-ppcre:...) That said, the above is OK for a one-off experiment, but not at all recommended for libraries or even for sets of experiments. For publishable libraries, I recommend to write a .asd file for your system. For sets of experiments, I recommend you use the asdf-package-system (itself inspired by quick-build, and soon to be released as part of asdf 3.1.1), whereby you have a single .asd file for all your experiments (say my-experiments.asd), and then you can (asdf:load-system :my-experiments/foo/bar) and it will read foo/bar.lisp under the directory of my-experiments.asd, deduce its dependencies from the defpackage, and compile and load everything. Adding a new system to the hierarchy is simply a matter of creating a file with a defpackage, and only files needed are compiled and loaded. (Sorry for a late reply, I am not actively following this mailing-list) —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "The direct use of force is such a poor solution to any problem, it is generally employed only by small children and large nations." – David Friedman

Jeff Cunningham <jeffrey@jkcunningham.com> writes:

I see you made changes in CVS. Does quicklisp pick these up if I update-dist? Or do I need to reinstall Slime from CVS to get this?

Quicklisp will pick the change up automatically, but you will not see the new code until I make a new dist release, which I hope to do on Saturday or Sunday. I generally like to make a new release on the first weekend of each month. Zach

On 10/31/2013 06:50 AM, Helmut Eller wrote:

On Thu, Oct 31 2013, Jeff Cunningham wrote:

...

I see you made changes in CVS. Does quicklisp pick these up if I update-dist? Or do I need to reinstall Slime from CVS to get this? I don't know exactly how/when quicklisp the code in is updated. Installing from CVS is the safe bet.

Thinking about it, if you add (setq swank-backend::*trap-load-time-warnings* nil) in ~/.swank.lisp should also get the behaviour that LOAD is called outside of WITH-COMPILATION-UNIT even without updating SLIME.

You could also C-x C-e instead of C-c C-c when loading code to see if the WITH-COMPILATION-UNIT is the culprit.

Helmut I just updated Slime from CVS and it works now. Much obliged.

Jeff

Jeffrey Cunningham <jeffrey@jkcunningham.com> writes:

On 10/30/2013 02:22 PM, Christophe Rhodes wrote:

...

As I have said in other mails, this is not the normal way of using the system. Using REQUIRE in source files is unusual; so too is not having the depended-on systems already compiled, which perhaps explains why it's taken this long just to understand your problem. (In general triggering a file compilation from within LOAD or COMPILE-FILE is hazardous; it doesn't completely surprise me that not everything is actually totally correct). You would make your life easier by writing your own defsystem forms which declare the relevant dependencies on other systems. Christophe

Okay, I tried the following in a new source file test2.lisp with a fresh emacs and slime environment.

(asdf:defsystem #:test2 :depends-on (:cl-ppcre))) (asdf:load-system "test2") (describe 'cl-ppcre:regex-apropos)

I compiled each of these lines with C-c C-c. The behavior is the same as before:

Source file: /home/jcunningham/slime/test1.lisp

If I am not implementing this in the "normal" way of using the system, perhaps you could give me some guidance here. What is the minimal code I need to write in a file so I can load, say, cl-ppcre, write a line of code that calls one of its functions, and allows me to navigate to it by Alt-. ?

I thought I was doing it the normal way.

That code should go in test2.asd: ;;;; test2.asd (asdf:defsystem #:test2 :depends-on (#:cl-ppcre) :serial t :components ((:file "package") (:file "test2"))) Then you have package.lisp: (defpackage #:test2 (:use #:cl) (:export #:my-great-test)) Then you can put this in test2.lisp: ;;;; test2.lisp (in-package #:test2) (defun my-great-test () (describe 'cl-ppcre:regex-apropos)) Put those files somewhere ASDF knows about. I like to put it in something like ~/quicklisp/local-projects/test2/, so you can, from the repl, use this: (ql:quickload "test2") Then you should be able to evaluate, in the REPL: (test2:my-great-test) Many people automate the process of setting up project directories, system files, and initial sources. My automation of it is the quickproject utility; there are several others. I don't recommend putting calls to code-loading functions directly in source files, most of the time. Zach

Jeffrey Cunningham <jeffrey@jkcunningham.com> writes:

But what about the very common (for me, at least) case where I want to experiment around with some code that is not intended to go in a package.

One option is to learn a different way. I start almost every experiment with the three-file system of foo.asd, package.lisp, and foo.lisp, which makes it and its dependencies trivially quickloadable. And the experiments often grow into something I want to save for easy reuse. Zach

The quicklisp/local-projects/ directory is just one convenient place to put stuff, nothing dictates that you have to put it there. You can also put your own .asd systems elsewhere and set up a call to: (pushnew ".../my-projects/" ql:*local-project-directories* :test #'string-equal) (ql:register-local-projects) Yes, you have to evaluate these two lines after making a new .asd file. But if you put all your projects under ".../my-projects/" then nothing else has to change so it can be part of a standard thingie you just call after setting up a new experimental project. On Wed, Oct 30, 2013 at 8:26 PM, Jeffrey Cunningham < jeffrey@jkcunningham.com> wrote:

On 10/30/2013 04:52 PM, Zach Beane wrote:

Jeffrey Cunningham <jeffrey@jkcunningham.com> <jeffrey@jkcunningham.com> writes:

But what about the very common (for me, at least) case where I want to experiment around with some code that is not intended to go in a package.

One option is to learn a different way. I start almost every experiment with the three-file system of foo.asd, package.lisp, and foo.lisp, which makes it and its dependencies trivially quickloadable. And the experiments often grow into something I want to save for easy reuse.

I'm fine with learning a different way. Actually, I experimented with your quickproject utility when you first released it. I like the idea of it. But the single significant drawback for me is that it seems to require project files to be located to suit the needs of the tool rather than the needs of (for me) the work. If I'm contracting for companyX their files all go somewhere within a companyX folder. Their data goes there. Their docs. Everything. I can tarball the whole thing up and send it to them. I can archive it. I can send it to Richard Snowden. Whatever. It seems to me that the requirements of the package methodology you are suggesting requires that I either do my contract work in subdirectories of the quicklisp folder of some other standard ASD visible folder, or be continually modifying the ASD paths so the loader knows where to find them. Maybe I haven't just wrapped my head around it right, but it seems odd to have to build a package just to load some other packages and use them.

I agree that many, many times what started out experimental developed into a package I've reused for other projects. I think if I could figure out how to keep files organized by job rather than by tool I'd be all for your idea.

--Jeff

-- My Best, Dave Cooper, Genworks Support david.cooper@genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699