Andrew Robin writes:

Hi Markus,

If Dr, Schafmeister hasn't suggested an area for you to work on, can you tell us what areas of Clasp you are interesting in working and where your skills are strongest (C++, build tools, Lisp)? Your efforts in getting Clasp to compile are appreciated, and I encourage you to keep working on this project!

hi andy,

nice to meet you too.

it looks like you are keen to keep me in the volunteer labor pool :-)

yeah, drmeister has been silent so far. i don't know if he even reads this mailing list. i have not tried to bug him harder yet.

i am primarily interested in lisp, precisely in order to avoid c++ and whatnot, but it looks like this won't be so simple with this project. i am actually wondering whether it is a good use of time. it seems to me like it still will take a huge amount of effort to turn this into a user friendly lisp and productive implementation. (this time might be better spent writing molecule design code...)

part of my own problem is that i don't try to spend time updating all of my software, environment, and dependencies constantly, all the time. in particular, i'm still using an old version of xemacs that i like, and i'm not likely to change all that. turns out now that slime has discontinued support for xemacs, apparently. i think this is crazy. but what it means is that i have not yet seen slime interact with clasp, which maybe would return actual stack backtraces ? does it actually ?

i tried copying over the clasp-specific file to my older slime installation and tweak it a bit, but it does not get very far. these are the types of error messages i get:

(progn (load "/lisp/elisp/slime/swank-loader.lisp" :verbose t) (funcall (read-from-string "swank-loader:init")) (funcall (read-from-string "swank:start-server") "/tmp/kr/slime.32538"))

Starting MPS Clasp 0.2 ... loading image... it takes a few seconds

Condition of type: SIMPLE-ERROR Could not compile func No restarts available.

Top level.

Debugger received error of type: SIMPLE-PROGRAM-ERROR Unbound symbol-value for PROGN Error flushed.

;;; Loading #P"/lisp/elisp/slime/swank-loader.lisp" T

;;; Loading #P"/home/kr/.slime/fasl/2012-04-27/clasp-mps-0.2-5d59a06e-linux-x86-64/swank-backend.fasl" ; Compiling file: /opt/elisp/slime/swank-clasp-prelude.lisp Writing fasl file to: #P"/home/kr/.slime/fasl/2012-04-27/clasp-mps-0.2-5d59a06e-linux-x86-64/swank-clasp-prelude.fasl" Linking app-resources:lib;release;intrinsics_bitcode_mps.o Linking #P"/home/kr/.slime/fasl/2012-04-27/clasp-mps-0.2-5d59a06e-linux-x86-64/swank-clasp-prelude.bc" Running link time optimization module pass manager Generating object file /home/kr/.slime/fasl/2012-04-27/clasp-mps-0.2-5d59a06e-linux-x86-64/swank-clasp-prelude.lbc --> /home/kr/.slime/fasl/2012-04-27/clasp-mps-0.2-5d59a06e-linux-x86-64/swank-clasp-prelude.o reloc-model: RELOC-MODEL-PIC- ;;; Loading #P"/home/kr/.slime/fasl/2012-04-27/clasp-mps-0.2-5d59a06e-linux-x86-64/swank-clasp-prelude.fasl" ; Compiling file: /opt/elisp/slime/swank-clasp.lisp Loading sockets.lisp;; ;; Error while compiling /opt/elisp/slime/swank-clasp.lisp: ;; FUNCTION special operator only supports symbol names or lambda expression - you gave: NIL ;; Aborting. ;; Debugger received error of type: SIMPLE-CONTROL-ERROR Restart ABORT is not active. Error flushed.

Debugger received error of type: SIMPLE-PACKAGE-ERROR In interpret_token_or_throw_reader_error ../../src/core/lispReader.cc line 349 COMMON-LISP:PACKAGE-ERROR :initializers (:PACKAGE "SWANK" )

Error flushed.

i find the error messages not very helpful:

Condition of type: SIMPLE-ERROR Could not compile func

well, which function was it trying to compile ?

or:

;; FUNCTION special operator only supports symbol names or lambda expression - you gave: NIL

it would help if a stack backtrace were dumped into the terminal.

and the line number in the lisp source file, where the error occurred, should be printed too.

so, if other people also don't see much more debugging info, maybe a first task for me would be to find out how to print more illuminating details of what goes wrong...

what are the top 5 under-explained error messages, and how can they be reproduced ?

Hey guys,

Sorry for taking so long to get on board. I’m in Florida at a meeting and well, whatever - I’m on board now.

Andrew, thank you very much for following up. I’ll throw in my 2 cents.

One note - Markus and I know each other for almost 20 years. We see each other every couple of years and we share interests in nanotechnology and now Common Lisp.

(1) Clasp to avoid C++.

I think it would be a good idea to start exposing the C++ library that you want to expose to Clasp using the clbind library. I can help get you started. Take a look at https://github.com/drmeister/demo-clasp-cxx - to see what I’m talking about.

Then you will get a sense for how easy (or hard) it is. You may decide to keep going this way. I exposed a lot of clang functionality to Clasp doing this by hand. It only took me a couple of days.

A more general C++ groveler and interface builder would be great but it will require more work to develop. The good thing is that it is possible and the final result will be perfect in terms of its ability to grovel C++ code because it will incorporate the Clang C++/C/Objective C compiler.

(2) I’m working to reduce the project dependencies of Clasp as much as possible. Currently they are all included in “externals-clasp”. The build system is in flux as I do this.

(3) Debugging with Clasp is not great. I have two compilers that I’m working with. The old bootstrapping compiler generates some debugging information and the backtraces are kind of useful. The new compiler Cleavir generates no debugging information yet but will generate great debugging information in weeks to months.

(4) I don’t know what 4 was.

Best,

.Chris.

On Jun 23, 2015, at 12:50 PM, Andrew Robin andrew.t.robin@gmail.com wrote:

Yes, I want you to volunteer--but drmeister didn't send me to recruit you. :) I just don't want to see people interested in this project dropping out because no one answers their post. I'm not sure drmeister reads this mailing list that he created either. He isn't posting to it much, for sure. I'd like to see the project in a state where he doesn't have to--where people like me can. Currently, work on the project seems to get done through the IRC #clasp channel. I have counted about 5 different modes of communication on the Clasp project. I prefer the mailing list format because it is threaded, coherent, archived, and doesn't challenge my limited attention span.

So let's address some of your ideas:

(1) Clasp to avoid C++

This is the reason I am interested in the project, too. I need to wrap a large C++ library with Lisp--I want to use Lisp where I can. Not because I dislike C++, but for what I am doing Lisp is better in a number of ways. C++1x gives more Lisp-like features, but it is just getting too complicated, IMO. I talked to drmeister and others on #clasp about having a more transparent way (read least C++ LOC) of doing the binding. His current clbind subproject requires the use of C++ boilerplate. It seems like having the AST of the C++ code would allow generating even easier to use binding code IN LISP. I have a more detailed post on this mailing list regarding this issue--no one commented on this one either :(

Ideally, a project like Clasp should help you do chemistry more efficiently, not navigate mind bending C++ code. It's obviously not there yet, but I haven't given up hope. I'm primarily a software developer, and have experience working with scientists and engineers on technical projects. I hope to contribute that experience to this project.

(2) Project dependencies

Unfortunately, with newer projects it takes time to get things sorted out. I get annoyed when I have to install a mess of dependencies which deprecate older versions of software I depend on. One recent project (which will remain unnamed) I installed had a dependency chain that wiped out X windows without asking! I'm working on understanding the external dependencies of Clasp better.

Again, ideally, it would be great to use standard (and older, stable) packages, prebuilt. drmeister has apparently patched LLVM and clasp (and boost?) to get his code running. Getting these patches accepted by established, production projects to support a fledgling project like Clasp would be hard (and maybe questionable). One idea I had was to see if the project dependencies could be extended (like with a plug-in) rather than patched.

Ir regard to your specific issues, could you install a supported of emacs in parallel, and use that for Slime? I haven't ever done this myself however. Another thing that might help you is creating a VM or a partition with a more recent version of your operating system for use with Clasp. Again, I am not the one to ask about doing this--just ideas.

(3) Debugging

Yes, we want to get the best debugging information we can. If I have any ideas about this after working with Clasp more, I will post them to this list. Please let us know what you come up with.

(4) Anybody else?

If you are still reading this--anybody else have comments? Chemists want self-organizing molecules, I want self-organizing code.

I agree with the subject prefix. Assuming it's using mailman look at:

http://www.gnu.org/software/mailman/mailman-admin.txt

Search for section:

2.1.1 General list personality

it is the subject_prefix variable

This has to be done by list administrator (you know who you are).

--Andy (o>

On 06/24/2015 05:34 AM, Markus Krummenacker wrote:

i also like it when the subject line has a prefix indicating the mailing list, such as [clasp-devel] chris, is there a way you could configure the ml to add such a prefix to the subject line ?

Marcus,

Here’s a bit of history on Clasp development.

(1) I wrote an S-expression walking Common Lisp interpreter and discovered that it was too slow for my needs. (2) I wrote a simple compiler that translated Common Lisp into LLVM-IR and I was able to get Common Lisp running but it was about 100x slower than SBCL. It runs CLOS. (3) I integrated a new compiler (Cleavir) that Robert Strandh is actively developing that makes extensive use of CLOS and performs many Common Lisp level optimization like escape analysis and it allows me to implement aggressive inlining. Robert is also adding a sophisticated type inference algorithm that will eliminate runtime type checks and dead code.

Cleavir will be the only compiler that users will interact with. The interpreter and bootstrapping compiler are just there for bootstrapping from a C++ compiler.

Unfortunately, Cleavir is not finished yet, but Robert is working on it and I am working on integrating his changes into Clasp. If you want to wait a few months until it is more complete and performance improves and things stabilize that would be fine with me. You may also decide that it’s just too raw and uncertain and avoid it all together - also fine.

The Cleavir compiler reuses most of the bootstrapping compiler code.

DWARF is the debugging format used by the GCC and Clang compilers. It’s the state of the art for C++/C debugging. I’m adapting it for Common Lisp. What it will allow is debugging of C++/Common Lisp code at the same time. It will also enable profiling of both C++ and Common Lisp code together.

I’m guessing your handle is kr_. Check out irccloud.com — it is the IRC client that I use.

But it’s great to use the mailing list - I think I’ve got it figured out now.

On Jun 24, 2015, at 1:26 AM, Markus Krummenacker kr@AI.SRI.COM wrote:

Christian Schafmeister writes:

...

Hey Markus,

You will have a better debugging experience if you use Slime.

We should talk and I’ll give you a better update as to where Clasp is wrt optimization and debugging. It’s kind of a complex picture that is changing rapidly. In a nutshell:

1. I’m incorporating inlining and a new compiler as we speak and this should produce a large improvement in performance.

hi chris,

so having 2 compilers floating around does create the puzzle of where exactly improvements should be made. i should probably pick something to do that will survive the transition from one compiler to the other...

...

2. The debugging experience is mixed and should be improved

soon. I’m incorporating a new compiler but it generates no debugging information at the moment. I’m adding DWARF debugging support so that we can debug using gdb or lldb.

i know nothing about dwarves, but if such debug info could be used on the lisp side as well, maybe i should study that a bit. (though it will probably take me quite a bit of time to digest. i don't want to prevent other people from investigating this as well.)

...

3. On freenode there is a #clasp chatroom - could you drop in and I can update you on where things are?

quite frankly, i have no idea how to use irc chat. i did use firefox to visit the freenode website, and i seem to have joined #clasp, but i don't know how i would type anything, it does not do anything for me, and i don't see any activity. maybe it is also getting late on the east coast.

anyways, my cultural preference generally is towards "traditional" email and mailing-lists, that are properly archived, and where communication is asynchronous.

i would think explanations regarding what glue holds clasp together these days would be of broad interest on the mailing list too, instead of just the presumably ephemeral irc channel.

--

Regards Markus Krummenacker

...

Best,

.Chris.

On Jun 14, 2015, at 5:17 PM, Markus Krummenacker kr@AI.SRI.COM wrote:

...

after a few more hickups, i finally succeeded in building clasp, on linux.

one thing i wonder is how to get a stack backtrace, if there is a break. just interacting at the prompt just seems to return to the prompt, flushing the error, like this:

...

(+ 2 'b)

Debugger received error of type: SIMPLE-TYPE-ERROR In lisp_errorUnexpectedType ../../src/core/foundation.cc line 163 COMMON-LISP:TYPE-ERROR :initializers (:DATUM B :EXPECTED-TYPE COMMON-LISP:NUMBER )

Error flushed.

...

that'll make it difficult to get to the bottom of various issues.

also, drmeister, you wanted me to look into speeding up clasp. but i'll need some pointers on how to get started on this.

it would be good if there were a rough "how to optimize" guide available. is there a worked out example for me to look at ? which function/file ?

and could you please assign one of those open tasks to me ? it will probably take some time for me to chew on, but i'd prefer to work on something that doesn't duplicate what other people might be working on.

thanks.

--

Regards Markus Krummenacker

It's worth a lot, since most of the development activity has happened on the IRC channel! Thanks, Nicolas.

On 06/24/2015 05:45 AM, Nicolas Hafner wrote:

For what it's worth, the channel is archived at http://log.irc.tymoon.eu/freenode/clasp