October 2006 - vial-devel@common-lisp.net

[vial-devel] Experiments in the land of Climacs
by Brad Beveridge 30 Oct '06

30 Oct '06

I mentioned in an earlier email that I felt I needed to at least investigate the possibility of using Climacs as the substrate for Vial. This weekend I have seriously investigated it. Basically, something of the order of 90% of things that Vial wants to do, Climacs also wants to do. So the good news is that there is a lot of potential for reuse, things like buffer management, syntax highlighting, basic editing commands, etc. The bad news is that Climacs is big and will take time to figure out, and converting it to use Vi style key handling is a big issue. Also, lots of Climacs assumes that it is in a Clim environment. This adds up to a huge task to take some of Climacs and no Clim and make something work. I prefer to develop incrementally, with small improvements. With that in mind here is my rough proposal for Vial's development. 1) Continue work on the current version of Vial until it has support for Slime, which is my original and main goal. This Vial will have the following features/bugs: - not aiming for exact Vim replication. - core editing features only, fancy features (macros, visual mode, correct syntax highlighting) are not implemented unless really required - code is allowed to be somewhat ugly as this version is almost a complete throw away - minor bugs will be tolerated as long as they are well known - The major feature that must be well coded is the Slime support, it is potentially the only part that will be kept. - At this stage we have a good debugger, and an acceptable editor. 2) There is ongoing Climacs work to make the key handling more general, so general that it will support Vi style modes. Hopefully by the time #1 is ready, this support will be part of Climacs. At this stage I will attempt to: - Integrate Vi style key handling and bindings into Climacs - Integrate Slime support into Climacs. - This will be Vial v2, I may stop here. 3) I'm largely a console guy, long term I would like to see a fancy Vial running in a terminal. This would be done by implementing an Ncurses backend for McClim. So, although I've spend much of the weekend trying to tease apart Climacs, ultimately I think I will go with the "worse is better" approach and continue with the code base I currently have. The major change in my thinking is that the current code base just became a throw-away item, and what is acceptable for the editing features dropped. I think that Vial is currently almost an acceptable editor, so I will now work on Slime support. Cheers Brad

1 0

[vial-devel] Re: What will break with flexichain
by Alessandro Piras 25 Oct '06

25 Oct '06

---------- Forwarded message ---------- From: Alessandro Piras <laynor(a)gmail.com> Date: Oct 24, 2006 5:41 PM Subject: Re: What will break with flexichain To: Brad Beveridge <brad.beveridge(a)gmail.com> On 10/24/06, Brad Beveridge <brad.beveridge(a)gmail.com> wrote: > .... > There is a grouping macro, WITH-UNDO-BLOCK that lets you group blocks > of undos. The saving of the cursor shouldn't be required - if you > think about it, when you undo/redo you are playing back or unplaying > keystrokes. If you are undoing an insert of "hello there", you should > delete 11 chars, and the point should be back before you inserted any > text. It works in most cases but in some don't... try theese things: insert "abcdefgh" move to the "d" letter delete "d" and "e" using the "x" command now undo the changes, you'll see the cursor going to the end of the line. instead of where it was before applying the x command. The same behavior presents when you insert text between words, let's say you have this line pino pigna you move the cursor between the 2 words, insert " is", go to normal mode and press u to undo. The cursor will go at the end of the line. Same thing for the "c<motion>" command and such. Good to know that there is already the macro for undo undo grouping ! > > > <SNIP> > > Tell me what do you think :) > > > I like it. I think that right now I would like it most if you thought > about issues like the "r" command, if we can get a framework that > covers all those cases it will be really good in the long run. I'll take a look to define pattern and think about a way to include all forms of vim commands, maybe those like d3d too :) I just hope it wont be too hard for a noob like me ^^ >Even > though I just posted saying "let's not move to Flexichain right now", > I'm leaning that way again. I'll write up my thoughts to the list, > but it basically boils down to the fact that being able to treat the > buffer as a giant string has many advantages over the list of lines > approach. > The problem with changing out buffers right now is that we make lots > of assumptions about the list of lines structure. We basically need > to change: > buffer, cursor, register, search, colours, autocomplete, mode-lisp, > regions, view, ncurses-io, undo > But, each module will benefit greatly from being able to treate the > buffer as a big string. > Guess I have a lot of work ahead :) > > Cheers > Brad > I thought you wanted to use a buffer made of a list of flexichains (gap buffers), not just one gap buffer to handle our buffer.. reading the flexichains docs it seems that it's better to use a list or a sequence of gap buffers rather than a big gap buffer.. In terms of performance it seems. I don't know wich are the great benefits that you are talking about though, so it could be that it's better having a little less performance and a clerarer code, at least now :) I think that we should put an abstraction layer also there, between the buffer module and all those modules that now depend on the implementation (the ones you listed above) ,if it makes sence, maybe one day we'll want to change the buffer implementation again, and it would be good not to change those modules again. Tell me what you think, and if it is feasible to put an abstraction layer also there :) I forgot to say, I find the idea of using the emacs api a really good one! Going to have a look at it soon. __ Alessandro

2 6

[vial-devel] Flexichain and Cursorchain
by Brad Beveridge 24 Oct '06

24 Oct '06

If it's not nailed down, steal it. That's my motto. Vial originally started life as a fun way to learn Common Lisp, which meant doing some stuff myself that maybe I shouldn't have. Particularly the buffer stuff. It might be worthwhile moving to Flexichain (http://common-lisp.net/project/flexichain/) which is the underlying implementation that Climacs uses for buffer management. I'm going to look at it harder tonight, and if I do decide to change then it will probably mean big changes for Vial's internal code - but probably changes that clean the code up quite a bit. If we do move to flexichain, the first step will be to abstract out all the code that currently plays with the internals of buffer or cursor objects, then re-implement those functions using Flexichain. Thoughts? Cheers Brad

2 3

[vial-devel] Re: What will break with flexichain
by Brad Beveridge 24 Oct '06

24 Oct '06

On 24/10/06, Alessandro Piras <laynor(a)gmail.com> wrote: > On 10/24/06, Alessandro Piras <laynor(a)gmail.com> wrote: > > On 10/24/06, Brad Beveridge <brad.beveridge(a)gmail.com> wrote: > > > Pretty much all the functions in buffer.lisp and cursor.lisp will be > > > preserved with the same semantics. Any place that directly references > > > lines-of buffer/cursor will need to change. > > > I'll make changes to remove any and all references to those buffer or > > > cursor members. > > > So, feel free to do what you like, but please: > > > Don't mess too much with cursor.lisp or buffer.lisp, > > > don't directly access (lines-of cursor) or (lines-of buffer) > > > Thanks :) > > > > > > Cheers > > > Brad > > > > > > > Ok, I'll keep working on commands and things. Though It would be a > > good thing defining a more abstract interface for buffers, a sort of > > "layer" that could be hacked inside that would be useful to: > > > > 1) handling some bugs of buffer implementation(s) we('ll) use > > 2) Isolate theese hacks to a place > > 3) Stabilizing a "language" used to write commands so they're > > independent from the internals. > > 4) Minimize the changes needed when internals have to change for some > > reasons (we should only fix/modify the layer code, that's used from > > many commands) > > > > When the buffer becomes stable, and it works well, we can tighten up > > the interface, keeping commands' code the same, > > > > Tell me how does it sound :) That sounds really good. And I think that we already have the API layed out - Emacs' internal API is actually really good : http://www.delorie.com/gnu/docs/elisp-manual-21/elisp_toc.html Trying to emulate the Emacs API has a few advantages 1) Don't need to invent a text-editing API 2) Already have good documentation, as long as we follow the behaviour 3) Low barrier to entry for people already familiar with that API. > I forgot to say, we need also another type of command, different from > motion, op or op-motion. With those three types it's hard (if > possible) to code commands like "r", which is used a lot (it replace > the char under the cursor with another, without entering insert mode), > that take a character as an argument. > Actually there are only a few commands that work that way (I only know > of 3, "r" "f" "F"), but given the importance of replace-char it should > be worth having a way to define them. also, maybe it's good if you're > going to touch the buffer code, to think about a good interface for > text replacing, so we could have a better implementation than > removing/inserting the char, that maybe just does actual overwriting > of text. I decided that this should go to the list, for archive purposes. Hope you don't mind :) You're right about the "r" command. We should find all those "special" cases, and make sure we can handle them. "r" is something I haven't considered yet, and I've been putting off replace mode :) > Another thing to think about is a little annoyance in the > current undo system. It works well, but some times not in the > predictable way... I'll explain this to you better showing the cases I > discovered, because it's better seeing than trying to figure it out > from an explanation. The undo system we have just needs a few things > more to work the way one would think, that are operation-grouping and > cursor position saving. > The good thing is that it can be easily (i guess) built on top of the > current undo system. It would work this way: > > let's say we have a command that executes 3 basic operations (the J > command is an example) > > op1 > op2 > op3 > > we just need to wrap those in a group. > also, we need to remember the cursor position before the command, so > the thing we have to save is actually There is a grouping macro, WITH-UNDO-BLOCK that lets you group blocks of undos. The saving of the cursor shouldn't be required - if you think about it, when you undo/redo you are playing back or unplaying keystrokes. If you are undoing an insert of "hello there", you should delete 11 chars, and the point should be back before you inserted any text. > <SNIP> > Tell me what do you think :) > I like it. I think that right now I would like it most if you thought about issues like the "r" command, if we can get a framework that covers all those cases it will be really good in the long run. Even though I just posted saying "let's not move to Flexichain right now", I'm leaning that way again. I'll write up my thoughts to the list, but it basically boils down to the fact that being able to treat the buffer as a giant string has many advantages over the list of lines approach. The problem with changing out buffers right now is that we make lots of assumptions about the list of lines structure. We basically need to change: buffer, cursor, register, search, colours, autocomplete, mode-lisp, regions, view, ncurses-io, undo But, each module will benefit greatly from being able to treate the buffer as a big string. Guess I have a lot of work ahead :) Cheers Brad

1 0

[vial-devel] darcs patch: normal mode commands, cc, c<motion>, s, S, -, +, J, X
by laynor＠gmail.com 23 Oct '06

23 Oct '06

Mon Oct 23 17:59:36 CEST 2006 laynor(a)gmail.com * normal mode commands, cc, c<motion>, s, S, -, +, J, X Added some normal mode commands: "cc": change the active line "c<motion>": change the region defined by <motion> "s": subst char, deletes a character like "x" and then goes to insert mode "S": subst line, same as "cc" "-": goto previous line beginning, moves the cursor to previous line at the 1st column "+": goto next line beginning, moves the cursor to the next line at the 1st column "X": deletes previous char (like backspace in insert mode, but it works in normal mode) "J": Joins the active line and the next line, It deserves to be fixed, because the undo is broken (it popups the debugger when the command has no effect (when applied on the last line)) ---- Strange Behaviors (with respect to vim) ---- "c<motion>" doesnt behave the same as vim, just try it in gvim and in vial, it's faster than trying to explain it here. In particular, "cj" doesnt open a new line after deleting, and cw eats the backspace that separates words. ---- Broken behavior ---- join-lines-command ("J") breaks the undo. To reproduce this behavior, just go to the last line of a file and press J, and then undo.

3 7

[vial-devel] darcs patch: clear-current-search, :noh ex command
by laynor＠gmail.com 23 Oct '06

23 Oct '06

Mon Oct 23 04:33:39 CEST 2006 laynor(a)gmail.com * clear-current-search, :noh ex command Added ex command :noh It clears the current search, and removes highlight. Got rid of some code duplication introduced defining remove-highlight, and using it in both search-forward-highlight-all and clear-current-search

1 0

[vial-devel] Re: vial, ncurses-cffi
by Alessandro Piras 22 Oct '06

22 Oct '06

I already ran make in the ncurses-cffi directory, but it seems that it's not the point... running make i get the ncurses.lisp and ncurses-cffi.lisp files, but loading with asdf says that thing anyway... I also tried to compile the lisp files manually, and reload with asdf but I just get the same error. Is it possible that it's a problem with common lisp controller? On 10/22/06, Brad Beveridge <brad.beveridge(a)gmail.com> wrote: > > On 22/10/06, Alessandro Piras <laynor(a)gmail.com> wrote: > > Hi, i was trying to have a test run of vial, but when i load it with > > > > (asdf:operate 'asdf:load-op 'vial) > > > > it says > > > > debugger invoked on a SB-INT:SIMPLE-FILE-ERROR in thread #<THREAD > "initial > > thread" {A820561}>: The path > > > #P"/var/cache/common-lisp-controller/1000/sbcl/local/home/ale/src/vial/ncurses-cffi/ncurses.fasl" > > does not exist. > > > > I actually don't know what to do, should i manually compile > > ncurses.lisp > > file (the output of swig) and copy it to the that path? > > I guess there's a > > better way to do it, but don't know how :) > > > > > > > Hi Alessandro, please feel free to mail to the vial mailing list. If > you have SWIG you can just run 'make' from the cffi directory. > Or you can do another "darcs get", I've checked that file in - it > should have been there from the start. > > Cheers > Brad >

2 7

[vial-devel] Re: [slim-vim] Vial has a home on common-lisp.net
by Brad Beveridge 22 Oct '06

22 Oct '06

On 22/10/06, Daniel Trstenjak <Daniel.Trstenjak(a)online.de> wrote: > > Hi Brad, > > At the moment you really seem to have a productive phase :). > I find that it's very admirable to start a new editor, > because for me personally it would be very painful to redo > the work all others already have done. Yah :) But I justify it to myself with a few reasons 1) I'm learning Lisp and need a project 2) There are no Vi style editors written in Common Lisp 3) Starting an editor from scratch may be stupid, but is certainly not unprecedented :) > In a former mail you said that you started Vial because > the structure of Vim wasn't sufficient enough for a > good Slime/Lisp development environment. > > Is it really fully illusive to make a fork of Vim > and restructure it, so that it will become a sufficient > Slime/Lisp development environment. > > I don't want to make any proposal to you, I'm only > interested in the fact. Firstly, I'd like to start out by saying that Vim is my editor of choice and from a user perspective it is a fantastic editor. Now I'm going to disparage Vim's code :) I'm a C guy by day, so I think I know my way around C code. Vim's code is some of the ugliest C code I've encountered. It is also pre-ansi C compatible, which makes it pretty ugly. Vim has been designed to allow features to be compiled in and out, but the way this is implemented is to surround code with #ifdef/#endif blocks - so the code is littered with feature specific code wrapped in blocks. There are functions that are pages and pages long. Modules are thousands or tens of thousands of lines long. The code is deeply entangled with itself, it is very hard to pick one small bit of code and confidently change it without knowing the entire system. There are over 350K lines of code in Vim. Vim also has a very odd execution structure, it can block waiting for input in many places and what you are allowed to do in each place might be different. Also, I don't think that Bram is super willing to accept major changes to Vim. There was an effort to integrate Vim to KDE, which eventually faltered after about 2 years. Those guys started the yzis.org project, an attempt to write an editor backend that runs like Vim. That might be a good project to hook in to, but they aren't moving that quickly. This all makes for a very painful development experience, and forking Vim would be very hard. It might have been worth looking at another Vi like editor (elvis, nvi), but the first steps there would be to integrate ECL and try from scratch. I think that so far my choice to start fresh has been a good one, I'm learning CL as I go, but so far my feature list is: - Unlimited undo/redo - Simple conceptual model - Easy to add Vim commands [1] - Multiple buffer support - Multiple window support - Regex searching - colour support - search highlighting - autocomplete for EX commands (ie :e filen<tab> -> :e filename.lisp) - basic commands: jkhl, d, dd, 0, $, a, A, i, I, y, yy, p, u, <C-R> All this in less than 2800 lines of code, and I intend to reduce that line count when I have a clean up soon. Less lines means less bugs and easier to understand. One of the goals of Slim-vim was to provide CL code that could be easily used to interface to Slime in other projects - that is still a goal. Slim-vim and Vial will stay separate projects, though pretty soon Vial will be getting Slim-vim support. For more thoughts about Vial, read the wiki page http://bradb.nfshost.com/stuff/index.cgi/Vial, or grab the code and read overview.notes I hope that I've managed to explain my reasoning well enough :) Vial is currently usable, but only if you are prepared to have some issues and basically be a developer. If you feel like checking it out, please do - I'd love to get another developer or two working on it. Cheers Brad [1] The command to delete a region is "d{motion}", ie "d$" "d0" "dj" "d2j". The code to implement this is: (defun command-d-motion (region) (active-register-needs-updating) (delete-region region :whole-lines t)) (define-pattern 'op-motion "d" #'command-d-motion) There is of course magic behind DEFINE-PATTERN, but for commands that operate as operation-motion, the function simply needs to take a region. Repeats of the command or of the motion are taken care of automatically.

1 0

[vial-devel] test
by Brad Beveridge 22 Oct '06

22 Oct '06

Just a test

1 0