I'd also would sometimes like to not use Slime's debugger. The
doc here

  http://common-lisp.net/project/slime/doc/html/Other-configurables.html

seems to be trying to say how to do it, but I cannot get it to
work. It says

  SWANK:*GLOBAL-DEBUGGER*
    When true (the default) this causes *DEBUGGER-HOOK* to be
    globally set to SWANK:SWANK-DEBUGGER-HOOK and thus for SLIME
    to handle all debugging in the Lisp image. This is for
    debugging multithreaded and callback-driven applications.

But when I tried putting

  (setq swank:*global-debugger* nil)

in my ~/.swank.lisp file, there seemed to be no effect: errors
still invoked SLDB in its own buffer, and the built-in debugger
was not invoked.

What am I missing? 

Thanks,

Mark