clim:destroy-port does not succeed in closing an already closed connection because a stream error is signaled. This might happen after xkilling an application (the X server terminates the connection) or network problems.
Using :abort t for xlib:close-display cuts the connection and there will be no buffer flushing. My purposed patch is attached.
Note: SBCL users should put
(sb-sys:ignore-interrupt sb-unix:sigpipe)
into their .sbclrc. This is necessary because the first attempt to write to a broken pipe will result in a SIGPIPE, which must be ignored. Another catchable error (stream-error) is signaled none the less. Imho sigpipe will becomes ignored per default as discussed on sbcl-devel recently.