[iolib-devel] crash with the current iolib - iolib-devel - mailman3.common-lisp.net

4 Jun 2007


      Hi,
First I want to thank you for your great effort. I tried to stress the
current iolib. Attached is a simple HTTP reply generator. I used sbcl
1.0.5 VM on a 32 bit AMD desktop machine. I used siege with lot's of
concurrent connections. Here is the first inspection:
This is my stupid test server with current iolib (with a small content
size).
aycan@zen ~ $ siege -b -c10 -t5S http://localhost:8001/
** siege 2.61
** Preparing 10 concurrent users for battle.
The server is now under siege...
Lifting the server siege..      done.
Transactions:                   6996 hits
Availability:                 100.00 %
Elapsed time:                   4.86 secs
Data transferred:             615648 bytes
Response time:                  0.01 secs
Transaction rate:            1439.51 trans/sec
Throughput:                126676.54 bytes/sec
Concurrency:                    9.53
Successful transactions:        6996
Failed transactions:               0
Longest transaction:            4.25
Shortest transaction:           0.00
I got 1490 transactions per second. The performance is great (I got
around 500 trans/sec with a threaded version) but when I repeat the
test, SBCL crashes silently and eats all my cpu. I don't get any errors
or some output. Also the :timeout parameter to add-fd should not be nil
by default, if so I got type errors with lengthy content sizes.
Here is the same siege parameters applied to *apache-tomcat-6-0-13* (the
content size is around 7600 bytes):
aycan@zen ~ $ siege -b -c100 -t5S http://localhost:8080/
** siege 2.61
** Preparing 100 concurrent users for battle.
The server is now under siege...
Lifting the server siege..      done.
Transactions:                  16083 hits
Availability:                 100.00 %
Elapsed time:                   6.71 secs
Data transferred:          118274382 bytes
Response time:                  0.00 secs
Transaction rate:            2396.87 trans/sec
Throughput:              17626584.40 bytes/sec
Concurrency:                   10.63
Successful transactions:       16083
Failed transactions:               0
Longest transaction:            4.18
Shortest transaction:           0.00
Is it possible to outperform java io performance?
--
aycan
-- 
Aycan iRiCAN
C0R3 Computer Security Group
http://people.core.gen.tr/~aycan.irican/


(in-package :asdf)

(eval-when (:compile-toplevel :load-toplevel)
  (asdf:oos 'asdf:load-op 'iolib))

(defpackage :iolib-problem
  (:use #:common-lisp #:net.sockets #:iomux))

(in-package :iolib-problem)

(defparameter *server* nil)
(defparameter *event-base* (make-instance 'iomux:event-base))
(defparameter *event-loop-thread* nil)
(defparameter *server-event* nil)

(defun handle-connection (sock handler)
  (unwind-protect
       (progn
     (funcall handler sock)
     (finish-output sock))
    (close sock)))

(defun make-server (port)
  (let ((sock (create-socket :address-family
    		     :internet
    		     :type
    		     :stream
    		     :connect
    		     :passive
    		     :ipv6 nil)))
    (bind-address sock +ipv4-unspecified+ :port port)
    (socket-listen sock)
    sock))

(defun add-connection-handler (event-base connection request-handler)
  (let ((handle))
    (labels
    ((close-handler ()
       (remove-event event-base handle)
       (close connection))
     (handler (fd evtype)
       (declare (ignorable fd)) 
       (handler-case
           (cond
    	 ((eq evtype :read)
    	  (funcall request-handler connection)
    	  (close-handler))
    	 (t (close-handler)))
         (condition (c)
           (declare (ignorable c))
           (describe c)))
       ))
      (setf handle (iomux:add-fd event-base (sockets::socket-fd connection) :read #'handler :persistent t :timeout 1)))))

(defun add-single-threaded-server (event-base listener-socket request-handler)
  (labels
      ((handler (fd evtype)
     (declare (ignorable fd))
     (case evtype
       (:read (let ((connection (accept-connection listener-socket)))
    	    (add-connection-handler event-base connection request-handler)))
       (:otherwise (break)))))
    (iomux::add-fd event-base (sockets::socket-fd listener-socket)
    	   :read #'handler :persistent t :timeout 1)))

(defun start-server (port)
  (setf *server* (make-server port))
  (setf *server-event*
    (add-single-threaded-server *event-base* *server* #'test-handler))
  (event-dispatch *event-base*))

(defun stop-server ()
  (close *server*)
  (remove-event *event-base* *server-event*)
  (setf *server* nil)
  (setf *server-event* nil))

(defparameter *response-body* "<HTML><HEAD><TITLE>Test</TITLE></HEAD><BODY>AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA</BODY></HTML>")
(defparameter *response-header* (format nil "HTTP/1.1 200 OK~%Content-Type: text/html~%Content-Length: ~A~%"
    				(length *response-body*)))

(defun test-handler (sock)
  (format sock "~A~%~A" *response-header* *response-body*))