September 2020 - cmucl-cvs - mailman3.common-lisp.net

[Git][cmucl/cmucl][master] 2 commits: Add CL-bench
by Raymond Toy 05 Sep '20

05 Sep '20

Raymond Toy pushed to branch master at cmucl / cmucl Commits: 6ae9025c by Raymond Toy at 2020-09-05T02:38:38+00:00 Add CL-bench >From Eric Marsden at http://emarsden.chez.com/downloads/cl-bench.tar.gz. Checked in as is. - - - - - 09c8327a by Raymond Toy at 2020-09-05T02:38:39+00:00 Merge branch 'rtoy-issue-87-add-cl-bench' into 'master' Fix #87: Add cl-bench to CI Closes #87 See merge request cmucl/cmucl!55 - - - - - 17 changed files: - .gitlab-ci.yml - + benchmarks/cl-bench/Makefile - + benchmarks/cl-bench/NEWS - + benchmarks/cl-bench/README - + benchmarks/cl-bench/TODO - + benchmarks/cl-bench/defpackage.lisp - + benchmarks/cl-bench/do-compilation-script.lisp - + benchmarks/cl-bench/do-execute-script.lisp - + benchmarks/cl-bench/do-interpret-script.lisp - + benchmarks/cl-bench/files/.gitignore - + benchmarks/cl-bench/files/arrays.lisp - + benchmarks/cl-bench/files/bignum.lisp - + benchmarks/cl-bench/files/boehm-gc.lisp - + benchmarks/cl-bench/files/cl-ppcre/api.lisp - + benchmarks/cl-bench/files/cl-ppcre/cl-ppcre-test.system - + benchmarks/cl-bench/files/cl-ppcre/cl-ppcre.system - + benchmarks/cl-bench/files/cl-ppcre/closures.lisp The diff was not included because it is too large. View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/69064be1625fc725cbc502… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/69064be1625fc725cbc502… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][rtoy-issue-87-add-cl-bench] Add benchmark path to artifacts
by Raymond Toy 04 Sep '20

04 Sep '20

Raymond Toy pushed to branch rtoy-issue-87-add-cl-bench at cmucl / cmucl Commits: ea0601c0 by Raymond Toy at 2020-09-04T15:05:41-07:00 Add benchmark path to artifacts Forgot to copy the artifacts path from linux to mac so that we save the benchmark results. - - - - - 1 changed file: - .gitlab-ci.yml Changes: ===================================== .gitlab-ci.yml ===================================== @@ -34,6 +34,7 @@ osx-runner: artifacts: paths: - ansi-test/test.out + - benchmarks/cl-bench/results tags: - osx before_script: View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/ea0601c06181f3929bdec8d… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/ea0601c06181f3929bdec8d… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][rtoy-issue-87-add-cl-bench] 2 commits: Remove debugging print
by Raymond Toy 04 Sep '20

04 Sep '20

Raymond Toy pushed to branch rtoy-issue-87-add-cl-bench at cmucl / cmucl Commits: 5551b146 by Raymond Toy at 2020-09-04T14:39:07-07:00 Remove debugging print - - - - - 42e0a041 by Raymond Toy at 2020-09-04T14:40:19-07:00 Run benchmarks on OSX * Remove some debugging script lines * Copy the linux script to run on osx - - - - - 2 changed files: - .gitlab-ci.yml - benchmarks/cl-bench/report.lisp Changes: ===================================== .gitlab-ci.yml ===================================== @@ -25,8 +25,6 @@ linux-runner: - make LISP="../dist/bin/lisp -batch -noinit -nositeinit" - grep 'No unexpected $successes\|failures$' test.out - cd ../benchmarks/cl-bench - - echo $PWD - - ls - mkdir tmp - CMUCL=../../snapshot/bin/lisp ./run-cmucl.sh - CMUCL=../../dist/bin/lisp ./run-cmucl.sh @@ -51,3 +49,8 @@ osx-runner: - cd ansi-test - make LISP="../dist/bin/lisp -batch -noinit -nositeinit" - grep 'No unexpected $successes\|failures$' test.out + - cd ../benchmarks/cl-bench + - mkdir tmp + - CMUCL=../../snapshot/bin/lisp ./run-cmucl.sh + - CMUCL=../../dist/bin/lisp ./run-cmucl.sh + - ../../snapshot/bin/lisp -load report ===================================== benchmarks/cl-bench/report.lisp ===================================== @@ -30,7 +30,6 @@ "CL-benchmark*.*")) data implementations benchmarks) (dolist (f (directory benchmark-path)) - (format t "*** f = ~A~%" f) (ignore-errors (with-open-file (f f :direction :input) (let ((*read-eval* nil)) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/bc890d42e09aed670c3195… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/bc890d42e09aed670c3195… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][rtoy-issue-87-add-cl-bench] Set output file for FPRINT benchmark.
by Raymond Toy 04 Sep '20

04 Sep '20

Raymond Toy pushed to branch rtoy-issue-87-add-cl-bench at cmucl / cmucl Commits: bc890d42 by Raymond Toy at 2020-09-04T14:28:55-07:00 Set output file for FPRINT benchmark. The FPRINT benchmark needs a file and it was set to "/tmp/fprint.tst", which doesn't exist on gitlab CI. Just use "./tmp/fprint.tst". Let CI recipes create the "tmp" directory so the scripts don't need to. - - - - - 2 changed files: - .gitlab-ci.yml - benchmarks/cl-bench/files/gabriel.lisp Changes: ===================================== .gitlab-ci.yml ===================================== @@ -27,6 +27,7 @@ linux-runner: - cd ../benchmarks/cl-bench - echo $PWD - ls + - mkdir tmp - CMUCL=../../snapshot/bin/lisp ./run-cmucl.sh - CMUCL=../../dist/bin/lisp ./run-cmucl.sh - ../../snapshot/bin/lisp -load report ===================================== benchmarks/cl-bench/files/gabriel.lisp ===================================== @@ -1018,7 +1018,9 @@ ;;; FPRINT -- Benchmark to print to a file. -(defparameter +fread-temporary-pathname+ "/tmp/fprint.tst") +(defparameter +fread-temporary-pathname+ + "./tmp/fprint.tst" + "Temporary file used for benchmarking fread") (defvar *fprint-test-atoms* '(abcdef12 cdefgh23 efghij34 ghijkl45 ijklmn56 klmnop67 View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/bc890d42e09aed670c31953… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/bc890d42e09aed670c31953… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][rtoy-issue-87-add-cl-bench] Add dir prefix to run run-cmucl.sh
by Raymond Toy 04 Sep '20

04 Sep '20

Raymond Toy pushed to branch rtoy-issue-87-add-cl-bench at cmucl / cmucl Commits: a7b237ae by Raymond Toy at 2020-09-04T14:10:18-07:00 Add dir prefix to run run-cmucl.sh Use ./run-cmucl.sh instead of run-cmucl.sh since . is probably not in $PATH. - - - - - 1 changed file: - .gitlab-ci.yml Changes: ===================================== .gitlab-ci.yml ===================================== @@ -27,8 +27,8 @@ linux-runner: - cd ../benchmarks/cl-bench - echo $PWD - ls - - CMUCL=../../snapshot/bin/lisp run-cmucl.sh - - CMUCL=../../dist/bin/lisp run-cmucl.sh + - CMUCL=../../snapshot/bin/lisp ./run-cmucl.sh + - CMUCL=../../dist/bin/lisp ./run-cmucl.sh - ../../snapshot/bin/lisp -load report osx-runner: View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/a7b237ae755f8e825b75355… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/a7b237ae755f8e825b75355… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][rtoy-issue-87-add-cl-bench] Add some debug
by Raymond Toy 04 Sep '20

04 Sep '20

Raymond Toy pushed to branch rtoy-issue-87-add-cl-bench at cmucl / cmucl Commits: 0dea3bcf by Raymond Toy at 2020-09-04T12:17:52-07:00 Add some debug Figure out why run-cmucl.sh doesn't exist, preventing us from running the benchmarks. - - - - - 1 changed file: - .gitlab-ci.yml Changes: ===================================== .gitlab-ci.yml ===================================== @@ -25,6 +25,8 @@ linux-runner: - make LISP="../dist/bin/lisp -batch -noinit -nositeinit" - grep 'No unexpected $successes\|failures$' test.out - cd ../benchmarks/cl-bench + - echo $PWD + - ls - CMUCL=../../snapshot/bin/lisp run-cmucl.sh - CMUCL=../../dist/bin/lisp run-cmucl.sh - ../../snapshot/bin/lisp -load report View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/0dea3bcfd2bdc218d3fbee6… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/0dea3bcfd2bdc218d3fbee6… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][rtoy-issue-87-add-cl-bench] 3 commits: Ignore files created from running the benchmarks
by Raymond Toy 04 Sep '20

04 Sep '20

Raymond Toy pushed to branch rtoy-issue-87-add-cl-bench at cmucl / cmucl Commits: 682553a2 by Raymond Toy at 2020-09-04T12:02:03-07:00 Ignore files created from running the benchmarks - - - - - ab2992c3 by Raymond Toy at 2020-09-04T12:04:40-07:00 Make run-cmucl.sh executable - - - - - a68d0c37 by Raymond Toy at 2020-09-04T12:06:37-07:00 Run benchmarks Run the benchmarks using the snapshot and then with the newly built version. Generate a report too. The benchmark results are added to the artifacts. - - - - - 3 changed files: - .gitlab-ci.yml - + benchmarks/cl-bench/files/.gitignore - benchmarks/cl-bench/run-cmucl.sh Changes: ===================================== .gitlab-ci.yml ===================================== @@ -7,6 +7,7 @@ linux-runner: artifacts: paths: - ansi-test/test.out + - benchmarks/cl-bench/results tags: - linux before_script: @@ -22,7 +23,11 @@ linux-runner: - bin/run-tests.sh -l dist/bin/lisp 2>&1 | tee test.log - cd ansi-test - make LISP="../dist/bin/lisp -batch -noinit -nositeinit" - - grep 'No unexpected $successes\|failures$' test.out + - grep 'No unexpected $successes\|failures$' test.out + - cd ../benchmarks/cl-bench + - CMUCL=../../snapshot/bin/lisp run-cmucl.sh + - CMUCL=../../dist/bin/lisp run-cmucl.sh + - ../../snapshot/bin/lisp -load report osx-runner: artifacts: ===================================== benchmarks/cl-bench/files/.gitignore ===================================== @@ -0,0 +1,13 @@ +/arrays.olisp +/bignum.olisp +/boehm-gc.olisp +/clos-janderson.olisp +/clos.olisp +/crc40.olisp +/deflate.olisp +/gabriel.olisp +/hash.olisp +/math.olisp +/misc.olisp +/ratios.olisp +/richards.olisp ===================================== benchmarks/cl-bench/run-cmucl.sh ===================================== View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/f15977bd7d814fd5ea30cf… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/f15977bd7d814fd5ea30cf… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][rtoy-issue-87-add-cl-bench] Store the results in our directory instead of /var/tmp
by Raymond Toy 04 Sep '20

04 Sep '20

Raymond Toy pushed to branch rtoy-issue-87-add-cl-bench at cmucl / cmucl Commits: f15977bd by Raymond Toy at 2020-09-04T11:03:27-07:00 Store the results in our directory instead of /var/tmp Adjust where the reports are stored when running the benchmarks and where the reports are read from. Instead of /var/tmp, use cl-bench/results. This will make it easier to find these files in CI. - - - - - 2 changed files: - benchmarks/cl-bench/report.lisp - benchmarks/cl-bench/support.lisp Changes: ===================================== benchmarks/cl-bench/report.lisp ===================================== @@ -14,18 +14,28 @@ (in-package :cl-user) +(defvar *benchmark-file-directory* + (merge-pathnames (make-pathname :directory '(:relative "results")) + (make-pathname :directory (pathname-directory *load-truename*))) + "Directory where the benchmark report file is stored") + (defconstant +implementation+ (concatenate 'string (lisp-implementation-type) " " (lisp-implementation-version))) (defun bench-analysis () - (let (data implementations benchmarks) - (dolist (f (directory "/var/tmp/CL-benchmark*.*")) + (let ((benchmark-path (concatenate 'string + (namestring *benchmark-file-directory*) + "CL-benchmark*.*")) + data implementations benchmarks) + (dolist (f (directory benchmark-path)) + (format t "*** f = ~A~%" f) (ignore-errors (with-open-file (f f :direction :input) (let ((*read-eval* nil)) (push (read f) data))))) + (format t "*** data = ~A~%" data) (setq implementations (delete +implementation+ (mapcar #'car data) :test #'string=)) (setq benchmarks (reverse (mapcar #'first (cdr (first data))))) (format t "~25a~10@a" "Benchmark" "Reference") ===================================== benchmarks/cl-bench/support.lisp ===================================== @@ -21,6 +21,10 @@ (defvar *benchmarks* '()) (defvar *benchmark-results* '()) +(defvar *benchmark-file-directory* + (merge-pathnames (make-pathname :directory '(:relative "results")) + (make-pathname :directory (pathname-directory *load-truename*))) + "Directory where the benchmark report file is stored") (defvar +implementation+ (concatenate 'string @@ -117,11 +121,14 @@ (defun benchmark-report-file () + (ensure-directories-exist *benchmark-file-directory*) (multiple-value-bind (second minute hour date month year) (get-decoded-time) (declare (ignore second)) + ;; Should we use pathnames directly instead of creating a string + ;; naming the file? (format nil "~aCL-benchmark-~d~2,'0d~2,'0dT~2,'0d~2,'0d" - #+win32 "" #-win32 "/var/tmp/" + *benchmark-file-directory* year month date hour minute))) ;; grr, CLISP doesn't implement ~<..~:> View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/f15977bd7d814fd5ea30cf0… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/f15977bd7d814fd5ea30cf0… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl] Pushed new branch rtoy-issue-87-add-cl-bench
by Raymond Toy 02 Sep '20

02 Sep '20

Raymond Toy pushed new branch rtoy-issue-87-add-cl-bench at cmucl / cmucl -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/tree/rtoy-issue-87-add-cl-bench You're receiving this email because of your account on gitlab.common-lisp.net.

1 0