Good day to everyone.
The bundle attached herewith contains patches to upgrade CL-JSON to the new version 0.4.0 which incorporates changes we have been discussing of late. The changes are rather wide-scale (the sheer size of the patches is 415k), almost every aspect of the implementation differs from what we had in version 0.3.1 and earlier. The most important points are:
1. The decoder now has the customization mechanism from decoder- vars.lisp, with imperative handlers and aggregate-scope variables.
2. The encoder provides a streaming API (borrowed from YASON). The standard encode methods are re-implemented using that API.
3. The CLOS decoder has undergone serious modifications to address the issue of degrading performance in some Lisp implementations. It shall be possible to use anonymous “fluid” objects created by the decoder in a way which makes for a “poor man's JavaScript”. The CLOS encoder does not emit prototype metadata anymore (unless specifically requested).
4. I have written a new pair of matching functions to convert between camel case and Lisp names.
5. There is an improved exception signalling mechanism. Some things which previously were handled by callbacks now use conditions / restarts.
6. JSON-BIND now uses the event-based decoder API with dynamic customization. This has the advantage that the user may choose the semantics with which to decode interior values. However, I had to forego the support for Lisp trees as arguments to JSON-BIND.
7. Some exported names were changed: Condition JSON-PARSE-ERROR → JSON-SYNTAX-ERROR Macro WITH-LIST-DECODER-SEMANTICS → WITH-DECODER-SIMPLE- LIST-SEMANTICS Macro WITH-CLOS-DECODER-SEMANTICS → WITH-DECODER-SIMPLE- CLOS-SEMANTICS
I have made sure the tests pass in Clozure CL, SBCL, CMUCL, CLisp, and ECL.
There is a new User's Manual in doc/cl-json.html (I have also uploaded a copy to http://dpworks.net/smilga/cl-json.html). And here I would like to humbly ask someone on this mailing list who is a native speaker of English to proof-read it. Russian, my natural idiom, is quite unlike English in grammar, conventional style, and punctuation. Though I am more or less fluent in the latter, I never fully trust myself, especially with the choice of articles, “which” / “that”, and modal and causative constructions. Also, if you think some passages confused or inarticulate, by all means do not hesitate to point me to them. Thank you in advance.
Sincerely, - B. Smilga.
Good day to everyone.
The previous version of this message seems to have been swallowed by a filter somewhere on the line, but if it has got through, I apologize for double posting.
I have prepared patches to upgrade CL-JSON to the new version 0.4.0 which incorporates changes we have been discussing of late. The changes are rather wide-scale (the sheer size of the patches is 415k), almost every aspect of the implementation differs from what we had in version 0.3.1 and earlier. The most important points are:
1. The decoder now has the customization mechanism from decoder-vars.lisp, with imperative handlers and aggregate-scope variables.
2. The encoder provides a streaming API (borrowed from YASON). The standard encode methods are re-implemented using that API.
3. The CLOS decoder has undergone serious modifications to address the issue of degrading performance in some Lisp implementations. It shall be possible to use anonymous “fluid” objects created by the decoder in a way which makes for a “poor man's JavaScript”. The CLOS encoder does not emit prototype metadata anymore (unless specifically requested).
4. I have written a new pair of matching functions to convert between camel case and Lisp names.
5. There is an improved exception signalling mechanism. Some things which previously were handled by callbacks now use conditions / restarts.
6. JSON-BIND now uses the event-based decoder API with dynamic customization. This has the advantage that the user may choose the semantics with which to decode interior values. However, I had to forego the support for Lisp trees as arguments to JSON-BIND.
7. Some exported names were changed: Condition JSON-PARSE-ERROR → JSON-SYNTAX-ERROR Macro WITH-LIST-DECODER-SEMANTICS → WITH-DECODER-SIMPLE-LIST-SEMANTICS Macro WITH-CLOS-DECODER-SEMANTICS → WITH-DECODER-SIMPLE-CLOS-SEMANTICS
I have made sure the tests pass in Clozure CL, SBCL, CMUCL, CLisp, and ECL.
There is a new User's Manual in doc/cl-json.html (I have also uploaded a copy to http://dpworks.net/smilga/cl-json.html). And here I would like to humbly ask someone on this mailing list who is a native speaker of English to proof-read it. Russian, my natural idiom, is quite unlike English in grammar, conventional style, and punctuation. Though I am more or less fluent in the latter language, I never fully trust myself, especially with the choice of articles, “which” vs. “that”, and the use of modal and causative constructions. Also, if you think some passages confused or inarticulate, by all means do not hesitate to point me to them. Thank you in advance.
As I suppose that it was the size of the attachment which has led to the previous message being blocked, I'm going to try to send it in a separate message, and if that fails the file can be downloaded from http://dpworks.net/smilga/cl-json-version-0.4.dpatch.gz.
Sincerely, - B. Smilga.
Thanks Boris and Hans for all this, I have just started to look at the new patches and it looks very impressive. I have pushed all pathes to the darcs repo, and added a tag "Pre-0.4-bundle" before and a tag "0.4-bundle" after.
One thing you might want to know before upgrading, when comparing the performance testcases the new version seems a bit slower on my sbcl (see below).
I'll look more into this.
Best wishes, Henrik Hjelte
SBCL 1.0.21.34
Encoder:
Pre-0.4-bundle: CL-USER> (5am:run! 'json-test::encoder-performance) Encoding 652 varying chars from memory 2000 times. Evaluation took: 0.160 seconds of real time 0.152009 seconds of total run time (0.144009 user, 0.008000 system) [ Run times consist of 0.008 seconds GC time, and 0.145 seconds non-GC time. ] 95.00% CPU 346,830,716 processor cycles 14,245,920 bytes consed
0.4-bundle:
CL-USER> (5am:run! 'json-test::encoder-performance) Encoding 652 varying chars from memory 2000 times. Evaluation took: 0.143 seconds of real time 0.144009 seconds of total run time (0.136008 user, 0.008001 system) [ Run times consist of 0.020 seconds GC time, and 0.125 seconds non-GC time. ] 100.70% CPU 309,824,182 processor cycles 26,578,824 bytes consed
Decoder:
Pre-0.4-bundle: CL-USER> (5am:run! 'json-test::decoder-performance) Decoding 1387 varying chars from memory 1000 times. Evaluation took: 0.667 seconds of real time 0.652041 seconds of total run time (0.632039 user, 0.020002 system) [ Run times consist of 0.024 seconds GC time, and 0.629 seconds non-GC time. ] 97.75% CPU 1,440,436,365 processor cycles 80,807,408 bytes consed
0.4-bundle: CL-USER> (5am:run! 'json-test::decoder-performance) Decoding 1387 varying chars from memory 1000 times. Evaluation took: 2.139 seconds of real time 2.108132 seconds of total run time (1.628102 user, 0.480030 system) [ Run times consist of 0.032 seconds GC time, and 2.077 seconds non-GC time. ] 98.55% CPU 4,624,819,100 processor cycles 105,295,912 bytes consed
On Tue, Apr 14, 2009 at 1:53 PM, Henrik Hjelte henrik@evahjelte.com wrote:
One thing you might want to know before upgrading, when comparing the performance testcases the new version seems a bit slower on my sbcl (see below).
To be sure, that's 200% deceleration, and it stays that way if the COUNT in the test is increased 10, 100, etc. times. “A bit” seems like an understatement here. I'm afraid that's the price we pay for dynamic customization, as most of the accrued run time is used up (prima facie) by the handler invocation machinery—which is there for every little char! Methinks there is an evident way to optimize this, and I'm going to try it out (tomorrow if I have time). ̈́ - B. Sm.
On Wed, Apr 15, 2009 at 8:55 PM, Boris Smilga boris.smilga@gmail.com wrote:
On Tue, Apr 14, 2009 at 1:53 PM, Henrik Hjelte henrik@evahjelte.com wrote:
One thing you might want to know before upgrading, when comparing the performance testcases the new version seems a bit slower on my sbcl (see below).
To be sure, that's 200% deceleration, and it stays that way if the COUNT in the test is increased 10, 100, etc. times. “A bit” seems like an understatement here. I'm afraid that's the price we pay for dynamic customization, as most of the accrued run time is used up (prima facie) by the handler invocation machinery—which is there for every little char! Methinks there is an evident way to optimize this, and I'm going to try it out (tomorrow if I have time).
Guess who is out in the left field again... Bypassing the handler mechanism does not, per se, gain much (my first impression was evidently wrong); moreover, checking whether to use the standard or the handler-free track actually impairs the performance.
However, I was able to pinpoint one factor which is responsible for about half of the performance regression (in the DECODER-PERFORMANCE test, the deceleration is down to ≈80% from ≈160% on Darwin/PPC, and to ≈120% from ≈190% on FreeBSD/i386). This factor was the use of vector accumulators in member handlers as opposed to list accumulators—the same thing that has caused test failures in SBCL 1.0.27. Ironically, it was intended as an optimization. (CLisp is the only implementation where it does work, and even in it the improvement is slight; in others, it is negative).
The attached patch altogether discards vector accumulators. I'll look if more can be done to enhance the performance.
- B. Sm.
Hello, I have now tried Boris vaccum-patch on sbcl 1.0.27.9, and it works (almost) perfectly. There was one failing testcase on the test json-number, but I believe it is because of a previous bug in sbcl that is now fixed.bugfix in sbcl. Boris, I have pushed a patch, you might want to take a look at it to see if I understood it right. Now all tests run OK.
And the performance is much improved, but still three times slower than before on decoding. But maybe that is a price worth paying? What do people think, should the current darcs version be released as 0.4, or should we wait on something?
Cheers, Henrik
Encoding 652 varying chars from memory 2000 times. Evaluation took: 0.163 seconds of real time 0.160010 seconds of total run time (0.124007 user, 0.036003 system) [ Run times consist of 0.032 seconds GC time, and 0.129 seconds non-GC time. ] 98.16% CPU 352,256,740 processor cycles 26,355,248 bytes consed
....................................................Decoding 1387 varying chars from memory 1000 times. Evaluation took: 1.875 seconds of real time 1.860117 seconds of total run time (1.320083 user, 0.540034 system) [ Run times consist of 0.104 seconds GC time, and 1.757 seconds non-GC time. ] 99.20% CPU 4,051,873,881 processor cycles 70,342,416 bytes consed
On 28 Apr 2009, at 01:48, Henrik Hjelte wrote:
Hello, I have now tried Boris vaccum-patch on sbcl 1.0.27.9, and it works (almost) perfectly. There was one failing testcase on the test json-number, but I believe it is because of a previous bug in sbcl that is now fixed.bugfix in sbcl. Boris, I have pushed a patch, you might want to take a look at it to see if I understood it right. [...]
It is more complex than just an old bug having been fixed: if I am not mistaken, that particular piece of behaviour in SBCL showed a difference between Darwin/PPC and FreeBSD/Intel, rather than a difference between versions. I am now trying to upgrade SBCL from 1.0.18 to 1.0.23 on the former platform, and, as soon as it is up, I will post a more informed follow-up on the issue.
And the performance is much improved, but still three times slower than before on decoding. But maybe that is a price worth paying?
Well, I have worked on the two versions with a profiler, and I think there is still much room for improvement. If you look into the table attached (which I have handcrafted from the reports of the deterministic profilers on 0.3 and on 0.4 — the design had changed a lot, so I had to juxtapose “blocks” of roughly comparable functionality), you would be able to see two most outstanding foci of regression: the string reader and the camel-case converter. The latter is actually fuller and more complex in 0.4, and probably deserves its 1 sec. extra runtime (I will be more than happy to see someone devise a lighter analogue though). The regression in the former, however, is completely gratuitious. I cannot quite see, at this moment, what the cause is, but, by all means, this thing should be brought back into the bounds of decency.
- B. Smilga.
On Wed, Apr 29, 2009 at 5:28 AM, Boris Smilga boris.smilga@gmail.com wrote:
It is more complex than just an old bug having been fixed: if I am not mistaken, that particular piece of behaviour in SBCL showed a difference between Darwin/PPC and FreeBSD/Intel, rather than a difference between versions. I am now trying to upgrade SBCL from 1.0.18 to 1.0.23 on the former platform, and, as soon as it is up, I will post a more informed follow-up on the issue.
Positive on that. In 1.0.23 under Darwin, I have (EXPT 10.0 444) => #.SB-EXT:SINGLE-FLOAT-POSITIVE-INFINITY; in 1.0.22 under FreeBSD, the same signals a FLOATING-POINT-OVERFLOW. It is an architecture issue rather than a version issue (and, for that matter, a known one—see https://bugs.launchpad.net/sbcl/+bug/309069).
Well, I have worked on the two versions with a profiler, and I think there is still much room for improvement. If you look into the table attached [...] , you would be able to see two most outstanding foci of regression: the string reader and the camel-case converter. [...] The regression in the former, however, is completely gratuitious. I cannot quite see, at this moment, what the cause is, but, by all means, this thing should be brought back into the bounds of decency.
Two things here:
1) READ-JSON-CHARS (more exactly, its helper READ-CHARS-UNTIL) in 0.3 used what can be called a “string-stream accumulator” to collect the decoded characters, by running (WITH-OUTPUT-TO-STREAM (OSTR) ... (WRITE-CHAR (FUNCALL CHAR-CONVERTER ...) OSTR)). It is much faster than vector or even list accumulation.
2) The escaped character lookup can be compiled from alist to a (CASE ...) construction by using a little metaprogramming trick. This gains some cycles.
I have tried to fix this, and it got me down to 90% regression on FreeBSD/i386 (SBCL 1.0.27), 65% on Darwin/PPC (SBCL 1.0.22). Not so bad as before, if I dare say. Henrik: would you also try it out in your setup?
I have also added a subsection on CLOS decoder security to the manual. Please find the patches attached.
Sincerely, - B. Smilga.
On Sat, May 2, 2009 at 3:31 PM, Boris Smilga boris.smilga@gmail.com wrote:
On Wed, Apr 29, 2009 at 5:28 AM, Boris Smilga boris.smilga@gmail.com wrote:
I have tried to fix this, and it got me down to 90% regression on FreeBSD/i386 (SBCL 1.0.27), 65% on Darwin/PPC (SBCL 1.0.22). Not so bad as before, if I dare say. Henrik: would you also try it out in your setup?
I have also added a subsection on CLOS decoder security to the manual. Please find the patches attached.
Applied. I haven't looked deeply, but I just ran the performance-decoder test and it took 1.4 to 1.7 secs vs. the old version 0.7 secs. My suggestion is that we release the new version as 0.4 but write in the docs somewhere that the 0.3 version can be used for situations where performance is very important.
Thanks, Henrik
-
Boris Smilga -
Henrik Hjelte