[cl-json-devel] cl-json depends on lisp reader?
Hi everyone! I use cl-json with LispWorks 6.1. When delivering an app, I am trying to minimize the size of it by setting delivery level 5. The cl-json function "decode-json-from-string" does not seem to like this without explicitly keeping the Lisp Reader using the delivery keyword (:keep-lisp-reader). Since the Lisp Reader is unnecessary for the rest of the app, I would like to leave it out when delivering. Any thoughts on this? PS. I will also send a similar message to the LispWorks list, if that is more appropriate. Thanks! Sven Emtell
On Wed, Oct 31, 2012 at 5:07 PM, Sven Emtell <sven.emtell@doremir.com> wrote:
I use cl-json with LispWorks 6.1. When delivering an app, I am trying to minimize the size of it by setting delivery level 5. The cl-json function "decode-json-from-string" does not seem to like this without explicitly keeping the Lisp Reader using the delivery keyword (:keep-lisp-reader). Since the Lisp Reader is unnecessary for the rest of the app, I would like to leave it out when delivering. Any thoughts on this?
The Lisp reader is used on just one occasion, namely, in the function PARSE-NUMBER in src/decoder.lisp (line 417ff., four uses of READ-FROM- STRING). You can easily ditch it by supplying another parser for numbers. For this end, you have three options: A) wait for someone to develop a replacement—without guarantees; B) write a replacement yourself, send a patch to the maintainers, and wait for it to be pushed to the repository; C) customize DECODE-JSON in your own code, by setting or binding the handlers for number token. If you choose option C, I'd suggest you read the relevant chapter in the User Guide (http://common-lisp.net/project/cl-json/#DECODER-CUSTOMIZATION). The handlers in question are represented by the variables *INTEGER- HANDLER* and *REAL-HANDLER*. Sincerely, — B. Smilga.
Thanks Boris! I'll see what to do. Best regards, Sven 31 okt 2012 kl. 23:26 skrev Boris Smilga:
On Wed, Oct 31, 2012 at 5:07 PM, Sven Emtell <sven.emtell@doremir.com> wrote:
I use cl-json with LispWorks 6.1. When delivering an app, I am trying to minimize the size of it by setting delivery level 5. The cl-json function "decode-json-from-string" does not seem to like this without explicitly keeping the Lisp Reader using the delivery keyword (:keep-lisp-reader). Since the Lisp Reader is unnecessary for the rest of the app, I would like to leave it out when delivering. Any thoughts on this?
The Lisp reader is used on just one occasion, namely, in the function PARSE-NUMBER in src/decoder.lisp (line 417ff., four uses of READ-FROM-STRING). You can easily ditch it by supplying another parser for numbers. For this end, you have three options: A) wait for someone to develop a replacement—without guarantees; B) write a replacement yourself, send a patch to the maintainers, and wait for it to be pushed to the repository; C) customize DECODE-JSON in your own code, by setting or binding the handlers for number token. If you choose option C, I'd suggest you read the relevant chapter in the User Guide (http://common-lisp.net/project/cl-json/#DECODER-CUSTOMIZATION). The handlers in question are represented by the variables *INTEGER-HANDLER* and *REAL-HANDLER*.
Sincerely, — B. Smilga.
On 10/31/12 Oct 31 -5:26 PM, Boris Smilga wrote:
On Wed, Oct 31, 2012 at 5:07 PM, Sven Emtell <sven.emtell@doremir.com> wrote:
I use cl-json with LispWorks 6.1. When delivering an app, I am trying to minimize the size of it by setting delivery level 5. The cl-json function "decode-json-from-string" does not seem to like this without explicitly keeping the Lisp Reader using the delivery keyword (:keep-lisp-reader). Since the Lisp Reader is unnecessary for the rest of the app, I would like to leave it out when delivering. Any thoughts on this?
The Lisp reader is used on just one occasion, namely, in the function PARSE-NUMBER in src/decoder.lisp (line 417ff., four uses of READ-FROM-STRING). You can easily ditch it by supplying another parser for numbers. For this end, you have three options: A) wait for someone to develop a replacement—without guarantees; B) write a replacement yourself, send a patch to the maintainers, and wait for it to be pushed to the repository; C) customize DECODE-JSON in your own code, by setting or binding the handlers for number token. If you choose option C, I'd suggest you read the relevant chapter in the User Guide (http://common-lisp.net/project/cl-json/#DECODER-CUSTOMIZATION). The handlers in question are represented by the variables *INTEGER-HANDLER* and *REAL-HANDLER*.
Wouldn't it be possible to replace all uses of READ-FROM-STRING here with PARSE-INTEGER? We would have to change the structure here from "try READ-FROM-STRING and if that fails, parse by hand" to just "parse by hand," but I think that would be reasonable. If so, I suggest we simply do that.... I have a patch, but it could use some review and check on multiple lisp implementations. I'll try to get it out today. Best, Robert
OK, I have a patch along the lines of my previous email. I will try to convince darcs to let me pass it on. It passes tests on ACL 8.2 SBCL 1.1 CCL 1.8 clisp 2.49 ABCL 1.0.1 [but I had to fix ARNESI, see below, or the tests wouldn't run] It *FAILS* the tests on ECL, seemingly because of problems with floating point comparisons: JSON-NUMBER []: (DECODE-JSON-FROM-STRING "-2.3e3") evaluated to -2300.0002, which is not = to -2300.0.. -------------------------------- -------------------------------- JSON-NUMBER []: (DECODE-JSON-FROM-STRING "-3e4") evaluated to -30000.004, which is not = to -30000.0.. -------------------------------- -------------------------------- JSON-NUMBER []: (DECODE-JSON-FROM-STRING "3e4") evaluated to 30000.004, which is not = to 30000.0.. -------------------------------- -------------------------------- JSON-NUMBER []: (DECODE-JSON-FROM-STRING "2e40") evaluated to 2.0000000000000315e40, which is not = to 2.e40.. -------------------------------- -------------------------------- JSON-NUMBER []: (WITH-FP-OVERFLOW-HANDLER (INVOKE-RESTART 'BIGNUMBER-STRING "BIG:") (DECODE-JSON-FROM-STRING "2e444")) evaluated to #.ext::single-float-positive-infinity, which is not EQUALP to "BIG:2e444".. -------------------------------- -------------------------------- JSON-NUMBER []: Unexpected Error: #<a ARITHMETIC-ERROR> #<a ARITHMETIC-ERROR>.. I do not know how to fix these. I need to check and see if these tests pass under the current version of CL-JSON with ECL, or if the failures are independent of my modifications. * I was not able to test on ABCL because the matcher function definition in ARNESI (which is used by FiveAM), cannot be compiled in ABCL: (defstruct (match-state (:conc-name ||)) target bindings matched) Changing the :CONC-NAME to "" fixed the problem. I don't know if this is a bug in ABCL or ARNESI.
Just want to let you know that we now use cl-json 0.5.0 without having to keep the lisp reader when delivering. Thanks everyone! Sven 4 nov 2012 kl. 22:57 skrev Robert Goldman:
OK, I have a patch along the lines of my previous email. I will try to convince darcs to let me pass it on.
It passes tests on
ACL 8.2 SBCL 1.1 CCL 1.8 clisp 2.49 ABCL 1.0.1 [but I had to fix ARNESI, see below, or the tests wouldn't run]
It *FAILS* the tests on ECL, seemingly because of problems with floating point comparisons:
JSON-NUMBER []: (DECODE-JSON-FROM-STRING "-2.3e3") evaluated to -2300.0002, which is not = to -2300.0.. -------------------------------- -------------------------------- JSON-NUMBER []: (DECODE-JSON-FROM-STRING "-3e4") evaluated to -30000.004, which is not = to -30000.0.. -------------------------------- -------------------------------- JSON-NUMBER []: (DECODE-JSON-FROM-STRING "3e4") evaluated to 30000.004, which is not = to 30000.0.. -------------------------------- -------------------------------- JSON-NUMBER []: (DECODE-JSON-FROM-STRING "2e40") evaluated to 2.0000000000000315e40, which is not = to 2.e40.. -------------------------------- -------------------------------- JSON-NUMBER []: (WITH-FP-OVERFLOW-HANDLER (INVOKE-RESTART 'BIGNUMBER-STRING "BIG:") (DECODE-JSON-FROM-STRING "2e444")) evaluated to #.ext::single-float-positive-infinity, which is not EQUALP to "BIG:2e444".. -------------------------------- -------------------------------- JSON-NUMBER []: Unexpected Error: #<a ARITHMETIC-ERROR> #<a ARITHMETIC-ERROR>..
I do not know how to fix these.
I need to check and see if these tests pass under the current version of CL-JSON with ECL, or if the failures are independent of my modifications.
* I was not able to test on ABCL because the matcher function definition in ARNESI (which is used by FiveAM), cannot be compiled in ABCL:
(defstruct (match-state (:conc-name ||)) target bindings matched)
Changing the :CONC-NAME to "" fixed the problem. I don't know if this is a bug in ABCL or ARNESI.
_______________________________________________ cl-json-devel mailing list cl-json-devel@common-lisp.net http://lists.common-lisp.net/cgi-bin/mailman/listinfo/cl-json-devel
--- Sven Emtell, CTO, DoReMIR Music Research www.scorecleaner.com Check out the ScoreCleaner introductory video here
participants (3)
-
Boris Smilga -
Robert Goldman -
Sven Emtell