On Wed, Dec 30, 2009 at 4:37 PM, Robert Goldman rpgoldman@sift.info wrote:

I am setting up a JSON-RPC server over sockets, rather than HTTP, and have a couple of questions about json-rpc.lisp:

1. Am I correct in thinking that INVOKE-JSON does not handle

notifications?  It looks like it will always try to return a result. Question:  would it be reasonable to modify INVOKE-JSON to handle notifications by just executing the function and /not/ returning a result?

Yes, but how would the API change. An idea is to let the caller decide if he wants return values, maybe by exposing two functions.

2. My socket server must parse JSON messages and decide what type they

are before it can call invoke-json.  This suggests refactoring invoke-json into two stages:

a.  the part that calls the decoder and pulls out the three slots

b.  an inside function that accepts method, params, id, and does the actual invoking.

I was thinking of submitting a patch along those lines.  Please let me know if that seems wrong.

I agree with the problem. Also, I think a problem with the json-rpc implementation is that the decoder-encoder is harded to be the "guessing" encoder (and decoder). I have had an idea for how to make an new type of parser that could fix this. It is quite easy to do, but I don't have enough egoistic motives for coding it myself yet. I wrote it down below however, maybe you or someone else wants to go this way? But if you have another solution for the json-rpc problem you are facing I'll accept patches too, best against the latest darcs version.

/Henrik

Idea for a new json-parser:

Rationale: ---------

The current parser only parses a whole json document (json-object) completely. If the json-document is big and you are only interested in a part of it this is very inefficient. Also, maybe you want to use one type of parser for a part of the big json-document, another parser for another part and no parser at all for another part.

Sketch: ------

A new function parses the json-document into a DOM(document object model) like object. Only the minimal info is parsed in each step, you might want to say that the dom-nodes are read lazily.

Lets call this function "scan" (just an idea), and it takes a stream as input. Assume the stream now contains a json-object.

The minimal API we might want is a way to query the json-object with a key and get the value (as an unparsed string). We could start reading from the stream, pass the opening curly bracket and some whitespace, read the first key (parse it to a string). If it matches the key we expect, we can pass the trailing colon and then return the stream to the caller. Now the caller could parse the value in any way he wants from the stream.

If the first key didn't match what we looked for, we now need to advance the stream past the value. But we don't really need to "parse it". Assume the value is a complex nested JSON object. Since JSON is so nicely uniform designed, even in this case we only need to read the stream as fast as we can and just count the numbers of opening and closing brackets. We don't need to parse the actual contents. Almost at least. We will need to keep track of backslashed characters and also ignore curly brackets inside strings. Fortunately skipping strings is easy since they start and end with doublequoutes.

Advancing past arrays is done in the same way, just by counting brackets and skipping strings. Advancing past strings is even easier. Advancing past numbers or the keywords "true", "false" or "null" is done by skipping whitespace, then reading anything up to whitespace, a comma or closing curly bracket.

I think that while we are scanning the stream we should collect the characters in a string for later access. Because in most cases we want to get the first level of the whole json-document, then do more processing with some of the values.

A minimal API could be like this:

scan-opening-bracket(stream): Skip and opening bracket so we can start using scan-key.

scan-key(stream): returns a string value for the key part of a JSON-object being read from stream. Also advances the stream past the trailing whitespace, colon, whitespace.

Returns nil when we are at the closing bracket (no more keys).

An option could be :get-whitespace and the function would then also return the leading and trailing whitespace as second and third return values. (If you want to build up an "exact" copy of the input for some reason).

scan-value(stream): returns a string value for the value.

scan-to-next-key(stream). To be called after scan-value. passes whitespace, comma and whitespace to the next key. Returns t. If there is no next key(we are at the closing curly-bracket), returns nil.

Now, this small API should be sufficient to build a little dom-like parser on top of. An easy version could be a function that reads all keys and unparsed values and preserves them in hash-table like structure.