After trying to run a super-simple Ruby script in JRuby that ran fine through the MRI, I found myself trying to debug a JSON generator exception.
The Ruby script required a RubyGem that had its own dependencies on the json gem and Merb + DataMapper extlib gem. I thought that I could simply rebuild the gem to use json_pure and run things on JRuby. Little did I know...
First, here are the versions of everything involved:
- Ruby 1.8.6
- JRuby 1.1.4
- rubygems 1.2.0
- json 1.1.3
- json_pure 1.1.3
- extlib 0.9.6
Let's take a simple example of converting Time to JSON. My environment uses UTC as the timezone. Bear with me. The code samples are repetitive, but I had to go through them to figure things out.
[sourcecode="ruby"]
require 'rubygems'
require 'json'
p [ JSON.parser, JSON.generator ]
p Time.now.to_json
[/sourcecode]
We'll run it using the MRI and JRuby.
MRI
[sourcecode="ruby"]
[JSON::Ext::Parser, JSON::Ext::Generator]
"\"Fri Sep 12 20:20:06 +0000 2008\""
[/sourcecode]
JRuby
[sourcecode="ruby"]
[JSON::Pure::Parser, JSON::Pure::Generator]
"\"Fri Sep 12 20:20:55 +0000 2008\""
[/sourcecode]
Seems like a reasonable conversion and in the same format. JRuby is using json_pure as expected. Good, it's consistent. Note that the output is not in ISO format.
Now, let's specify the the json_pure gem.
[sourcecode="ruby"]
require 'rubygems'
require 'json/pure'
p [ JSON.parser, JSON.generator ]
p Time.now.to_json
[/sourcecode]
Run it through the MRI and JRuby.
MRI
[sourcecode="ruby"]
[JSON::Pure::Parser, JSON::Pure::Generator]
"\"Fri Sep 12 20:29:13 +0000 2008\""
[/sourcecode]
JRuby
[sourcecode="ruby"]
[JSON::Pure::Parser, JSON::Pure::Generator]
"\"Fri Sep 12 20:29:45 +0000 2008\""
[/sourcecode]
Again, everything is consistent. Good news again.
Since things are going well, let's try using extlib in the script so that the Time is in ISO format.
[sourcecode="ruby"]
require 'rubygems'
require 'extlib'
require 'json'
p [ JSON.parser, JSON.generator ]
p Time.now.to_json
[/sourcecode]
Run it through the MRI and JRuby.
MRI
[sourcecode="ruby"]
[JSON::Ext::Parser, JSON::Ext::Generator]
"\"2008-09-12T20:44:06+00:00\""
[/sourcecode]
JRuby
[sourcecode="ruby"]
[JSON::Pure::Parser, JSON::Pure::Generator]
"\"2008-09-12T20:44:31+00:00\""
[/sourcecode]
Excellent! Everything is working wonderfully. No problems at all!
At this point, let me do what I wanted to do in the first place and modify the script to be a basic variation of the problematic Ruby script that gave me a headache.
I'm going to place the time in a Hash, but I won't include extlib yet.
[sourcecode="ruby"]
require 'rubygems'
require 'json'
p [ JSON.parser, JSON.generator ]
p Time.now.to_json
h = {"created_on" => Time.now}
p h.to_json
[/sourcecode]
Run it through the MRI and JRuby.
MRI
[sourcecode="ruby"]
[JSON::Ext::Parser, JSON::Ext::Generator]
"\"Fri Sep 12 20:48:50 +0000 2008\""
"{\"created_on\":\"Fri Sep 12 20:48:50 +0000 2008\"}"
[/sourcecode]
JRuby
[sourcecode="ruby"]
[JSON::Pure::Parser, JSON::Pure::Generator]
"\"Fri Sep 12 20:49:26 +0000 2008\""
"{\"created_on\":\"Fri Sep 12 20:49:26 +0000 2008\"}"
[/sourcecode]
Fantastic! It's a useless script, but it's going places: The Time is in a Hash that is being converted to JSON.
Ok, let's add extlib so that we can get the JSON Time ISO format...and because our problematic Ruby script uses a particular gem (a very useful gem for the Ruby script) that happens to depend on extlib.
[sourcecode="ruby"]
require 'rubygems'
require 'extlib'
require 'json'
p [ JSON.parser, JSON.generator ]
p Time.now.to_json
h = {"created_on" => Time.now}
p h.to_json
[/sourcecode]
Run it through the MRI and JRuby.
MRI
[sourcecode="ruby"]
[JSON::Ext::Parser, JSON::Ext::Generator]
"\"2008-09-12T20:51:43+00:00\""
"{\"created_on\":\"2008-09-12T20:51:43+00:00\"}"
[/sourcecode]
JRuby
[sourcecode="ruby"]
[JSON::Pure::Parser, JSON::Pure::Generator]
"\"2008-09-12T20:52:02+00:00\""
/home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:251:in `to_json': wrong # of arguments(2 for 0) (ArgumentError)
from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:251:in `json_transform'
from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:245:in `each'
from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:245:in `map'
from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:245:in `json_transform'
from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:218:in `to_json'
from time_extlib_to_json.rb:9
[/sourcecode]
EH?! Now why in the world would it give me an error? Everything was fine up until this point. Why was it giving me a
"wrong # of arguments(2 for 0) (ArgumentError)" exception?
Well, I had a great time with JRuby debugging sessions looking at the
json_transform method in json_pure's
generator.rb.
Just like the exception said, I came to see that the Time.to_json method no longer accepted two arguments...even though the json_transform method was trying to pass them in through the
s << value.to_json(state, depth + 1) line. When the exception was thrown, the Time instance, aka
value, had a to_json method that took 0 arguments.
[sourcecode="ruby"]
def json_transform(state, depth)
delim = ','
delim json_transform method call in
generator.rb is in the
module Hash, and it's responsible for calling to_json with two arguments,
state and
depth + 1, to convert a Hash key's value.
[sourcecode="ruby"]
module JSON
module Pure
module Generator
module GeneratorMethods
module Hash
def json_transform(state, depth)
[/sourcecode]
Nice. So now what? Well, I tried requiring 'json/add/core' and running it under JRuby...
[sourcecode="ruby"]
require 'rubygems'
require 'extlib'
require 'json'
require 'json/add/core'
p [ JSON.parser, JSON.generator ]
p Time.now.to_json
h = {"created_on" => Time.now}
p h.to_json
[/sourcecode]
It works because 'json/add/core' gives Time.to_json a variable number of arguments again. But it gave this JSON output:
JRuby
[sourcecode="ruby"]
[JSON::Pure::Parser, JSON::Pure::Generator]
"{\"json_class\":\"Time\",\"s\":1221254881,\"n\":82427000}"
"{\"created_on\":{\"json_class\":\"Time\",\"s\":1221254881,\"n\":83467000}}"
[/sourcecode]
I'm not into that Time format at all!
If I require 'extlib' after 'json' and 'json/add/core', I get the same error of course.
Maybe I'll just drop using the gem that requires extlib and override Time.to_json myself...That doesn't seem right since that gem has other functionality that I need.
Right now, I feel like this is both an extlib and json_pure issue. extlib's Time.to_json doesn't accept a variable number of arguments, and the 'json/add/core' time format isn't what I was looking for.
Hmmm...
Update
Actually, I take back what I said about it being either an extlib or json_pure issue. This might just be a case of code coming together and clashing. :-(
Update 2
I guess eating helps clear the head! It looks like it's a json_pure issue since it happens on both JRuby and the MRI when the script requires 'json/pure'. Here are the parts of the code in the C extension where it differs from Ruby:
static VALUE mHash_to_json(int argc, VALUE *argv, VALUE self)
{
VALUE Vstate, Vdepth, result;
long depth;
rb_scan_args(argc, argv, "02", &Vstate, &Vdepth);
depth = NIL_P(Vdepth) ? 0 : FIX2LONG(Vdepth);
if (NIL_P(Vstate)) {
long len = RHASH(self)->tbl->num_entries;
result = rb_str_buf_new(len);
rb_str_buf_cat2(result, "{");
rb_hash_foreach(self, hash_to_json_i, result);
rb_str_buf_cat2(result, "}");
} else {
GET_STATE(Vstate);
check_max_nesting(state, depth);
if (state->check_circular) {
VALUE self_id = rb_obj_id(self);
if (RTEST(rb_hash_aref(state->seen, self_id))) {
rb_raise(eCircularDatastructure,
"circular data structures not supported!");
}
rb_hash_aset(state->seen, self_id, Qtrue);
result = mHash_json_transfrom(self, Vstate, LONG2FIX(depth));
rb_hash_delete(state->seen, self_id);
} else {
result = mHash_json_transfrom(self, Vstate, LONG2FIX(depth));
}
}
OBJ_INFECT(result, self);
return result;
}
static int hash_to_json_i(VALUE key, VALUE value, VALUE buf)
{
VALUE tmp;
if (key == Qundef) return ST_CONTINUE;
if (RSTRING_LEN(buf) > 1) rb_str_buf_cat2(buf, ",");
tmp = rb_funcall(rb_funcall(key, i_to_s, 0), i_to_json, 0);
Check_Type(tmp, T_STRING);
rb_str_buf_append(buf, tmp);
OBJ_INFECT(buf, tmp);
rb_str_buf_cat2(buf, ":");
tmp = rb_funcall(value, i_to_json, 0);
Check_Type(tmp, T_STRING);
rb_str_buf_append(buf, tmp);
OBJ_INFECT(buf, tmp);
return ST_CONTINUE;
}
Update 3
Follow-up post:
json_pure 1.1.3 patch
Feedback is welcome!
