rewrite > blog @ idearise.com http://blog.idearise.com Most recent posts at rewrite > blog @ idearise.com posterous.com Mon, 23 Nov 2009 07:20:00 -0800 paperclip processors and after_post_process http://blog.idearise.com/paperclip-processors-and-afterpostprocess http://blog.idearise.com/paperclip-processors-and-afterpostprocess

A few things to note when trying to process an uploaded file with paperclip:

  • The only way a post-processor will run is if a style is defined

class MyModel < ActiveRecord::Base

has_attached_file :mp3, :url => '/:class/:id/:style.:extension', :path => ':rails_root/assets/:class/:id_partition/:style.:extension', :whiny => true, :styles => { :text => { :do_something => true } }, :processors => [:my_processor]

# ...

end

  • To access the file after processing but before paperclip is finished, use the :after_process callback -- model instance attributes can be modified and will be saved automatically as part of the paperclip process

class MyModel < ActiveRecord::Base

# ...

after_post_process :do_one_more_thing

# ...

private

def do_one_more_thing

logger.info "MyModel [paperclip] after_post_process"

logger.info "MyModel [paperclip] file #{mp3.queued_for_write[:original].path}"

# Do more with the file

# Make model attribute changes as needed

# self.my_attribute = "x"

end

end

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Thu, 12 Nov 2009 04:35:13 -0800 rails log rotation http://blog.idearise.com/rails-log-rotation http://blog.idearise.com/rails-log-rotation No need to restart by following this post: http://www.hostingrails.com/401/What-is-the-best-practice-for-rails-log-rotat... #!/bin/sh mkdir /mnt/app/releases/mysite.com/log/archive -p tar czf "/mnt/app/releases/mysite.com/log/archive/`date +%Y%m%d`_production.tar.gz" -C / mnt/app/releases/mysite.com/log/production.log cat /dev/null > /mnt/app/releases/mysite.com/log/production.log find /mnt/app/releases/mysite.com/log/archive/* -mtime +180 -exec rm {} \;

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Wed, 11 Nov 2009 03:44:15 -0800 Go http://blog.idearise.com/go http://blog.idearise.com/go http://golang.org/ [youtube=http://www.youtube.com/watch?v=wwoWei-GAPo] http://arstechnica.com/open-source/news/2009/11/go-new-open-source-programming-language-from-google.ars

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Tue, 10 Nov 2009 04:59:36 -0800 Google Wave http://blog.idearise.com/google-wave http://blog.idearise.com/google-wave I wish they had full iPhone Safari support. Reloading/logging back in to see new waves #ugh

Posted via email from roberto

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Sun, 01 Feb 2009 02:50:47 -0800 "Roxanne" by Songsmith and The Police http://blog.idearise.com/roxanne-by-songsmith-and-the-police http://blog.idearise.com/roxanne-by-songsmith-and-the-police Another off-topic post...but when Microsoft Songsmith does this to a song, I need to share it: [youtube=http://www.youtube.com/watch?v=ypycpKQxXR0]

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Fri, 09 Jan 2009 21:03:37 -0800 Happy with... http://blog.idearise.com/happy-with http://blog.idearise.com/happy-with Nothing earth-shattering in this post...just wanted to note my good experiences with:
  • formtastic: I had looked into the air_budd_form_builder and RubyPond's semantic_form_builder but moved on to formtastic after liking its concise approach to building forms.
  • suspenders: Not too heavy for a Rails starter app with lots of nice development plugins from the thoughtbot folks.
  • paperclip: Really easy file attachments for ActiveRecord.

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Mon, 10 Nov 2008 10:19:49 -0800 Installing MySQL 5.1.29-RC & Sphinx SE 0.9.8.1 on OS X Leopard http://blog.idearise.com/installing-mysql-5129-rc-sphinx-se-0981-on-os http://blog.idearise.com/installing-mysql-5129-rc-sphinx-se-0981-on-os Sphinx 0.9.8.1 was recently released, and I still didn't have MySQL re-installed on OS X. Therefore, it was a good time to install both Sphinx and MySQL. I prefer using the SphinxSE MySQL storage engine in conjunction with Sphinx since I'm comfortable with SQL and like being able to join directly with the SphinxSE tables in queries. SphinxSE also provides other benefits when developing an app with an ORM. For this task, I followed these useful posts: You can setup your path according to Dan Benjamin's post under "Setting the Path" by adding the following to your .bash_profile, .bash_login, or .profile: 
export PATH="/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin:$PATH"

Following another Dan Benjamin post under "Setting Up", make a src directory for storing the source code: 
sudo mkdir /usr/local/src
sudo chgrp admin /usr/local/src
sudo chmod -R 775 /usr/local/src

Download and extract the MySQL 5.1.29-RC source and Sphinx 0.9.8.1 source: 
cd /usr/local/src
mv ~/Downloads/mysql-5.1.29-rc.tar .
chgrp admin mysql-5.1.29-rc.tar
tar -xf mysql-5.1.29-rc.tar
curl -C - -O http://sphinxsearch.com/downloads/sphinx-0.9.8.1.tar.gz
tar -xvf sphinx-0.9.8.1.tar.gz

Make a sphinx directory in the MySQL storage directory: 
cd mysql-5.1.29-rc
mkdir storage/sphinx

Copy the Sphinx sphinxse files to mysql storage/sphinx directory: 
cp -R ../sphinx-0.9.8.1/mysqlse/ storage/sphinx

Run autorun.sh: 
sudo sh BUILD/autorun.sh

Run configure with the Sphinx plugin and InnoDB options: 
sudo ./configure --prefix=/usr/local/mysql \
--with-charset=utf8 --with-collation=utf8_general_ci --with-extra-charsets=complex \
--enable-thread-safe-client --enable-local-infile \
--with-plugins=sphinx --with-innodb

Run make and make install: 
sudo make
sudo make install

Copy one of the sample MySQL my.cnf files found in /usr/local/mysql/share/mysql and edit it on your desktop as needed, paying close attention to the socket file and tmpdir (I've seen many installation issues caused by an unintended value in my.cnf): 
[client]
socket                = /tmp/mysql.sock

[mysqld]
socket                = /tmp/mysql.sock
tmpdir                = /tmp

Copy your my.cnf to /etc/my.cnf: 
sudo cp ~/Desktop/my.cnf /etc/my.cnf

Change the mysql directory ownership: 
cd /usr/local/mysql
sudo chown -R mysql .
sudo chgrp -R mysql .

Take a MySQL install tip from Mike, and edit bin/mysql_install_db to change hostname='localhost' so that you won't have strange "Access denied" for localhost issues: 
# Try to determine the hostname
#hostname=`/bin/hostname`
hostname='localhost'

Install the system tables, and change the file & directory ownership again: 
sudo ./bin/mysql_install_db --user=mysql
sudo chown -R root .
sudo chown -R mysql var

Now to start MySQL on boot, create a file called com.mysql.mysqld.plist on your Desktop as provided by Dan under "Auto-Starting MySQL": [code="xml"] KeepAlive Label com.mysql.mysqld Program /usr/local/mysql/bin/mysqld_safe RunAtLoad UserName mysql WorkingDirectory /usr/local/mysql [/code] Move the plist file to the LaunchDaemons directory, and change its owner: 
sudo mv ~/Desktop/com.mysql.mysqld.plist /Library/LaunchDaemons
sudo chown root /Library/LaunchDaemons/com.mysql.mysqld.plist

Start MySQL: 
sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysqld.plist

Check that MySQL is running: 
ps -ef | grep mysql
   74 19729     1   0   0:00.02 ??         0:00.03 /bin/sh /usr/local/mysql/bin/mysqld_safe
   74 19868 19729   0   0:00.16 ??         0:00.39 /usr/local/mysql/libexec/mysqld 
--basedir=/usr/local/mysql --datadir=/usr/local/mysql/var 
--log-error=/usr/local/mysql/var/Al.local.err --pid-file=/usr/local/mysql/var/Al.local.pid 
--socket=/tmp/mysql.sock --port=3306
  503 19870   457   0   0:00.00 ttys003    0:00.00 grep mysql

Note: It may take some time for your InnoDB data files to be created if you are not using a file per table (innodb_file_per_table) and if you specified large InnoDB data file sizes. You can always check the var directory for the InnoDB data file sizes as the files are being created: 
sudo ls -al var

Finally, login to MySQL, and check for the Sphinx storage engine: 
Al:mysql-5.1.29-rc roberto$ mysql -u root
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.1.29-rc Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> show engines;
+------------+---------+------------------------------------------------------------+--------------+-----+------------+
| Engine     | Support | Comment                                                    | Transactions | XA  | Savepoints |
+------------+---------+------------------------------------------------------------+--------------+-----+------------+
| CSV        | YES     | CSV storage engine                                         | NO           | NO  | NO         | 
| MRG_MYISAM | YES     | Collection of identical MyISAM tables                      | NO           | NO  | NO         | 
| MEMORY     | YES     | Hash based, stored in memory, useful for temporary tables  | NO           | NO  | NO         | 
| InnoDB     | YES     | Supports transactions, row-level locking, and foreign keys | YES          | YES | YES        | 
| MyISAM     | DEFAULT | Default engine as of MySQL 3.23 with great performance     | NO           | NO  | NO         | 
| SPHINX     | YES     | Sphinx storage engine 0.9.8                                | NO           | NO  | NO         | 
+------------+---------+------------------------------------------------------------+--------------+-----+------------+
6 rows in set (0.00 sec)

mysql>

Wrap up by installing Sphinx: 
cd /usr/local/src/sphinx-0.9.8.1
sudo ./configure --with-mysql
sudo make
sudo make install

You can check to see that searchd and indexer are installed by executing the commands and seeing the failure response and help output: 
Al:sphinx-0.9.8.1 roberto$ searchd
Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

FATAL: no readable config file (looked in /usr/local/etc/sphinx.conf, ./sphinx.conf).
Al:sphinx-0.9.8.1 roberto$ indexer
Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

Usage: indexer [OPTIONS] [indexname1 [indexname2 [...]]]

Options are:
--config                 read configuration from specified file
                        (default is sphinx.conf)
--all                        reindex all configured indexes
--quiet                        be quiet, only print errors
--noprogress                do not display progress
                        (automatically on if output is not to a tty)
--rotate                send SIGHUP to searchd when indexing is over
                        to rotate updated indexes automatically
--buildstops  
                        build top N stopwords and write them to given file
--buildfreqs                store words frequencies to output.txt
                        (used with --buildstops only)
--merge  
                        merge 'src-index' into 'dst-index'
                        'dst-index' will receive merge result
                        'src-index' will not be modified
--merge-dst-range   
                        filter 'dst-index' on merge, keep only those documents
                        where 'attr' is between 'min' and 'max' (inclusive)

Examples:
indexer --quiet myidx1        reindex 'myidx1' defined in 'sphinx.conf'
indexer --all                reindex all indexes defined in 'sphinx.conf'
Al:sphinx-0.9.8.1 roberto$

All finished! :-)

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Thu, 06 Nov 2008 01:37:43 -0800 Tip: ExtJS TabPanel Renders Twice in Internet Explorer http://blog.idearise.com/tip-extjs-tabpanel-renders-twice-in-internet http://blog.idearise.com/tip-extjs-tabpanel-renders-twice-in-internet I just spent a few minutes reading and debugging some JavaScript code that renders an ExtJS TabPanel... Oddly, IE 6 and IE 7 rendered the TabPanel twice, including the tabs. It turned out to be the name of the JavaScript variable used for the TabPanel: [code="javascript"] tabs = new Ext.TabPanel({ [/code] Making the minor name change fixed it: [code="javascript"] huh = new Ext.TabPanel({ [/code]

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Thu, 30 Oct 2008 17:49:49 -0700 Net::Flickr::People#find_by_user_id http://blog.idearise.com/netflickrpeoplefindbyuserid http://blog.idearise.com/netflickrpeoplefindbyuserid ...a short, random post... It's the end of the month, and I need to meet my self-imposed quota so that the archives have a link to October 2008. ;-) After looking around at the different Ruby Flickr libraries, Net::Flickr seems to be the most active today. It's undergoing some major refactoring for its next release. The current release is tagged 0.0.1 and Pre-Alpha. I think it was tagged this way to give people a warm fuzzy feeling when they use it on production apps. :-) Anyway, 0.0.1 Pre-Alpha needs a lookup to find a person by user_id. There's code in Net::Flickr::Person that does a lookup by user_id, but you need to know the person's email address or username. 0.0.1 Pre-Alpha Net::Flickr::People#find_by_user_id [sourcecode="ruby"] module Net; class Flickr class People # Looks up a Flickr user based on their user_id. # # See http://flickr.com/services/api/flickr.people.getInfo.html for # details. def find_by_user_id(user_id, args = {}) args['user_id'] = user_id response = @flickr.request('flickr.people.getInfo', args) Person.new(@flickr, response.at('person')) end end end; end [/sourcecode] Based on the current code in SVN, the next release might need something like this: [sourcecode="ruby"] module Net; class Flickr class People # Looks up a Flickr user based on their user_id. # # See http://flickr.com/services/api/flickr.people.getInfo.html for # details. def find_by_user_id(user_id, args = {}) args['user_id'] = user_id response = Net::Flickr.instance().request('flickr.people.getInfo', args) Person.new(response.at('person')) end end end; end [/sourcecode]

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Wed, 17 Sep 2008 18:36:33 -0700 json_pure 1.1.3 patch http://blog.idearise.com/jsonpure-113-patch http://blog.idearise.com/jsonpure-113-patch Following up on a previous blog post about a JSON transformation issue, there is now a patched version of the json_pure 1.1.3 gem on GitHub. The issue had to do with transforming Hashes and Arrays to JSON when they contain objects (classes) that have their own to_json methods. A typical error message is: 
lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:251:in 
`to_json': wrong # of arguments(2 for 0) (ArgumentError)
 See the README file and the commit history for generator.rb for more info. No warranties or guarantees that this will work for other users and uses of the json_pure gem! See the GPL license. Feedback is welcome!

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Fri, 12 Sep 2008 21:52:17 -0700 Ruby and JRuby JSON Time with json, json_pure, and Merb + DataMapper extlib http://blog.idearise.com/ruby-and-jruby-json-time-with-json-jsonpure-a http://blog.idearise.com/ruby-and-jruby-json-time-with-json-jsonpure-a After trying to run a super-simple Ruby script in JRuby that ran fine through the MRI, I found myself trying to debug a JSON generator exception. The Ruby script required a RubyGem that had its own dependencies on the json gem and Merb + DataMapper extlib gem. I thought that I could simply rebuild the gem to use json_pure and run things on JRuby. Little did I know... First, here are the versions of everything involved:
  • Ruby 1.8.6
  • JRuby 1.1.4
  • rubygems 1.2.0
  • json 1.1.3
  • json_pure 1.1.3
  • extlib 0.9.6
Let's take a simple example of converting Time to JSON. My environment uses UTC as the timezone. Bear with me. The code samples are repetitive, but I had to go through them to figure things out. [sourcecode="ruby"] require 'rubygems' require 'json' p [ JSON.parser, JSON.generator ] p Time.now.to_json [/sourcecode] We'll run it using the MRI and JRuby. MRI [sourcecode="ruby"] [JSON::Ext::Parser, JSON::Ext::Generator] "\"Fri Sep 12 20:20:06 +0000 2008\"" [/sourcecode] JRuby [sourcecode="ruby"] [JSON::Pure::Parser, JSON::Pure::Generator] "\"Fri Sep 12 20:20:55 +0000 2008\"" [/sourcecode] Seems like a reasonable conversion and in the same format. JRuby is using json_pure as expected. Good, it's consistent. Note that the output is not in ISO format. Now, let's specify the the json_pure gem. [sourcecode="ruby"] require 'rubygems' require 'json/pure' p [ JSON.parser, JSON.generator ] p Time.now.to_json [/sourcecode] Run it through the MRI and JRuby. MRI [sourcecode="ruby"] [JSON::Pure::Parser, JSON::Pure::Generator] "\"Fri Sep 12 20:29:13 +0000 2008\"" [/sourcecode] JRuby [sourcecode="ruby"] [JSON::Pure::Parser, JSON::Pure::Generator] "\"Fri Sep 12 20:29:45 +0000 2008\"" [/sourcecode] Again, everything is consistent. Good news again. Since things are going well, let's try using extlib in the script so that the Time is in ISO format. [sourcecode="ruby"] require 'rubygems' require 'extlib' require 'json' p [ JSON.parser, JSON.generator ] p Time.now.to_json [/sourcecode] Run it through the MRI and JRuby. MRI [sourcecode="ruby"] [JSON::Ext::Parser, JSON::Ext::Generator] "\"2008-09-12T20:44:06+00:00\"" [/sourcecode] JRuby [sourcecode="ruby"] [JSON::Pure::Parser, JSON::Pure::Generator] "\"2008-09-12T20:44:31+00:00\"" [/sourcecode] Excellent! Everything is working wonderfully. No problems at all! At this point, let me do what I wanted to do in the first place and modify the script to be a basic variation of the problematic Ruby script that gave me a headache. I'm going to place the time in a Hash, but I won't include extlib yet. [sourcecode="ruby"] require 'rubygems' require 'json' p [ JSON.parser, JSON.generator ] p Time.now.to_json h = {"created_on" => Time.now} p h.to_json [/sourcecode] Run it through the MRI and JRuby. MRI [sourcecode="ruby"] [JSON::Ext::Parser, JSON::Ext::Generator] "\"Fri Sep 12 20:48:50 +0000 2008\"" "{\"created_on\":\"Fri Sep 12 20:48:50 +0000 2008\"}" [/sourcecode] JRuby [sourcecode="ruby"] [JSON::Pure::Parser, JSON::Pure::Generator] "\"Fri Sep 12 20:49:26 +0000 2008\"" "{\"created_on\":\"Fri Sep 12 20:49:26 +0000 2008\"}" [/sourcecode] Fantastic! It's a useless script, but it's going places: The Time is in a Hash that is being converted to JSON. Ok, let's add extlib so that we can get the JSON Time ISO format...and because our problematic Ruby script uses a particular gem (a very useful gem for the Ruby script) that happens to depend on extlib. [sourcecode="ruby"] require 'rubygems' require 'extlib' require 'json' p [ JSON.parser, JSON.generator ] p Time.now.to_json h = {"created_on" => Time.now} p h.to_json [/sourcecode] Run it through the MRI and JRuby. MRI [sourcecode="ruby"] [JSON::Ext::Parser, JSON::Ext::Generator] "\"2008-09-12T20:51:43+00:00\"" "{\"created_on\":\"2008-09-12T20:51:43+00:00\"}" [/sourcecode] JRuby [sourcecode="ruby"] [JSON::Pure::Parser, JSON::Pure::Generator] "\"2008-09-12T20:52:02+00:00\"" /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:251:in `to_json': wrong # of arguments(2 for 0) (ArgumentError) from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:251:in `json_transform' from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:245:in `each' from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:245:in `map' from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:245:in `json_transform' from /home/share/storage/jruby-1.1.4/lib/ruby/gems/1.8/gems/json_pure-1.1.3/lib/json/pure/generator.rb:218:in `to_json' from time_extlib_to_json.rb:9 [/sourcecode] EH?! Now why in the world would it give me an error? Everything was fine up until this point. Why was it giving me a "wrong # of arguments(2 for 0) (ArgumentError)" exception? Well, I had a great time with JRuby debugging sessions looking at the json_transform method in json_pure's generator.rb. Just like the exception said, I came to see that the Time.to_json method no longer accepted two arguments...even though the json_transform method was trying to pass them in through the s << value.to_json(state, depth + 1) line. When the exception was thrown, the Time instance, aka value, had a to_json method that took 0 arguments. [sourcecode="ruby"] def json_transform(state, depth) delim = ',' delim json_transform method call in generator.rb is in the module Hash, and it's responsible for calling to_json with two arguments, state and depth + 1, to convert a Hash key's value. [sourcecode="ruby"] module JSON module Pure module Generator module GeneratorMethods module Hash def json_transform(state, depth) [/sourcecode] Nice. So now what? Well, I tried requiring 'json/add/core' and running it under JRuby... [sourcecode="ruby"] require 'rubygems' require 'extlib' require 'json' require 'json/add/core' p [ JSON.parser, JSON.generator ] p Time.now.to_json h = {"created_on" => Time.now} p h.to_json [/sourcecode] It works because 'json/add/core' gives Time.to_json a variable number of arguments again. But it gave this JSON output: JRuby [sourcecode="ruby"] [JSON::Pure::Parser, JSON::Pure::Generator] "{\"json_class\":\"Time\",\"s\":1221254881,\"n\":82427000}" "{\"created_on\":{\"json_class\":\"Time\",\"s\":1221254881,\"n\":83467000}}" [/sourcecode] I'm not into that Time format at all! If I require 'extlib' after 'json' and 'json/add/core', I get the same error of course. Maybe I'll just drop using the gem that requires extlib and override Time.to_json myself...That doesn't seem right since that gem has other functionality that I need. Right now, I feel like this is both an extlib and json_pure issue. extlib's Time.to_json doesn't accept a variable number of arguments, and the 'json/add/core' time format isn't what I was looking for. Hmmm...

Update

Actually, I take back what I said about it being either an extlib or json_pure issue. This might just be a case of code coming together and clashing. :-(

Update 2

I guess eating helps clear the head! It looks like it's a json_pure issue since it happens on both JRuby and the MRI when the script requires 'json/pure'. Here are the parts of the code in the C extension where it differs from Ruby: 
static VALUE mHash_to_json(int argc, VALUE *argv, VALUE self)
{
    VALUE Vstate, Vdepth, result;
    long depth;

    rb_scan_args(argc, argv, "02", &Vstate, &Vdepth);
    depth = NIL_P(Vdepth) ? 0 : FIX2LONG(Vdepth);
    if (NIL_P(Vstate)) {
        long len = RHASH(self)->tbl->num_entries;
        result = rb_str_buf_new(len);
        rb_str_buf_cat2(result, "{");
        rb_hash_foreach(self, hash_to_json_i, result);
        rb_str_buf_cat2(result, "}");
    } else {
        GET_STATE(Vstate);
        check_max_nesting(state, depth);
        if (state->check_circular) {
            VALUE self_id = rb_obj_id(self);
            if (RTEST(rb_hash_aref(state->seen, self_id))) {
                rb_raise(eCircularDatastructure,
                        "circular data structures not supported!");
            }
            rb_hash_aset(state->seen, self_id, Qtrue);
            result = mHash_json_transfrom(self, Vstate, LONG2FIX(depth));
            rb_hash_delete(state->seen, self_id);
        } else {
            result = mHash_json_transfrom(self, Vstate, LONG2FIX(depth));
        }
    }
    OBJ_INFECT(result, self);
    return result;
}

static int hash_to_json_i(VALUE key, VALUE value, VALUE buf)
{
    VALUE tmp;

    if (key == Qundef) return ST_CONTINUE;
    if (RSTRING_LEN(buf) > 1) rb_str_buf_cat2(buf, ",");
    tmp = rb_funcall(rb_funcall(key, i_to_s, 0), i_to_json, 0);
    Check_Type(tmp, T_STRING);
    rb_str_buf_append(buf, tmp);
    OBJ_INFECT(buf, tmp);
    rb_str_buf_cat2(buf, ":");
    tmp = rb_funcall(value, i_to_json, 0);
    Check_Type(tmp, T_STRING);
    rb_str_buf_append(buf, tmp);
    OBJ_INFECT(buf, tmp);

    return ST_CONTINUE;
}

Update 3

Follow-up post: json_pure 1.1.3 patch Feedback is welcome!

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Mon, 08 Sep 2008 00:08:23 -0700 CouchDB 0.8.1 Simple One-Way Replication http://blog.idearise.com/couchdb-081-simple-one-way-replication http://blog.idearise.com/couchdb-081-simple-one-way-replication CouchDB 0.8.1 has been available for a few weeks, so I decided to see how it was coming along. For no real reason at all, I wanted to get a very simple replication process going between two CouchDB databases hosted on different machines. When I say "simple", I mean "don't get all excited because this is not a post about replicating all the documents reviewed in a Big Tobacco lawsuit!" Not that you would get excited about that either. I just wanted to see a few documents show up in another database on another machine. Simple enough. The CouchDB website, wiki, and mailing list archives were actually pretty good for getting started.

Easy Install

Thankfully, CouchDB has some pretty good installation information on its wiki and in its source distribution README file. In a short amount of time, I had a working CouchDB server on Ubuntu and OS X.

Ubuntu Hardy Heron 8.04 build and install

The CouchDB wiki page for Ubuntu contains links to the two following blog posts. I followed the Barking Iguana instructions. http://barkingiguana.com/2008/06/28/installing-couchdb-080-on-ubuntu-804 http://www.chetanmittal.com/2008/6/15/install-couchdb-on-ubuntu-hardy-heron-8-04 Notes: The Barking Iguana update-rc.d step and the init.d copy step should be switched. Also, I would add the couchdb user as a system user.  The CouchDB source distribution README file has this step in its instructions, but the home directory should be /usr/local/var/lib/couchdb. sudo adduser --system --home /usr/local/var/lib/couchdb --no-create-home \ --shell /bin/bash --group --gecos "CouchDB Administrator" couchdb To allow CouchDB to listen for external requests, modify the BindAddress in the /usr/local/etc/couchdb/couch.ini file, and update any firewall settings as needed: ;original ;BindAddress=127.0.0.1 ;modified BindAddress=0.0.0.0 Bind the CouchDB server to an externally available IP address or to 0.0.0.0.

OS X Leopard binary

http://jan.prima.de/~jan/plok/archives/142-CouchDBX-Revival.html Thanks to Jan L, one of the Apache CouchDB committers, CouchDBX is the easiest way to get a CouchDB server running on OS X. To allow CouchDB to listen for external requests, view the CouchDBX.app package contents, edit the /Contents/Resources/CouchDb/etc/couchdb/couch.ini file, and update the OS X firewall as needed.

Replicating 3 Documents

CouchDB comes with the Futon Utility Client, a browser-based user interface to setup databases, add/update/delete documents, run a CouchDB test suite, and manually initiate replication through its "Replicator" page. The Replicator tool offers replication from a source to target database, hosted locally or remotely available through HTTP. Pretty straightforward to use. Strangely, error messages are displayed in JSON format through a JavaScript alert, but it's not a big deal. So, at this point, I had two CouchDB servers on two different machines. I created a database on each server by using the Futon client and used the same name for both databases. I then proceeded to add 3 documents with randomly added fields and values into my designated "source" database. Now that I had servers, databases, and documents, it was time to figure out how to replicate the database automatically... Lo-and-behold, I found out that CouchDB currently doesn't offer "automated" replication without writing some code. Ok, fair enough. To get things started, I modified the couch.ini file by adding a line for DbUpdateNotificationProcess. More information can be found on the CouchDB wiki page for updating document views. DbUpdateNotificationProcess=/usr/local/var/lib/couchdb/potatoe.rb Why is the information on a page about updating document views? Well, the notification hook can be used to update views before a user actually queries those views. The page contains an example Ruby script that does just that. In addition, the same notification hook can be used for kicking off full-text indexing. But I had simpler (useless) needs. CouchDB will take care of starting the process and will output (stdout) a short JSON message to notify the process each time a local database is updated. Make sure that the couchdb user has the proper privileges to run the script/executable. [sourcecode="js"] # example database update notification JSON message {"type":"updated","db":"mytestdb"} [/sourcecode]Anyway, here's the (simple || stupid) script that got one-way replication going between two databases containing 3 documents hosted on 2 different machines sitting 3 feet apart. It uses the RestClient gem. [sourcecode="ruby"] #!/usr/bin/ruby require 'rubygems' require 'logger' require 'json' # sudo gem install json #require 'json/pure' # sudo gem install json_pure require 'rest_client' # sudo gem install rest-client logger = Logger.new('/usr/local/var/log/couchdb/potatoe.log', 3, 1024000) logger.level = Logger::INFO REPLICATE = "http://192.168.0.4:5984/_replicate" SOURCE = "http://192.168.0.4:5984/mytestdb" TARGET = "http://192.168.0.2:5984/mytestdb" begin logger.info "Ready for CouchDB..." replicate = RestClient::Resource.new REPLICATE replicationMsg = {:source => SOURCE, :target => TARGET}.to_json loop do unless (jsonOut = gets).nil? logger.debug jsonOut message = JSON.parse jsonOut if message["type"] == "updated" and message["db"] == "mytestdb" logger.info "'#{message['db']}' database updated." logger.info "Replicating..." response = replicate.post replicationMsg, :content_type => 'application/json' logger.debug response results = JSON.parse response if results["ok"] logger.info "Replication succeeded. " + "session_id: #{results['session_id']} " + "source_last_seq: #{results['source_last_seq']}" else # Currently, CouchDB 0.8.1 doesn't work this way. # It returns an HTTP 500 error instead of false. logger.info "Replication error: #{results}" end end else logger.info "CouchDB has gone away..." break end end rescue Exception => e logger.error "Error message: #{e.message}" logger.error "Stack trace: #{e.backtrace.inspect}" ensure logger.close end [/sourcecode]Basically, the script makes sure that it cares about the database that was updated and POSTs a JSON string to the CouchDB server's _replicate "resource" (I know...calling it a resource doesn't make sense). This initiates the replication process between the SOURCE and TARGET databases. The script doesn't stagger the updates -- it starts the replication process after every database update. This means that replication will occur each time a document is updated in the source database. And it doesn't do anything if the target database is unavailable...fault-tolerance-shmolerance! With the script in place, permissions all set, and the CouchDB server restarted, all I had to do was update a document in the source database to initiate the replication process.

That's it.

Yes, that's it. Three documents were replicated on the other machine. Seriously, it was everything I had hoped for, and more. :-) I'm sure other people are going to replicate millions of documents across thousands of CouchDB servers and databases on all types of hardware and networks. With documents containing a ridiculous number of fields and content. And maybe using another CouchDB database to store a list of databases to replicate, and replicating that database as well. And obviously all in Erlang too...but not me, at least not today. Notes:
  • The _replicate response message contains the "ok" result as well as a history of replication events. I found out through the CouchDB mailing that the response will contain information about the last 50 replication events. However, I noticed that the returned events were unique to the client that initiated the replication. I didn't see the same history when I kicked off replication through a Ruby irb session, the DbUpdateNotificationProcess script, or through the Futon client.
  • You can also do a hot-backup copy of the database files if you don't need "replication" like this and if your database files are small.

Update

I placed a JRuby script on GitHub that does batch and timed replications: Jpotatoe: A simple JRuby script for one-way CouchDB 0.8.1 replication Feedback is welcome.

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Thu, 04 Sep 2008 18:31:31 -0700 merb 0.9.5 + haml + datamapper + sqlite3 http://blog.idearise.com/merb-095-haml-datamapper-sqlite3 http://blog.idearise.com/merb-095-haml-datamapper-sqlite3 Since Merb is constantly changing prior to its 1.0 release, a lot of the tutorials did not work exactly the way I expected them to work. I was looking for a tutorial that covered a simple Merb + HAML + DataMapper + SQLite application. I found that a good candidate to tweak was the simple chat wall tutorial that was prominently linked on the Merb wiki. The following are some code updates to get it working with Merb 0.9.5, HAML, DataMapper, and SQLite. Feedback is welcome.

merb-gen app

merb-gen app allows you to specify the template engine, ORM, and testing framework directly on the command line: merb-gen app slapp --template-engine haml --orm datamapper --testing-framework rspec

init.rb

DataMapper dependencies should be specified in init.rb to get counts/aggregates, automatic created_at timestamps, and validations. Datamapper can also handle session storage. And don't forget to set the :session_id_key! init.rb contents [sourcecode="ruby"] dependency "dm-aggregates" dependency "dm-timestamps" dependency "dm-validations" dependency "merb_helpers" ... use_orm :datamapper ... use_template_engine :haml ... c[:session_id_key] = 'chat_wall_session_id' ... c[:session_store] = 'datamapper' ... [/sourcecode]

SQLite database configuration and setup

DataMapper offers a rake task to create a sample database.yml file: rake dm:db:database_yaml Here's what a database.yml file for development and test SQLite databases looks like: database.yml contents [sourcecode="ruby"] --- :development: &defaults :adapter: sqlite3 :database: db/dev.db :test: sqlite db/dev.db sqlite db/test.db Database sessions migration: rake dm:sessions:create Database migration: rake dm:db:automigrate

Post model

Model validations didn't seem to be working properly... post.rb contents [sourcecode="ruby"] class Post include DataMapper::Resource property :id, Serial property :body, String property :created_at, DateTime validates_length :body, :minimum => 2 end [/sourcecode]

HAML

I had problems with the Merb form helper in the index HAML file... app/views/posts/index.html.haml contents [sourcecode="ruby"] !!! Loose %html{html_attrs{'en-us'}} %body %h1 Welcome to Slapp %h2 A simple chat wall %p Recent Posts: .container{:id => 'posts'}= partial("shared/post", :with => @posts) %div Post Something: - #form_for(:post, :action => url(:posts) ) do %form{ :action => '/posts/create', :method => :post} = text_field(:name => "body", :size => 40) = submit "Post Message!" [/sourcecode]The individual post HAML file... app/views/shared/_post.html.haml contents [sourcecode="ruby"] .post{:id => 'post-' + post.id.to_s} .body= h(post.body) .created= relative_date(post.created_at) [/sourcecode]

RSpec

Lastly, I had to change some of the CSS selectors and the deletion method for the index spec... spec/views/posts/index_spec.rb contents [sourcecode="ruby"] ... it "should have a containing div for the posts" do @body.should have_selector("div#posts.container") end it "should have a div for each individual post" do @posts.each do |post| @body.should have_selector("div#posts.container div#post-#{ post.id }.post") end end it "should have the contents of each post inside a div with an id and class" do @posts.each do |post| @body.should have_tag("div#posts") do with_tag(:div, :id => "post-#{ post.id }", :class => "post", :content => post.body) end end end it "should have a form to create new posts with a single input and submit button" do @body.should have_selector("form[@action='/posts/create']") @body.should have_selector("form[@action='/posts/create'] input[@name='body']") @body.should have_selector("form[@action='/posts/create'] input[@type='submit']") end after(:each) do Post.all.destroy! end ... [/sourcecode]

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Tue, 02 Sep 2008 23:00:34 -0700 merb 0.9.5 session_secret_key has changed http://blog.idearise.com/merb-095-sessionsecretkey-has-changed http://blog.idearise.com/merb-095-sessionsecretkey-has-changed Being new to merb, I kept getting this error message after generating a basic merb 0.9.5 app: ~ Maybe the site's session_secret_key has changed? - Merb::CookieSession::TamperedWithCookie) /usr/lib/ruby/gems/1.8/gems/merb-core-0.9.5/lib/merb-core/dispatch/session/cookie.rb:162:in `unmarshal' /usr/lib/ruby/gems/1.8/gems/merb-core-0.9.5/lib/merb-core/dispatch/session/cookie.rb:77:in `initialize' /usr/lib/ruby/gems/1.8/gems/merb-core-0.9.5/lib/merb-core/dispatch/session/cookie.rb:14:in `new' /usr/lib/ruby/gems/1.8/gems/merb-core-0.9.5/lib/merb-core/dispatch/session/cookie.rb:14:in `setup_session' What I didn't know was that I needed to uncomment (and set) the :session_id_key in init.rb: [sourcecode="ruby"] # Sets up a custom session id key which is used for the session persistence # cookie name.  If not specified, defaults to '_session_id'. # c[:session_id_key] = '_session_id' [/sourcecode] The comments threw me off since I was expecting a default value. Update: It's hard to find good merb tutorials right now. I wish I had seen the first of these two posts in advance. It specifically says to uncomment the session_id_key:

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Mon, 01 Sep 2008 11:58:12 -0700 Ruby on Rails Plugin-based Creator and Updater Model Relationships http://blog.idearise.com/ruby-on-rails-plugin-based-creator-and-update http://blog.idearise.com/ruby-on-rails-plugin-based-creator-and-update Leonhart was nice enough to post about how easy it is to create a Ruby on Rails plugin. The post described how to create a plugin to help with user creator and updater relationships. One of my colleagues decided to try it. A few days later, he noticed issues with RSpec tests that were doing simple things like user.projects and sent me an email to see if I could spot anything. After spending some time this morning wondering why the RSpec tests were acting really strangely, I tracked down the issue to this section of code in a lib module: [sourcecode="ruby"] belongs_to :creator, :foreign_key => "created_by", :class_name => "User" belongs_to :updator, :foreign_key => "updated_by", :class_name => "User" User.class_eval "has_many: #{self.to_s.tableize}, :foreign_key => :created_by" [/sourcecode]One quick note before you use this in the future: Make sure you have Creator and Updater models/classes that extend User, and use them to establish the relationship with the target model. Otherwise, you'll be wondering why method calls like user.projects returns the projects that a user created instead of the user's projects! [sourcecode="ruby"] belongs_to :creator, :foreign_key => "created_by" belongs_to :updater, :foreign_key => "updated_by" Creator.class_eval "has_many: #{self.to_s.tableize}, :foreign_key => :created_by" Updater.class_eval "has_many: #{self.to_s.tableize}, :foreign_key => :updated_by" [/sourcecode]

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Tue, 11 Dec 2007 02:18:48 -0800 The Links http://blog.idearise.com/the-links-17 http://blog.idearise.com/the-links-17 Mikontalolights [youtube=http://www.youtube.com/watch?v=iyMbAI67hUM&rel=1] The Rest

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Mon, 10 Dec 2007 01:01:01 -0800 The Links http://blog.idearise.com/the-links-16 http://blog.idearise.com/the-links-16 Ruby and Rails The Rest

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Thu, 06 Dec 2007 02:59:10 -0800 The Links http://blog.idearise.com/the-links-15 http://blog.idearise.com/the-links-15 Git The Rest

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Wed, 05 Dec 2007 02:13:18 -0800 The Links http://blog.idearise.com/the-links-14 http://blog.idearise.com/the-links-14 Ruby and Rails The Rest

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto Tue, 04 Dec 2007 03:27:12 -0800 The Links http://blog.idearise.com/the-links-13 http://blog.idearise.com/the-links-13 Ruby and Rails The Rest

Permalink | Leave a comment  »

]]> http://files.posterous.com/user_profile_pics/295185/gravatar2.png http://posterous.com/users/36ueuu1qceKB roberto roberto