I felt compelled to post this because this book is just great & I wanted to share it with anyone who is interested. If you are just getting into behavior driven development (BDD), RSpec or cucumber then I have a book to recommend to you! Having heard nothing but good things, I recently purchased “The RSpec Book” by David Chelimsky, Dave Astels, Zach Dennis, Aslak Hellesøy, Bryan Helmkamp, Dan North from Pragmatic Programmers.

It’s a easy read that is packed with great tips on how to approach BDD – as well as detailed info on rspec’s features, and tons of other very useful stuff.

The book is in beta right now (not finished) but you can purchase & get the PDF right now. As updates are made you can re-download the PDF. Once the book is actually finished as paperback will be sent to me. I can’t wait!

Worth mentioning is that I  am just a fan — not affiliated with anyone who makes money from this, nor do I make any money for posting this or my website in general.

Performance Optimization
————————
- optimization doesn’t end with speeding up my Ruby code (or Rails itself).
The performance of production application heavily depends on its environment – server filesystems, databases, load balancing solutions.

Testing
——-
- use gems like UnitRecord or dust to cut off the database for unit testing. Mock all unit tests out to speed things up drastically.
- to scale large tests you can offload them to remote BAMs(Bad Ass Machines), or send tests to multiple processors. Deep Test allows you to spawn multiple remote threads to run functional tests
- to benchmark a specific url you can use the ab (Apache HTTP server benchmarking tool) command:  ab -n 100 http://localhost:3000/quick_reply/list
- Errorlytics web service detects 404 errors via JS & redirects user based on rules that you define at errorlytics.com – http://www.errorlytics.com/
- Using factories instead of fixtures.
- Fixtures are more brittle & they seperate the data from the test, creating a assumption that the data already exists & is correct to begin with.
- Factories …. Fixjour, Factory Girl, Object_daddy and Machinist are plugins that aim at creating factories.
- Cucumber tips: http://media.railscasts.com/videos/159_more_on_cucumber.mov

Cucumber
——–
- Write tests first, the run cucumber. To run a specific feature run: cucumber features/name.feature
- Use output of cucumber to create stub step definitions.
- Make each test pass, from the top down.

Feature: title
In Order To [business value]
As a [role]
I want to [action] some feature

Scenatio: title
When [context]
And [more contect]
When I do [action]
And [other action]
Then I should see [outcome]
But I should not see [outcome]

Database
——–
- CouchDB is a new way to think about database storage. Schema-less design. http://couchdb.apache.org/

Video / Screen / Pod casts
————————–

http://railscasts.com/episodes

http://peepcode.com

http://pivotallabs.com/talks/

http://www.railsenvy.com/podcast

http://confreaks.com/events

I was working with a colleague today who was kind enough to introduce me to autotest for continuous test integration. Change a file in your app & tests are automatically run for you… Nice.

screenshot of my mumble alert (top-right)

One thing that really stood out aesthetically was how status messages were passed to Growl (in OSX) for eye-pleasing popup notifications outside of a terminal window.

Being a ubuntu user, I started looking for a similar solution for linux. It turns out there is a great utility for linux that resembles Growl — it’s called Mumbles. The challenge was getting Mumbles to work with autotest.

After searching google, I found one post that got me started. This is a good post, but I had to fill in a few blanks to get everything to work.

Here’s what I did:

1. Install ZenTest:

sudo gem install ZenTest

1. Install Mumbles
   • grab the latest .deb file or source from the mumbles-project page:

   wget https://sourceforge.net/project/showfiles.php?group_id=193587&package_id=227998

   • test out mumbles by running:

   mumbles &

   mumbles-send ‘a title’ ‘it works!!’

   • assuming mumbles has been installed correctly, you should see a little notification pop up. I recommend that at this point you add mumbles to your system session so it will start when you login to ubuntu.

   gnome-session-properties

   add -> “mumbles” for name & command -> ok

2. Install ruby-dbus. I could not find a gem, so install manually:
   • grab the latest from https://trac.luon.net/ruby-dbus/

   wget https://trac.luon.net/data/ruby-dbus/releases/ruby-dbus-0.2.1.tar.gz

   • extract the archive to a local directory

   tar -xvf ./ruby-dbus-0.2.1.tar.gz

   • look over the README file — it provides simple instructions:

   ruby setup.rb config

   ruby setup.rb setup

   sudo ruby setup.rb install

   • test this out by requiring dbus via irb:

   irb

   irb(main):001:0> require ‘dbus’

   => true

3. Create a .autotest file in your application root directory (or your home directory to make the settings global) Thanks to CaffinatedCode for this script — they wrote it:

require ‘dbus’def send_message(title, message, icon)
begin
bus = DBus::SessionBus.instance
mumbles_service = bus.service(“org.mumblesproject.Mumbles”)
mumbles = mumbles_service.object(“/org/mumblesproject/Mumbles”)
mumbles.introspect
mumbles_iface = mumbles["org.mumblesproject.Mumbles"]
sig = mumbles_iface.signals["Notify"]
bus.emit(mumbles_service, mumbles, mumbles_iface, sig, title, message, icon)
rescue Exception => e
end
end

Autotest.add_hook :ran_command do |at|
begin
output = at.results.last.slice(/(\d+) examples?, (\d+) failures?(, \d+ pending)?/)
if output =~ /.*[1-9] failure.*/ then
send_message(“FAIL”, “#{output}”, “fail.png”)
else
send_message(“PASS”, “#{output}”, “pass.png”)
end
rescue Exception => e
end
end

Now for the fun part. From within your app root, type:

autotest

If everything works out, you should see a few things flash by – I get a error about “Insecure world writable dir” in my home dir. I just ignored that. After that, you should see a test summary in the terminal window, followed by a mumbles popup notificaton. Kick ass!

I feel like I have been in a cave for the last 6 months… Well, I sorta have. I’ve been working on a rails project for Epocrates.com for the last 6 months — a type of social network that joins wall street with MDs opted into market research. What a great learning experience & opportunity. It’ll be great one day…

I could only get to chapter 4 today (F.Y.I. A.D.D.4-I) — but it was a nice fresh take on ruby & programming in general. Rather basic & elementary so far, but recommended nonetheless.I never thought of an array as a caterpillar…

Check it out if you have not already!

I just got home from my trip to the RailsConf 2008 trip — it was the shit!

No time to say much now, other than it was great & I hope you made it there…

What a great conference.

So many things from this event will change my approach to programming — I wish I had enough energy to post more details at this time, but I am exhausted. Look for a post in the near future about what I have learned — there’s a lot to share.

For now, check out info on the RailsConf website.

A company I work for uses Instiki to as a wiki-type system on the company intranet. Instiki is a simple RoR (Ruby on Rails) wiki clone written by David Heinemeier Hansson (… the creator of the Ruby on Rails framework). It’s a great app & I would recommend looking into it if you have not already.

Anyway, I found that upgrading from v0.10 to v0.13 was a little challenging. Here’s what I did to make it work the way I wanted it to:

1. Download & Install instiki v0.13. It’s damn easy. Follow onscreen directions. If you get stuck, see the Instiki website.

2. Dump data from v0.10 to a sql file.

There is a dump script for recent versions of instiki, but I did not see the v0.10 install. The dump script is called “import_storage”.

You can use the “import_storage “script that comes with v0.13 to dump the data from your v0.10 install by executing the following from within instiki v0.13 app root directory:

ruby script/import_storage -t /tmp/i2500 -i /usr/local/instiki-0.10.1/ -d mysql -o /tmp/instiki_20080408-1.sql

where:
/usr/local/instiki-0.10.1/ is the path to the 0.10 app
/tmp/instiki_20080408-1.sql is the path of the dump file
/tmp/12500 is a temp path (may or may not be needed depending on system permissions)

From there, you should have a sql file that contains a dump of the current v0.10 database in the form of a .sql file located at: /tmp/instiki_20080408-1.sql

3. Load up 0.10 data into the 0.13 database

When you installed instiki 0.13, you created a new database. Let’s call that database “wiki”. You need to load the sql file you just created into that database now.

You can load the .sql file into your mysql ‘wiki’ database by executing:

$ mysql -u[username] -p[password] wiki < /tmp/instiki_20080408-1.sql

4. Empty the SPAM pattern file

v0.13 contains a SPAM detection feature that uses keywords. Cool idea, but the default list of patterns is kinda odd. For example, any page that contains the following words will be marked as SPAM: airplane, good job, hamburger…. and a ton of other totally random words. I would definitely recommend emptying the SPAM pattern file (and adding words as necessary if needed).

You can empty the SPAM pattern file by executing:

echo “” > [app root]/config/spam_patterns.txt

5. Allow forward-slash (/) to appear in page names.

With instiki v0.10, page names could contain forward slashes in them, for example: “CARS/TRUCKS”. When I installed v0.13, I found that links to these pages would cause a internal server error – a error screen was displayed instead of the page I wanted. Not sure if this is due to changes with rails, routing, or instiki itself, but I was able to change the routes to behave the way I wanted them to by adding 1 line to [app root]/config/routes.rb….

after the last “connect_to_web” line in routes.rb, I added:

connect_to_web map, ‘:web/show/*id’, :controller => ‘wiki’, :action => ’show’

This should “glob” any text after :web/show to an array. Also, you will then need to edit the “load_page” method in [app root]/app/controllers/wiki_controller.rb to work with the array (it expects a string by default).

Search [app root]/app/controllers/wiki_controller.rb for “def load_page”, and replace the line:

@page_name = params['id']

with:

@page_name = params['id'].to_a.join(‘/’)

At that point, you should be able to use forward slashes in page names.

6. Tweak the HTML sanitization file

Lastly, I tweaked the [app root]/lib/sanitize.rb file a bit. You may need to add elements to the “acceptable_elements” array as needed (top of the file). But more importantly, I found that HTML that was entered in uppercase (for example, <BR> or <H1>) was being escaped. I know it’s old-school HTML, but that’s what the company’s wiki contained. So, rather than updating the HTML, I made the wiki accept it. It seems that because it is uppercase, it fails to match the acceptable list of tags. I changed the sanitize_html method to be:

def sanitize_html(html)
if html.index(“<”)
tokenizer = HTML::Tokenizer.new(html)
new_text = “”

while token = tokenizer.next
node = XHTML::Node.parse(nil, 0, 0, token, false)
new_text << case node.tag?
when true
if ALLOWED_ELEMENTS.include?(node.name.downcase)
if node.closing != :close
node.attributes.delete_if { |attr,v| !ALLOWED_ATTRIBUTES.include?(attr) }
ATTR_VAL_IS_URI.each do |attr|
val_unescaped = CGI.unescapeHTML(node.attributes[attr].to_s).gsub(/[00-40\177-\240]+/,”).downcase
if val_unescaped =~ /^[a-z0-9][-+.a-z0-9]*:/ and !ALLOWED_PROTOCOLS.include?(val_unescaped.split(‘:’)[0])
node.attributes.delete attr
end
end
if node.attributes['style']
node.attributes['style'] = sanitize_css(node.attributes['style'])
end
end
node.to_s
else
node.to_s.gsub(/</, “<”)
end
else
node.to_s.gsub(/</, “<”)
end
end

html = new_text
end
html
end

… the main difference is the addition of “downcase” methods to the comparison lines.

That’s it. Works great for me. Please let me know if this was helpful, or if you have a better way to achieve these results!

In case any of you need to parse the PubMed database, here’s what I came up with as a first try. Run via a ruby script. You will need ruby, ActiveRecord (if you have rails setup, you should be good), and a few libraries — use gem install [x] for the libraries that you don’t have — if you have rails installed, you probably only need to get “xmlsimple”.

gem install xmlsimple

I used 3 tables in my mysql database: articles, authors, and a join table articles_authors

Here’s the schema:

DROP TABLE IF EXISTS `pubmed`.`articles`;
CREATE TABLE `pubmed`.`articles` (
`id` int(11) NOT NULL auto_increment,
`pubmed_id` int(11) NOT NULL,
`source` varchar(50) character set latin1 default NULL,
`title` varchar(255) character set latin1 default NULL,
`full_journal_name` varchar(255) character set latin1 default NULL,
`author_list` varchar(255) character set latin1 default NULL,
`pub_date` date default NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=34013 DEFAULT CHARSET=utf8;

DROP TABLE IF EXISTS `pubmed`.`articles_authors`;
CREATE TABLE `pubmed`.`articles_authors` (
`article_id` int(11) NOT NULL,
`author_id` int(11) NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

DROP TABLE IF EXISTS `pubmed`.`authors`;
CREATE TABLE `pubmed`.`authors` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(255) default NULL,
`pubmed_id` int(11) default NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=69002 DEFAULT CHARSET=latin1;

Here’s the ruby script:

# #!/usr/local/bin/ruby -w

# Require Files & Libs

require 'net/http'

require 'rubygems'

require 'active_record'

require 'xmlsimple'

# App Configuration Settings

empty_tables = true

# Search Settings

searchRelDate = 7

searchLimit = 100000

batch_size = 100

searchUrlBase = "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&retmax=#{searchLimit}&"

summaryUrlBase = "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&retmode=xml&id="

# Database Settings:

db_host    = "localhost"

db_user    = "root"

db_pass    = "password"

db_schema  = "pubmed"

# Header Output:

puts  "#"

puts  "# PubMed parsing tool."

dash = "========================================================================="

# Connect to DB using Active Record:

ActiveRecord::Base.establish_connection(:adapter=>"mysql", :database=>db_schema, :username=>db_user, :password=>db_pass, :host=>db_host)

# Define SearchResults model:

class Article < ActiveRecord::Base

  set_table_name "articles"

end

class Author < ActiveRecord::Base

  set_table_name "authors"

end

class ArticleAuthor  0

          property['Item'].each{|author| article[:authors] < "#{article_summary['Id']}",

        :source => article['Source'],

        :title => article['Title'],

        :full_journal_name => article['FullJournalName'],

        :pub_date => article['PubDate'],

        :author_list => article[:authors].join(', ')

      )

      # Add Authors to database:

      article[:authors].each{|author|

        # See if the author exists already:

        exists = Author.find_all_by_name("#{author}")    

        if exists.length == 0

          new_author = Author.create(

            :name => author,

            :pubmed_id => "#{article_summary['Id']}"

          )

        else

          new_author = exists[0]

        end

        # Add to join table:

        ArticleAuthor.create(

          :author_id =>  new_author.id,

          :article_id =>  new_article.id

        )

      } 

    }

  batch_current += 1

end

From there, you can run queries like this:

SELECT author_id, authors.name, count(author_id)
FROM articles_authors, authors
WHERE articles_authors.author_id = authors.id
GROUP BY author_id
ORDER BY count(author_id) desc

SELECT articles.pubmed_id, authors.name, articles.title, articles.source, articles.pub_date, articles.id
FROM articles_authors, authors, articles
WHERE articles_authors.author_id = authors.id and articles_authors.article_id = articles.id
and authors.name = “Li Y”
GROUP BY articles.pubmed_id

Installing
I found a post by “Loren Johnson” that had code that worked well for me:

rails test
sudo gem install daemons
cd test
ruby script/plugin install http://svn.kylemaxwell.com/rails_plugins/daemon_generator/trunk
ruby script/generate daemon tester

Using
First off, check out the (brief) Daemon Generator readme file.

Open up the file that was generated from the “generate” command above:
/lib/daemons/tester.rb
It’s short — take a look. This is where your custom stuff will added. More on that later.

Open up your /log/xyz.log file (development.log, production.log, etc) — it should have a message like this: This daemon is still runnig at Tue Oct 09 13:43:38 -0700 2007.

…That was inserted by a function stub created within tester.rb.

Change tester.rb to do something useful… As a simple example, I changed the line in tester.rb from:
ActiveRecord::Base.logger << “This daemon is still running at #{Time.now}.\n”
to:
ActiveRecord::Base.logger << “Disk space free: “+`df -hl | grep /dev/sda2`+”\n”

….. you get the idea.

NOTE that changing the code within the file tester.rb does not take effect until you restart the daemon: ruby script/daemons restart

Links
http://kylemaxwell.typepad.com/everystudent/2006/08/after_writing_r.html
http://wiki.rubyonrails.org/rails/pages/HowToRunBackgroundJobsInRails
http://douglasfshearer.com/blog/cron-jobs-in-ruby-on-rails

I’m working on a Rails project that requires LDAP authentication.

I searched the net for a long time, but finally got something to work for me.

…  just use your own username, password & hostname.

Note that the username and password should be contained in SINGLE QUOTES unless you have some reason to use double quotes…. that messed me up for a while.

<pre>
require ‘rubygems’
require ‘net/ldap’

user = ‘DOMAIN_NAME\my_username’
pass = ‘my_password’
ldap = Net::LDAP.new :host => “x.x.x.x”, :port => 389, :auth => {:method => :simple, :username => user, :password => pass}
p (ldap.bind) ? “Authorization Succeeded!” : “Authorization Failed: #{ldap.get_operation_result.message}”
</pre>

I have been working on a program that will help with mp3 ID3 tags. The idea is that the program will analyze any available data (artist, album, track length, genre, etc) against a huge database of information. From there, you could add data to the mp3 tags based on match results.

Anyway, here’s what I have so far — it’s a ruby program that you execute from a command line. Combined with a freedb database (freedb.org), this script parses the data into a mySQL database. I work on one genre at a time — there are over 700k records in the rock & misc categories — any more would start to create memory issues.

Send me a email if you are interested in the code.