NetBeans 6.1 has been released recently, and the upgrade was easy and pain-free for me. I haven’t found any serious problems so far and the release looks very solid and performant. The Java stuff works just fine, and Ruby capabilities are great. For those who look for a great and intelligent Ruby editor, NetBeans is one of the great candidates.

One minor issue with NetBeans 6.1 is that it ships by default with very basic IRB console for Ruby: no history, no pop-ups for code completion. Since I’m used to JRuby IRB Console which provides those advanced features, that was a bit of inconvenience for me.

One of NetBeans guys, Martin Krauskopf is to the rescue! It turned out that there is a special property that enables the full JRuby IRB Console. Just add

   1: -J-Dirb.jruby=true

to the netbeans_default_options entry in the etc/netbeans.conf file. A word of caution though: The JIRB Console was disabled by default due to some problems it was causing in some environments. For me, on Windows, it works just great.

Ben Collins-Sussman posted an interesting blog entry “Subversion’s Future?”, where one of the main points made was that while distributed source control systems are OK for smallish/open-source projects, Subversion’s sweet spot is with huge projects. I couldn’t disagree more. And here’s my response.

I’ve been using the distributed source controls systems for more than a decade and been watching other big projects using the distributed systems and it seems to me the DVCS provide the more benefits the bigger the size of the project. What are the characteristics of a huge project? In most cases, it means that there is a big team working on it. Big team means global team, all over the globe. This is not an “open-source thing”, this is a reality of corporate software development too. Most companies (well, at least those who are actually producing huge projects) are global companies, with offices in the U.S, Europe, India, etc. Working globally on the central repository *is* painful and slow. Tried to bisect the regression introduced between the releases, switching between many  revisions, tried to follow the history of some code in Subversion? Doing this when the main repository is overseas is not fun.

A big team of engineers is typically organized in a hierarchy of smaller sub-teams focusing on particular area of the product. Again, it’s much more natural to organize a hierarchy of workspaces matching the structure of the organization. There are many benefits to that approach: mostly the members of such sub-team care about their area, not touching/changing anything in other places. And they better know their code, so they could find/fix new bugs and regressions faster. Typically, there is a special QA force for each sub-team, trained and specialized in testing particular area. Once they tested/OK’ed the particular state of the team code, it can be pushed upwards to the integration workspace. Thus, members of other teams won’t even be disrupted by local problems/regressions, since they would get the more stable and better tested code. The distributed source control allows to do that beautifully and naturally. Doing this in Subversion is seriously painful.

Also, what are those mythical huge projects that nobody knows about? How about OpenSolaris or Linux?Are they “huge enough”? How about Mozilla or Ubuntu or NetBeans or JDK or MySQL? And all these projects do use distributed source control tools. Solaris and Java SDK only recently were open-sourced. Before that they were, by all means, huge commercial software projects, each with many, many years of development by hundreds of people. They were developed with distributed source control system. There *is* a reason why Linux never used CVS/Subversion and why even commercial non open source distributed system was used to develop Linux (since there were no good open DVCS at the time). And the reason, of course, is that distributed source control helps managing the overwhelming complexity of the huge projects much better than centralized one.

One other point in the Ben’s blog entry was about usability and ease of use of Subversion. Yeah, it is easier to use in simple scenarios, but once the size of a project grows, it gets harder and harder. Besides, if engineers are smart enough to develop and maintain a huge project, adjusting to distributed source control systems would be piece of cake! And for those folks who are stuck with Subversion, there is a great git-svn tool that would allow to leverage the power of distributed source control while working with centralized Subversion repository.

I’m not really saying that Subversion is “bad”. It was actually great for its time, but now there are better and smarter tools out there.

Currently, there are at least FIVE actively developed Ruby implementations out there in the wild. For an excellent overview of the current state of affairs take a look at the Charles’ article: “Promise and Peril for Alternative Ruby Impls”. Having that many implementations, despite all the good things, also brings a new set of challenges. One of the biggest challenges is the compatibility. Some are even starting to say “balkanization” word. (You know who you are!)

That’s where RubySpecs come in. The great promise of the RubySpecs is to provide a unified compatibility and conformance test suite, universally used by all Ruby implementations. The Ruby implementers are not only going to use the tests, but also contribute back the new specs, increasing the value and the quality of the test suite over time. At the moment, we know that Rubinius folks (who started the whole RubySpec thing) actively use them on a daily basis, with a couple of continuous integration bots, and using the RubySpecs as a driver to implement new features in a test-driven approach. Brian Ford is doing an outstanding job supporting the mspec (the engine behind the specs) and accommodating users requests for new features and enhancements. JRuby uses the RubySpecs equally actively, running them on MacOS, Linux and partially on Windows. All the new tests that are of compatibility/conformance nature typically go directly to the RubySpec repository. IronRuby uses the RubySpecs too. Hopefully, we’ll see some contributions from them too, that would be really sweet! And last, but not least, during the first “Design Meeting” with ruby-core folks (including Matz), the RubySpec question was raised, and it seems that even Matz Ruby folks would start looking into it, and eventually using it.

So, if all goes well, most of the current active Ruby implementers would be using the common set of conformance tests. The RubySpecs are going to be forked off from the Rubinius repository and would live in the proper place as a main project on its own, with a bug tracker, a web site, etc. Also, some more folks would start working on the specs via Google’s Summer of Code program. Very exciting!

As an example, take a look at the BigDecimal support. At the beginning of April, there were no specs for BigDecimal. JRuby had a skeleton implementation, missing some methods or just having stubs for some other methods. Rubinius had no BigDecimal support altogether. Three weeks and 43 commits later (by the way, five different folks contributed to the tests), there are 1772 test cases for BigDecimal in the RubySpecs, JRuby has already fixed hundreds of failures, now passing most of them (with 6 remaining tests to be fixed soon) and Rubinius has a actively improving BigDecimal support too.

I guess, the main thing I’m trying to say here is that now it’s the perfect time to look into the RubySpecs and to start contributing. That way, you’ll affect ALL the Ruby implementation, making ALL of them better! For more info, take a look here and here.

This was always confusing me, since Ruby has many ways to perform the division. So, here’s a fresh quote by Matz himself:

We have /, div, fdiv, and quo:
  /     - normal division, according to the class of operands
div    - integer division
fdiv   - float division
quo   - most accurate division (float if either operand is float, otherwise rational)
rdiv   - (which always results in rational) is obsolete.

So, you have that new and shiny JRuby 1.1 and would like to try it out with rails. Here’s a quickest guide to do so! The guide assumes that you want to use MySQL as the database, and it has already been installed.

First, install the following gems:

• mongrel - simple but powerful web server.
• activerecord-jdbcmysql-adapter - all you need for activerecord on JRuby to talk to MySQL.
• rails - well, the Ruby on Rails proper.

The command line:

   1: jruby -S gem install mongrel activerecord-jdbcmysql-adapter rails

Create a sample rails application with MySQL backend:

   1: jruby -S rails myapp -d mysql

Enter the newly-created “myapp” directory, then modify the config/database.yml. First and foremost, you need to adjust the adapter name, and instead of ‘mysql’ you should specify ‘jdbcmysql’. You might also want to delete the lines starting with “socket:”.

Here’s a simple example for the development environment:

   1: development:

   2:   adapter: jdbcmysql

   3:   encoding: utf8

   4:   database: myapp_development

   5:   username: root

   6:   password:

Now, it’s time to create our database:

   1: jruby -S rake db:create:all

The next step is to create some minimal scaffolding so that you could actually play with some dynamic functionality and database access:

   1: jruby script/generate scaffold post title:string body:text published:boolean

We need to update the database after that:

   1: jruby -S rake db:migrate

And we’re basically done here, just start rails via:

   1: jruby script/server

and point your browser to the:

   1: http://localhost:3000/posts

Enjoy!

This is a quick guide for those, who’d like to run the Rubyspecs with minimal effort. For those who have JRuby repository checked out, the steps are even easier: ‘ant spec’ - and you’ll run all the unexcluded specs against the freshly built JRuby.

1. Pull the Rubinius repository (and you have to have git for that):

      1: git clone git://git.rubini.us/code rbx

2. Alternatively, you could just download the snapshot of the repository in a compressed file. (Look for “snapshot” links at the page).
3. Basically, that’s it for the setup. Now, just run the tests:

      1: cd rbx  # Step into the directory you’ve created when pulled

      2: bin/mspec -t ruby spec/ruby/1.8 # Run all Ruby 1.8 specs

      3: bin/mspec -t ruby spec/ruby/1.8/core/kernel # Run Kernel specs

      4: bin/mspec -f s -t ruby spec/ruby/1.8 # More verbose output

4. During the execution, the dots will be printed out, and the final summary, but with “-f s” option, the output is much more interesting:

      1: Kernel.proc

      2: - returns a Proc

      3: - raises an ArgumentError when no block given

      4: - raises an ArgumentError when given too many arguments

      5: - returns from block into caller block

So, that’s about it. The rubyspec format is compatible with Rspec, so there should be no major surprises there. More info *is* available at the official page. I’d suggest to start with “Specs - Overview” and explore from there.

The long awaited JRuby 1.1 is finally out. Working on it was fun, much more fun than I expected — so much to do, so many interesting things, so little time! It is a perfect mixture of Java and Ruby, actually improving my skills in both areas.

For me, it all started last Fall, after a couple of very convincing blog posts by Charlie: How Easy Is It To Contribute To JRuby? and Easy JRuby 1.0.2 Bugs. So I started playing with JRuby, looking for things I could fix, and it turned out that the easiest way to get involved is to look at the Rubyspecs and try to pass as many of them on JRuby as possible. I’ve started filing bugs and submitting patches, and eventually become a JRuby core comitter.

And since then, my main focus is on the compatibility with Ruby, on tests and especially the Rubyspecs. Having spent about 10 years in Java SE and Java ME compatibility, it was rather natural choice for me. We now have the rubyspecs nicely integrated into JRuby build, we maintain an exclude list for those specs that currently fail (and for every new failure we have the bug filed), we sync-up with the main rubyspecs repository frequently and contribute new specs back.

Some major things to do in the future: set up the continuous integration runs for rubyspecs, make the rubyspec runs on Windows and fix the specs to work properly on Windows. It is expected that much more effort will be put into Java Integration infrastructure, and we’ll need an extensive set of tests/specs for the area. The barriers to entry are very low now, and it’s very easy to contribute, so consider doing so!

Some random stats: So far, I’ve filed 232 bug reports, fixed 112 bugs and made 195 commits.

On to JRuby 1.1.1!

It was a “JVM performance week” in JRuby-land. It all started when some recent code changes is JRuby caused an interesting performance degradation with JDK6 on multi-core CPUs. See the original post to the [jvm-l] list and extended coverage of what happened on Charlie’s blog. So, while following the discussion and participating, I learned a few new things and got some interesting additional info and links that I’d like to share:

1. Let’s start with basics. John Rose posted a good starting guide for those who’d like to micro-benchmark things on JVM: So You Want to Write a Micro-Benchmark.
2. Performance techniques used in the Hotspot JVM wiki entry with the list of optimizations JVM does, sweet and to the point.
3. Fundamental work by Ulrich Drepper: What Every Programmer Should Know About Memory, a MUST read, but painful
4. Deep dive into assembly code from Java blog post by Kohsuke Kawaguchi, on how to print out the assembly code that JIT is producing.
5. Examining generated code with OpenJDK 7, exciting stuff!
6. Polymorphism Performance Mysteries Explained, the Issue #158 of The Java Specialists’ Newsletter.
7. How fast is Java Volatile? or Atomic? or Synchronized?, with nice charts and comparisons.
8. Escape analysis can help optimize synchronization by Brian Goetz.

Enjoy!

The Rubyspecs, an extensive set of conformance tests for Ruby platform, is a great tool to enforce the compatibility of JRuby (and Rubinius, and IronRuby). But the specs are currently being mostly run on Linux/BSD/MacOS (against Ruby, JRuby, Rubinius), and are rarely (if ever) executed in Windows environment. Until now, that is.

I tried to run them on Windows some time ago and the mspec, the testing framework that powers Rubyspecs, had some issues with Windows paths. Since then, the issues have been resolved and the Rubyspecs are functional on Windows.

I’ve run them against JRuby, found 4-5 bugs right away (crashes in calls that are not supported on Windows even in Matz Ruby), fixed those, and was able to run the specs to the completion! There are couple of hang-ups here and there, and lots of failures due to fact that rubyspecs were never used on Windows and don’t expect Windows-specific behavior.

I guess, that’s the next big thing, to go over all the specs and adjust them to take Windows specifics into account when needed.

To my surprise, Git is becoming one of the most popular source control systems in Ruby community. New blog entries on how to use Git are popping up all over the place and the amount of excitement is just very unexpected. Who would have thought that source control tools might be so exciting! Well, here are my recipes on how to use Git with Ruby/JRuby subversion repositories.

Personally, I started using Git about 6 months ago, since Rubinius is under Git, and I really wanted to get access to “rubyspecs” which are part of Rubinius repository. At first, I really disliked Git just because the command line is rather complicated, and it takes some time to get used to it and to get familiar with basic Git concepts. After a while, I noticed that I started using Git more and more, and slowly but surely the full power of the tool just started to amaze me. Switches between branches are LESS THAN A SECOND, instant access to the entire history, powerful branch merging, git grep, and the list could go on and on.

So I started to think about switching to Git as my main SCM tool, even on projects that are under Subversion. Again, the git svn turned out to be very capable and complete tool, allowing bidirectional access to subversion repository from Git. Basically, git svn allows to clone the subversion repository with ENTIRE HISTORY into Git repository, and the changes to that Git repository can be pushed back to main Subversion one. On top of that, one can use Stacked Git to maintain a series of patches. This is very useful in case when there is no write access to the main repository. So the patches are handled by Stacked Git and are being sent out to the maintainers via email, or attached to the bug reports. And that’s how I worked on JRuby until I got the commit rights:

1. stg pull svn - get the latest sources from the main (svn) repository, this command will first pop all your currently applied patches, get the latest sources, and re-apply the patches on top of them.
2. stg edit - create new patch, provide description
3. hack, hack, hack
4. stg refresh - record the changes into the patch
5. stg export -p - export the patches into standalone files
6. send the patch to the maintainers

It’s sooo much easier to maintain the patches that way rather than just having the diff files flying around. First, the patches are always against the latest sources, and never outdated (well, at least when somebody changes something in the main repository that conflicts with the patch, you’ll see the issue immediately during the step #1 above, and can correct it right away). Second, it’s trivial to push/pop patches, reorganize them, test with and without them, easy to see which ones applied and which ones are not, etc.

For those who are serious on working on some open source project which is currently under subversion, I recommend to investigate git-svn and stgit in more detail.

Now, here are a couple of examples on how to create a Git repository out of public subversion one.

Let’s start with Ruby itself:

git svn clone -Ttrunk -ttags -bbranches http://svn.ruby-lang.org/repos/ruby ruby.git

Warning: This would take about 12 hours to complete! For those who don’t want to wait that long, here’s the repository I’ve already created: ruby.git.tgz (md5sum: 576b5667affe040fd33478ea074c13b8). Think about it, these 60mb contain entire Ruby history! This is LESS than size of subversion’s working copy! I measured the time to switch between Ruby 1.8 and Ruby 1.9 branches, that’s 500.000 lines changed in 2000 files, and it takes on my Ubuntu Linux about  1.5 SECONDS. git-grep of entire 1.9 branch is 0.2 seconds. git-log of entire 1.9 branch is 0.3 seconds, from the very last revision to the revision #1, is 0.3 seconds. What else can I say?

Now, the JRuby repository:

git-svn clone -Ttrunk/jruby -ttags -bbranches http://svn.codehaus.org/jruby jruby.git

If you have commit rights to JRuby repository, just change HTTP to HTTPS. The process would take about 2.5 hours (and I saw reports that it can be interrupted and then resumed too).

Once you have the local Git repository, you’ll probably need to do the following (when your are in the root of the repository) :

   1: git gc –aggressive –prune

   2: (echo; git-svn show-ignore) >> .git/info/exclude

This would compact the repository further, and will set the proper excludes.

This is just a very basic outline of the process, so if you’re interested, do check out the official documentation and feel free to ask questions.