Archive for April, 2007

With the recent fall of prices on Intel’s Core Duo processors, it’s that time of the year when I upgrade my hardware. I’ve been reading literally everything that I can find on the net about the recent hardware: reviews, blogs, articles, magazines, you name it.

And here’s the line-up:

• Intel Core 2 Duo 6600
• Scythe Infinity SCINF-1000 CPU Cooler
• Coolermaster Stacker STC-T01 Case
• Asus P5B-Deluxe Motherboard
• 4 GB of DDR2-800 memory
• Any recent video card with Dual-DVI outputs
• Westen Digital “Raptor” Hard Drive (10000 rpm)

Processor choice was easy. You can’t go wrong with the Core 2 Duo CPUs, they are much faster and more energy efficient then AMD offerings. The 6600 was the cheapest CPU with 4mb of cache, and I really wanted to get as much cache as possible - my (unfounded) theory is that big cache should be nice for various build/compiles that I do all the day. Note: Intel recently released lower-end CPU models with 4Mb cache too, if you want to be cheap.

Core 2 Duo is extremely overclockable, from 266Mhz front bus up to 400Mhz! So, the great cooler is in order. And the Scythe Infinity seems to be in the sweet spot - it cools great, especially with 2×12cm low-speed coolers and it’s very quiet. The only downside - the cooler is huge!

Well, with huge Cooler one needs a huge case! Personally, I selected Coolermaster Stacker STC-T01, it looks nice, it has lots of space, and I can use quiet power supply and 12cm low-speed fans in it.

So, the plan is to have highly-overclocked PC (up to 400Mhz front side bus), and for this I need a very solid motherboard. Based on all the reviews, the Asus P5B-Deluxe is the king! This board, it seems, is de-facto standard for seasoned overclockers. And, personally, I always loved Asus boards.

Next, the memory. Obviously, 4GB is the bare minimum now. Java IDEs can be quite a memory hogs, and running various OSes in emulators very quickly eats up all the memory. A warning: in WinXp and WinVista (32-bit), only about 3GB can be used, which is funny - we reached yet another memory barrier. Hopefully, there will be no “memory expanders” and “emulators” like the last time with 640kb in DOS. Seriously, the faster everybody switches to 64 bit the better. But based on various reviews, it seems that 64bit versions of Microsoft OSes are not the best option right now - too much quirks and incompatibilities. I’ll go with Linux/Solaris dual-boot, thank you.

The next thing - a hard drive. I’m still debating this. The RAID support in the latest motherboards (and in Intel 865 chipset, in particular) is good, and probably it’s the time to start seriously using it. As for the “Raptor” hard drives, I’ve been using them in the past, and they work great. Obviously, it’s not that easy to notice ANY performance improvements between 10000 rpm drives and the rest of 7200rpm drives, but it gives a nice warm feeling…

Video card. Since I don’t plan to play games much, there is no point to buy expensive video card. Any decent card would be fine, as long as it has two DVI outputs. Dual monitors with DVI out is the _only_ way to go for software developers. I’ve been using this setup for years and can say that switching from one monitor to two monitors was the most successful upgrade for me. I can see more, I can have more windows, editors, browsers, consoles, etc. Seriously, if you have just one monitor, do yourself a favor and buy a second one, the rest of hardware upgrades can wait!

I must admit, I’m a book freak. I read a lot of technical books, just for fun. In early days, when I was in California, it was easy to find a good place with lots of new technical books and with good coffee. Now, being in Germany, I mostly rely on online resources.

So while I do enjoy reading paper books, I tend to use online libraries more and more. I’m aware of the following two online libraries:

• Safari Library
• Books24×7

Basically, hundreds and hundreds of technical books are present in both libraries, and with good share of new, recent books. It seems that Safari Library has more extensive book collection, with, naturally, most of O’Reilly titles. But Books24×7 also has lots of books that are not present in Safari. So, the libraries complement each other nicely.

In Safari Library, there’s also a “Rough Cuts” section, with preliminary versions of books that are not yet in stores! For both libraries, there are RSS feeds provided, with the latest book arrivals, so it’s easy to be up-to-date.

But I guess, the most valuable service I get from online libraries is the search. Just think of it, fully indexed content of ALL available in the library books! Now, any time I have a need to refresh my knowledge on any specific topic, I directly go to these two resources and search there. Most of the time, there are quite few books on the topic. And the quality of the info in the books is generally much higher then in the typical online resources.

Naturally, the service is not free, and it definitely helps to have a corporate account in these libraries! But even before my company provided such account, I was a happy paying customer of Safari Library - the basic subscription plan doesn’t cost much.

If I had to select only one library, I’d go with Safari. It has better user interface, better books, more intelligent search, ability to download book chapters in PDF format.

And if I had to select only one book, I’d really recommend “Code Complete” by Steve McConnell. Take a look, this book radically changed the way I look at my code.

Come see us (ME Framework team) at JavaOne conference in San Francisco May 8-11 2007. We will be sharing our Java ME testing and testing tools experience at the technical session TS-5906 (”Building a Java ME Test Suite in 15 Minutes”) which is scheduled for May 9, 6:35 PM.

You can meet our team at Sun Microsystems booth #970 on the pavilion floor. Stop by to chat, discuss ideas, and see the demos.

Personally, I won’t be at JavaOne this year, I managed to break my arm (quite some time ago, so no worries) and now almost done with recovering. But plenty of our folks from Santa Clara and Saint Petersburg are going to be there.

Update: Terrence has quite a lot of additional info on Java ME at JavaOne.

Here’s a very neat trick to quickly open Javadoc documentation for particular Java class, in Firefox browser. I got it from Dmitri and been using it for years. Basically, it adds a “Quick Search” bookmark to Firefox, with shortcut to invoke it.

So, open up your Firefox bookmarks and create a new one, with the following info:

• Name: Quick Javadoc Search
• Location: Copy This Link
• Keyword: jd

This is it. If you look at the link closer, you’ll notice that we just use Google search for “(Java Platform 5.0)” on java.sun.com site in “I’m feeling lucky” mode.

Now, any time you enter in the Firefox, for example,

jd String

it will get you to the javadocs for java.lang.String. In most cases you don’t even need to fully qualify the class name!

With the recent ME Framework open sourcing, and with all those discussions on what ME Framework can do for application developers (as opposed to TCK writers), I took a look at the APIs for popular JUnit-like unit testing frameworks for Java ME, to see what we can do better in our ME Framework.

Please note that this comparison is focused mostly on the API.

Popular Unit testing framework for Java ME:

1. J2MEUnit
2. JMUnit
3. Sony Ericsson Mobile JUnit 1.0

All three frameworks are close to the original JUnit, with adjustments needed to support the Java ME Platform. In all 3 cases, a test class should extend a base class from the appropriate framework. And all typical JUnit helper methods are provided to the subclasses, e.g.: assertTrue, assertFalse, assertEquals, etc. (J2MEUnit provides the bare minimum of such methods, leaving quite some of them out).

JUnit standard hook methods are also present: setUp() and tearDown(), to setup test environment and to tear it down after the test is done.

So, in all 3 cases, developers who are familiar with JUnit, can start writing new unit tests for Java ME with fewer problems, since they basically know the core API.

The differences can be seen in areas where support for Java ME was added (one of the major problems is the lack of reflection in Java ME, while the original JUnit relies heavily on it). While the tests themselves look pretty similar for all three frameworks, the way the test suites are constructed differs.

1. J2MEUnit

With absence of reflection, J2MEUnit authors decided that test suites should be created manually, explicitly adding new tests to the test suite, like this:

  aSuite.addTest(
      new TestOne("testOne", new TestMethod() {
          public void run(TestCase tc) {
              ((TestOne) tc).testOne();
          }
      })
   );

The name of the test provided in the constructor, and the functionality of the test is enclosed in the anonymous inner class that implements the TestMethod.

Pretty heavyweight and not that simple, if you ask me.

2. JMUnit

JMUnit test class must call a two-parameters constructor of the base class, specifying the number of the “sub”-tests, and the name of the test.

public TemperatureConversionTest() {
    super(4,"TemperatureConversionTest");
}

The test selection and execution is done via method test(int), that should be implemented by every test:

public void test(int testNumber) throws Throwable {
    switch(testNumber) {
        case 0:testfahrenheitToCelsius();break;
        case 1:testcelsiusToFahrenheit();break;
        case 2:testisHotter();break;
        case 3:testisCooler();break;
        default: break;
    }
}

The base class will call this method with all int values, from 0 to number-of-tests.

In my opinion, this approach is less convoluted and more straightforward then J2MEUnit. JMUnit also provides better GUI on a device that shows the results.

JMUnit provides two versions, one is suitable for CLDC 1.0, and the another one is for CLDC 1.1.

Overall, JMUnit provides more extensive set of helper methods (asserts), with better GUI, and with more straightforward configuration.

Netbeans’ Mobility Pack has switched from J2MEUnit to JMUnit recently.

3. Sony Ericsson Mobile JUnit 1.0

The folks from Sony Ericsson did an excellent job on this, providing good extension to JUnit for Java ME, and with great documentation

Sony Ericsson approach, it seems, was to provide experience as close to original JUnit as possible. In most cases, to port JUnit tests to Sony Ericsson unit tests is just a matter of replacing the base test class all the tests extend (from JUnit one to Sony Ericsson one.)

Couple of innovative things were done in this library. The absence of reflection is handled on Java SE side, where the tests are being analyzed and proper extra classes are created, so that developers are freed from manual work of specifying all their test names. Sony Ericsson’s version is clearly better here then the two alternatives (J2MEUnit and JMUnit).

The library also provides nice additions on top of typical JUnit API. For example, it not only provides setUp() method, but also setUp(MIDlet midlet), for those tests that would like to access the midlet that runs the tests.

The GUI on the device looks nice too, with tabbed interface for passed/failed tests. Sample project for WTK is also provided. And the basic code coverage can be calculated, which is an unexpected plus.

Integration with Ant, NetBeans and Eclipse is provided.

Overall, Sony Ericsson Mobile JUnit 1.0 is very well done and easily beats other alternatives, discussed above. They provide the best JUnit “emulation”, and resolve absence of reflection in innovative way simplifying developers life.

For additional information, take a look at the following article: “JUnit Testing Using Java ME JUnit Frameworks“.

I’ve been looking for ways to improve performance in some of my code, at least trying to grab all those “low-hanging” performance fruits, when you just change a couple of lines and the performance dramatically improves. With profilers like NetBeans Profiler, it’s almost too easy and doesn’t take much time.

I’ve noticed that we use StringTokenizers here and there, mostly to parse quite big text files, while the javadoc for the StringTokenizer states:

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

OK, I thought, let’s switch to regular expressions and see, they are indeed more powerful and convenient to use. Well, yes, except for the fact that the new code runs typically 4-5 times slower than the original one. Come to think of it, the regexp-based version indeed should have been much slower when compared to very simple and straightforward Tokenizer.

So, the lesson for me here is to use String.split() for non-performance sensitive operations and to keep using good old StringTokenizer when the performance is critical. Oh, and not to trust everything I read in javadocs.

A great move, now NetBeans Profiler is integrated into the latest Netbeans 6.0 milestone 8, and comes directly with NetBeans, no need to download and install it separately.

I’ve been using the Profiler for months now, and can honestly say that the strong desire to buy a commercial profiler is no more. The latest NetBeans Profiler does all or almost all that I need, including performance analysis, memory analysis, heap dumps, compatibility with jhat, partial class instrumentation (no need to instrument the whole application).

The two missing in Profiler 5.5 features are now implemented, and work:

1. Heap Walker
2. Compare Memory Snapshots

I’ve been also quite surprised by performance of instrumented code. Yes, it’s slower than non-instrumented code, but the difference is not that big, so I can execute my apps under profiler and they run without major performance degradation.

Overall, the memory and performance analysis in Java has become much more easy and fun.

In addition to my previously posted small HOWTO on Finding Memory Leaks in Java, here’s a collection of various articles and blogs with some additional info on how to use standard JDK tools to detect and analyze memory leaks:

• Tracking Down Memory Leaks by Greg Sporar
• Plugging memory leaks with weak references by Brian Goetz
• Using Mustang’s jmap/jhat to profile GlassFish by Jean-Francois Arcand
• Heap Dump Snapshots by Kelly O’Hair
• What’s in my Java heap? by A. Sundararajan
• Heap dumps are back with a vengeance! by Alan Bateman

Here is a small HOWTO on how to find memory leaks with Java SE.

I’ve written it while trying to find memory leaks in our testing tools: JTHarness and ME Framework, and then wanted to share the HOWTO with the world, but I didn’t have my blog at that time, so I posted this info as a comment to a relevant entry in the excellent Adam Bien’s blog.

Note: Use the latest JDK 6, because it has the latest tools, with lots of bug fixes and improvements. All the later examples assume that JDK6’s bin directory is in the PATH.

Step 1. Start the application.

Start the application as you usually do:

java -jar java_app.jar

Alternatively, you could start java with hprof agent. Java will run slower, but the huge benefit of this approach is that the stack traces for created objects will be available which improves memory leak analysis greatly:

 java
  -Dcom.sun.management.jmxremote
  -Dcom.sun.management.jmxremote.port=9000
  -Dcom.sun.management.jmxremote.authenticate=false
  -Dcom.sun.management.jmxremote.ssl=false
  -agentlib:hprof=heap=dump,file=/tmp/hprof.bin,
   format=b,depth=10
  -jar java_app.jar

When the application is up, perform various actions that you think might lead to memory leaks.

For example, if you open some documents in your app, the memory graph could rapidly go up. If closing the docs and invocation of full garbage collection did not bring the memory back to normal level, there is probably a leak somewhere.You might use jconsole from JDK 6 to see the memory consumption graph to have a clue whether memory leak is present or not:

jconsole

It will pop up a dialog with a list of Java applications to connect to. Find the one with java_app.jar and connect. Also, jconsole allows to invoke full GC providing nice button just for that.

Step 2. Find the application pid.

Find out the application’s process id via:

jps

It will print something like:

15976 java_app.jar
7586 startup.jar
22476 Jps
12248 Main
5437 Bootstrap

In our case the pid is 15976.

Step 3. Dump the heap into file.

Dump heap into the file:

jmap -dump:format=b,file=/tmp/java_app-heap.bin 15976

We just told jmap to dump the heap into /tmp/java_app-heap.bin file, in binary from (which is optimized to work with large heaps). The third parameter is the pid we found in Step 2.

Alternatively, if you started java with hprof agent, you could just use Ctrl-\ on Solaris/Linux or Ctrl-Break on Windows to dump heap into the file, specified in hprof agent arguments.

Step 4. Visualize the heap.

Use jhat tool to visualize the heap:

jhat -J-Xmx326m /tmp/java_app-heap.bin

Jhat will parse the heap dump and start a web server at port 7000. Connect to Jhat server by pointing your browser to:

http://localhost:7000

And start investigating.

Jhat allows you to see what objects are present in the heap, who has references to those objects, etc.

Here are some tips:

• Investigate _instances_, not _classes_.
• Use the following URL to see the instances: http://localhost:7000/showInstanceCounts/
• Use “Reference Chains from Rootset” (Exclude weak refs!!!) to see who’s holding the instance.