Comments on: Don’t always trust javadocs

By: Si

Si — Mon, 03 Dec 2007 11:26:57 +0000

For speed related regular expressions you are recommended to use the Pattern / Matcher classes in Java5. Also while it is not an issue for simple tokenizing, more complicated Regex can cause a performance hit if you code it incorrectly.

By: Dan

Dan — Mon, 09 Jul 2007 16:28:41 +0000

Just curious… How many times were you calling String.split()? Was it just one time and it still was 4-5 times slower than tokenizer? Or was it many times. If it was many times, I would expect it to be slower since internally, String.split() is compiling a regex Pattern, an expensive operation. If you need to split many String objects, all using the same regex, you will get much better performance by using Pattern.split(), which will only compile the Pattern one time. The split() methods in String are only considered “convenience” methods.

Regards,

Dan

By: Markus Kohler

Markus Kohler — Fri, 04 May 2007 11:59:27 +0000

Hi,
Even StringTokenizer is not very efficient, if you don’t need all the features it supports.

If you only need to split by one char, you can write a routine that is several times faster and allocates much fewer objects than StringTokenizer does

Regards,
Markus

By: Evgeniy

Evgeniy — Sat, 07 Apr 2007 17:50:32 +0000

In the end you can just grab StringTokenizer.java from JDK’s src.zip and go with it forever, even after it’s removed from JDK.

By: vvs

vvs — Thu, 05 Apr 2007 22:57:04 +0000

Thanks, K. Will take a look. Luckily for us, we have very simple file format, and most probably StringTokenizer is enough. On anything more complex, I’d imagine, more powerful libraries will be quite handy!

By: K

Thu, 05 Apr 2007 22:51:10 +0000

You should try the CSV Parser from Ostermiller

http://ostermiller.org/utils/CSV.html

By: Joe Attardi

Joe Attardi — Thu, 05 Apr 2007 15:13:47 +0000

vvs:

I suppose that makes sense. Fair enough!

By: vvs

vvs — Thu, 05 Apr 2007 09:17:11 +0000

Joe, sure, being working in Java SE/ME conformance teams for years, I *do* trust the javadocs a great deal. That’s why when I saw a note that StringTokenizer should not be used, I tried to make the suggested change, only to figure out that the new code is a bit too slow for my purposes.

As for the future, within the next two years, the hardware will become faster, and by then the performance of StringTokenizer will not probably matter much…

By: Joe Attardi

Joe Attardi — Wed, 04 Apr 2007 20:34:01 +0000

Seeing how the Javadocs are generated directly from source code comments, I’d say it’s pretty safe to trust them. The javadocs for StringTokenizer make no guarantees about performance. So your point about what the javadocs say are pretty much irrelevant to the performance debate.

Should you use code that has been explicitly stated as being legacy code just because it performs better? I’m not so sure. I can almost promise you that in another version or two, StringTokenizer will become deprecated. Will you still advocate its use then?