Software Development - Programming Language Comparison - RoboProg's

RoboProg's / Software Development

Last Month

Dec 20, 2009

My Language Is Better Than Yours!

Really? Better [1] in what situation?

A while back, I put together a test to compare forking and threading, using several implementations. I recently expanded my choice of languages a bit, as well as adding an option to run the entire task set sequentially, just to get an idea how each language would handle a batch. Keep in mind that the bulk of the "work" done by these demo programming examples is creating / updating / discarding strings, just like most real business applications, rather than calculating fibonacci sequences, prime number generation, supernova simulations, or tracking the path of a projectile from the BFG.

So, here are the scenarios I considered, as well as the language [2] that did the best in each category.

• Fork based processing: The unix fork() system call works reasonably well on unix-like operating systems (e.g. Linux), if not under Windows. Since the page table for a process will have to be replicated, it is helpful to have a small memory image, so as to have fewer pages to replicate. It is also helpful to avoid scattering writes across those pages, so as to reduce the amount of copying and updating the memory page content that a copy-on-write scheme would try to defer / avoid. Not surprisingly, the language in the set I am using that had the smallest footprint, C, did the best at using fork().
• Thread based processing: Threads impose little-to-no memory and I/O channel management burden on an operating system, only a scheduling / stack-context-switching load. However, a program running with threads needs to be careful to use a mutual exclusion mechanism to avoid stomping on shared resources. It also helps if this mechanism is efficient. Not surprisingly, the language that was implemented to make heavy use of threads since version 1.0, Java, did the best at using threads.
• Sequential processing: Surprise! Perl did the best in this category, even edging out C. Of course, in this case there is only a single task at a time executing, so operating system process management is not an issue. For the purposes of this little test, the only thing that is really asked of the language and its runtime is to create, destroy, size and append strings as quickly as possible. The native C string implementation is somewhat handicapped by using a sentinel value to terminate each string ('\0'), causing strlen() to be a O(n) operation. In contrast, most other languages use an out-of-band length indicator, so that getting the length of a string is a O(1) (constant time) operation. And apparently, Perl has a very fast string implementation, despite being ready, willing, and able to treat strings as numbers at the drop of a hat.

I have summarized the results below

Each task was run 10,001 times [6], using the method (sequential; fork; thread) inticated. The median time of 5 trials is the time reported above.

Further details: The Atom is a "slow processor, fast memory bus" combination (1600 MHz CPU, 512 KB cache with 1024 MB of 533 MHz memory); while the Pentium-D is a "fast processor, slow memory bus" combination (3000 MHz CPU, 1024 KB cache with 1024 MB of 300 - 400 MHz memory -- I lost the papers on the RAM, so I am not completely sure of its unannounced speed). Also, I seem to have managed to burden each machine with additional crud since last spring, so they are now running a little slower. For the Atom, most of the slow down might be due to enabling the hyper-threading. For the Pentium-D, the change must be due to additional server bloat I've gratuitously installed. Nevertheless, these are the level, but bumpy, playing fields I have provided for each implementation.

Please feel free to download the source (and my logs) and see how these work on your system(s). I would like to know if there are any unexpected results on any much slower or faster systems.

Some parting thoughts: Generating a bunch of tasks, via fork or new threads, turns out to be a really bad way to handle incoming requests on a server. A better way [7] is to have a front end process which inserts commands/requests into a queue, which is itself dequeued by a small number of worker tasks, and another queue which communicates results back from the workers to the front end, or some variation on this theme. But that is not the subject matter for today's rant.

Notes:

1. By "better" I mean "faster", of course. Some would argue that other factors matter, but for the sake of this discussion, let's pretend that they don't.
2. Purists will note that one does not speak of the speed of a language, but rather of its implementation. So replace "language" with "implementation" in the rest of this discussion. Note that the run logs in the source archive have a blurb about the implementation (version) of each language used.
3. Technically, C of course has threads. However, by the time you replace various and sundry runtime library calls with the reentrant versions, it might as well be another language. Put another way, I was too lazy to bother with threaded C.
4. Some would argue that green threads, such as those used by Perl and Ruby, are not real threads. While they do not do much to exploit multiprocessor systems, they do at least let you handle tasks asynchronously.
5. You could use the Java Native Interface to implement a call to fork (or download a library that already does it). But as soon as you use JNI, it's not really 100% pure Java anymore, is it? And, it would not work on Windows (which everybody needs?). And, the size of the JVM, combined with page table updates (due to copy-on-write) caused by the GC, would probably make it prohibitively slow anyway. Plus, I'm lazy anyway.
6. There are runs of 1; 5,001; 10,001 and 20,001 iterations in the archived run logs as well, so as to allow comparison of start up times, as well as to see how linear the time taken per iteration was.
7. Again, "better" meaning "faster", not better meaning conceptually simple.