I have brought all the pieces together today in yet another IronCoding session. I discovered two things:

1. Distribution doesn’t mean raw speed improvement. At least not on the scale of my test files. Running my parser over a couple of machines is slower – so far, but I am waiting to see what’ll happen with the monster files –than on just one.
   Which probably means I’d better get a dual core, very very fast machine. nnnyessss… /me looks at wallet… nnnnnoooooo….
2. Erlang *is* powerful. 110 lines of code are enough to set everything [concurrency, distribution, tokenization of strings, centralized sql output to a file] up.

On my TiBook, it is a tad slow – besides, it is a busy machine – but on the dedicated server we have with a friend at OVH, which is pretty much underused, a puny Celeron @2.6GHz with half a gig of RAM, it parsed 1,010 complex records [30,000 lines] in a little over 1 second. I tried there too to run it distributed over two local nodes, no dice. 10 seconds or so. sigh…

Now, I need to dump that into an sqlite database. I do. Don’t ask. Any ideas? Google and #erlang weren’t helpful…

Erlang

This entry was posted on Thursday, March 23rd, 2006 at 10:52 pm and is filed under Code, My Kit, Erlang. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.