I have been cooperating on and off with the good people from Dot.Tunes, based in Australia. I have coded bits of the application, notably the iTunes Library XML file import engine. Back when they integrated my engine, D-T saw huge improvements while importing the XML database, especially with large files. If memory serves, the previous import engine stalled on some files. With the first two or three version of my code, large files took some time to import, but at least D-T didn’t fail. When I mean large, I mean real large. Like 50,000 record-large. 35MB+ of XML to digest and convert to sqlite. Gulp.

50K records took, give or take, 20 minutes. Note the use of past tense. Took. Now, it takes 3 minutes. Yessir. On the same machine. But to achieve that I had to break out the big guns. The real big ‘uns. I had been playing for a while with the idea of writing some iTunes/mp3/etc related code in Erlang and possibly Yaws. So one of the things I started writing is a new engine for the iTunes Library XML format, using message passing, concurrency and gazillions of lightweight threads.

The result is astounding. 2,000 records take 2 seconds on my MacBook, and/or a late-model PowerMac G5. 50,000 records take three minutes on that same PowerMac G5 – I don’t have yet the file to make the tests, and I am quite curious to see how the MacBook will fare. To be fair, there’s extra overhead then to inject the .sql file into an sqlite database, but it’s minimal. The only problem is that we’ll need to ship CEAN, a stand-alone version of Erlang. That adds +/-12MB to the package, but I think it’s well worth it.

We’re still in the testing stages, and Jeff over at D-T will certainly make announcements when we’re ready, but I am already quite excited by the progress we’ve made….

I have written over the last two days RB bindings for libgd, in the continous search for a replacement to the Picture object, which is missing in console apps. As long as the promised-and-long-forgotten SwordFish project won’t be released, there’ll be little need, I think, or more accurately little clamoring for a Picture object – but once people start writing web apps in RB, you bet your ass they’ll want to produce images on the fly, like everybody else…

So. GD’s one option I had been thinking about – while still thinking I could build my own. And fPic works well enough, albeit with quirks. And it covers lots of ground – including PDFs, thanks to CoreGraphics. But it’s not very portable – my attempts to make it work on Linux, sans PDF and vImage.framework, failed so far. Since gd works on all platforms, I thought I could try it out. And sure enough it was easy to write bindings to the library, with very few quirks – I am having problems with saving to wbmp format, but who cares about that format anyway?

One format that’s missing in GD is TIFF – which fPic handles very well, na na na – so I re-read fPics Tiff code [a wrapper to libtiff really] and converted that to RB. Which means I ended up writing bindings to libgd *and* libtiff, oh so much fun! It only reads TIFF files for the moment, as I haven’t worked out how to get access to the int **tpixels array from RB, in order to save as Tiff. Might have to write a small dylib to go with it – or use fPic’s very own manip.dylib!

Like in fPic, when using freetype2 to draw text, one has to pass the path to a .ttf font file. It’s not exactly user-friendly, so I am trying to find a way to provide for a better interface into that. Maybe a Font class that’ll take care of the nitty gritty work. I found a good code sample to retrieve the name of a ttf font. Could be useful…

Understanding Classes and Objects in REALbasic, by Mark Choate, a regular poster on the RB Nug, and author of REALbasic Cross-Platform Application Development – which I haven’t read yet. This is actually a chapter from the book.

I built a functioning UM in around four hours, plus some debugging and tweaking to help me understand more what the code does. Step by step debugger and all the trimmings. Nifty.

But… I can’t get past the decryption level. I am doomed. Oh well, just saved my week-end I guess!

Haven’t decided yet whether to join this year’s ICFP contest – will have to see the task. And even so – which language? Ah well… Erlang would be cool, even if I am not yet up to speed on it. RB could work – after all this is one language I am most comfortable in. But Functional it is not

Weird, innit? This is from the codex posted yesterday, I think.
There’s a lot of gobbledygook in there, including a Gif89a header. Waiddaminit! This I know from my fPic efforts. RB sure came in handy, to extract the gif file. Never mind the jpeg above, what I did was remove the 0×1a73 first bytes, and save that as a Gif. Since you can stuff anything and everything after a valid gif without breaking it – it will just be ignored, which is, among others, a crude way to perform steganography… – the file displayed alright, but was still 2.2MB heavy… Screenshot, conversion from PDF to JPG – with an RB app, of course! – and here we go.

The theme is supposed to be computational archaeolinguistics. Read at face value – I am alas very good at that – it is right down my field of [supposed] expertise. Reading between the lines, as some #erlang fellows mentioned, it could be that the computational is not the means – doing linguistic research with ‘puters – but rather digging an old computer language, or else:

<marc_vw>	it is probably about data mining
<marc_vw>	maybe looking at computation results
<marc_vw>	and trying to figure out what language the hardware used was
		programmed in
<marc_vw>	my guess is reengineering a dead computer language
<noss>		mine too, evaluating some kind of program specification.

Which makes more sense… Other tidbits include Latin ignoti et quasi occulti ie “unknown and almost hidden”, English, including written from right to left: welldonedaedsi luap “Well done, Paul is dead”, several appearances of the word apply, which could be a computer command, a magical word – abracadabra – and other meaningless stuff.

Four more hours to go.