Mobile application development middleware?

I am the author of an ExtJS based cluster management web-application (Scyld IMF at Penguin Computing) that allows controlling a high performance compute cluster from any web-browser, be it from your desktop or from a mobile device. This works fairly well already even without writing a special view for the mobile devices as current smartphones (iphone, android) come essentially with a full blown javascript capable web-browsers and lots of processing power. However performance still could be better by having a native app that uses the same data sources / api as the full blown web-application (javascript app running on the browser being the client to the webservice-api on the server side), but implementing the view natively.

After hacking up a proof of concept for the iphone (thanks John for your awesome iphone JSON flickr tutorial and xcode example for download) and thinking of how to do the same for android without duplicating effort, I realized that it would be nice to be able to have some kind of middleware that allows me to create and populate native gui components without code duplication.

One such middleware is phonegap which requires you to write your application in javascript and html, and allows accessing phone specific features through javascript calling the middleware. It then is bundled as a native application that can be submitted to the app store for purchase and can run completely offline once downloaded, unless the programmer chooses to access remote resources. Their initial focus seems to have been allowing access to smartphone specific features like the address book, vibrate and sound, gps location and such, but there also is code in there to use native gui components as well.

I have two problems with the approach:

1. development of javascript – while palm pre developers might tout this as an advantage, I find javascript development and debugging rather tedious and would prefer writing my code in java or objective-c and be able to run it through a full control debugger on my desktop before deploying it to the simulator. I know about and am actively using firebug, the javascript debugger for firefox, but a lot of the bugs I had to deal with were incredibly hard to hunt down and rather than the debugger it was the forums and interacting with developers on #extjs chat, as well as using jslint and manually analyzing code, that helped me figure out a root-cause, workaround or fix.

2. performance – developers report that the phonegap version of their app appears to consume more resources rather than less compared to a remotely hosted javascript app, accessed through the mobile devices web-browser. Apart from the overhead of having to parse javascript at runtime, developers sometimes need platform specific knowledge and implement different than for a web-application, i.e. iphone onclick events being much slower from the touchscreen than ontouch events.

The advantage on the other hand appears to be that it is relatively easy to add new mobile platforms to phonegap. So far they appear to have iphone, android, blackberry and some nokia platforms covered, with palm pre being on the horizon to be added beginning of next year.

And according to phonegaps website, there are already several real-world phonegap powered apps in the marketplaces of both the iphone and android.



more ec2 playtime

Turns out it is much easier to launch fresh images and configure them on the fly than to keep doing iterations of download, unbundle, mount, customize, unmount, bundle, upload… also I don’t really like that a command line tool has to invoke a jvm to get simple things done.

The perfect tool for this is aws which is a standalone (mostly, invokes curl) script allows you to command ec2 and s3 with minimum of configuration / parameters. (key and cert go into a single file .awsecret and thats it).

Now if I want to start 20 instances of my cluster nodes, I can do that with a shell script I wrote which invokes aws to start the instances, waits for them to become available, and then deploys the necessary pieces to them and distributes the /etc/hosts so all nodes know about each other.

Penguin releases HPC as a service

If you want to know what all the hype around diskless beowulf clusters is about and run it on your own 64bit x86_64 hardware, you can now get a free trial from

UPDATE: If you are not ready to buy the hardware, or have a run that is larger than your current cluster, take advantage of Penguin On Demand (POD) which is the latest cool thing: HPC as a Service, low latency interconnect and HPC specific support…

large linux cluster

large linux cluster

linux ramdisks – howto pick the right kernel parameters to create 800MB large ramdisks…

To play with the application ansys / fluent doing an out-core solving process that requires fast scratch space, I was attempting to combine several cluster nodes ram-disks through lustre.

The surprise here is two-fold: With kernel 2.6 the old ramdisk-parameter is obsoleted by the newer ramdisk_size and ramdisk_blocksize. The other one is that by default, it fails in a bizarre way:

mkfs seems happy, but mount says “mount: wrong fs type, bad option, bad superblock on /dev/ramdisk, or too many mounted file systems”. A peek in the output of dmesg told me:

EXT2-fs: Magic mismatch, very weird !

Very wierd indeed. I was puzzled, but thanks to I got to the bottom without having to read the kernel source: He ran into the same issue and figured out that it was the blocksize and solved it by specify the blocksize of the filesystem to be 1k instead of the default 4k.

I now use the kernel parameters ‘ramdisk_size=819200 ramdisk_blocksize=4096’ instead since all data I want to store will be in very few large files anyways and so the 4k blocksize is more efficient and more friendly to the lustre tools anyways…

Note that the ramdisk_size is in KB whereas the blocksize is in bytes.