Supercomputing 2006 BOF: "Is 99% Utilization of a Supercomputer a Good Thing?"

For those few readers who are interested in High Performance Computing and might be going to this year's Supercomputing conference in Tampa, my advisor Allan Snavely and Jeremy Kepner from Lincoln Labs are putting on a BOF with an intriguing subject: "Is 99% Utilization of a Supercomputer a Good Thing?"

It'll be on Thursday Nov. 16th at 12:15PM.

As with all really interesting questions, this has a quick, easy answer that reveals your point of view. My immediate answer is "No", because of the productivity problems the focus on high utilization causes for many HPC users, mainly through the use of a batch queue. However, when considering the interests of everyone involved (including the people who have to evaluate how money is spent on these systems), the only responsible way to answer is "it depends".

It's an interesting topic that never fails to generate some controversy, and from what I can tell we'll have some diverse speakers at the BOF (including myself, discussing user surveys we've conducted as part of DARPA HPCS). It should be enlightening.

Back it up

My Powerbook's been back for a couple of weeks now, and I had a good overall experience dealing with OC Data Recovery to get my data off the drive. Their San Diego Location is in Sorrento Valley. I didn't test their speed, since my main concern was cost, but they managed to get about 98% of my files back by their estimate, and my most important files are fine: my VoodooPad notes, my preferences, and my iTunes purchased music. Everything else is duplicated elsewhere - all my projects and research files, for instance - but some things change often enough that even weekly backups aren't often enough, like my notes in VoodooPad.

That's a relief, but the $2,200 cost should be a warning to anyone who hasn't taken the time to start a comprehensive backup strategy - if the idea of spending a few hundred bucks on backup media is stopping you, consider that a likely alternative is to wait and spend much, much more to get almost back to square one, without any way of knowing for sure which files are OK or even present until you need them later. That's not a good scene.

So back it up.

Finally, despite earlier reservations, my current backup scheme does use Apple's Backup 3 - I've tested it a few times recently and I'm comfortable with its restoring, and it's just simpler to use than Retrospect for some things like very frequent backups to .mac and backing up to multiple DVDs. Those programs are the only ones I own, so I can't say anything useful about alternatives. I've heard good things about other programs, and the cardinal rule is to use whatever makes it easiest for you to back up often, starting now, and don't worry about finding the perfect solution first.

Dear GMail spam filter

Dear spam filter,

I cannot read Russian. If a message is in Russian, it is spam, just like all the other Russian messages I have marked as spam.

Thank you for listening - your pal, mike

More Windows

Andy Ihnatko paints a colorful picture of Windows use. He describes Windows as first the boorish, greasy haired acquaintance on a long road-trip, then later as disposable underpants. I can't resist anthropomorphizations and descriptions of software as other daily objects.

I wasn't going to post that article, but just now I was keeping notes in Notepad and wanted to write something unrelated down quickly, so I press control-N. You know, "New". A dialog box pops up, and before I realize it, I've lost the notes I was keeping, because I accidentally hit "No" in response to "Do you want to save the changes?". OK, my mistake, but in my defense, I wasn't expecting that decision and didn't really read the dialog box, surely the twentieth dialog I've seen in two hours.

Also, why am I closing one file to open a new one? In Firefox, if I press control-N, I get a new window and my old one is still there. What's the deal, Notepad? Do most real Windows users just never use you?

I miss VoodooPad.

MyDreamApp Finalists

Thanks to Gus, I took a quick tour through the MyDreamApp finalists today, and voted for two of them, Portal and Blossom. Portal is a long shot - it's basically proposing to solve multi-computer file syncing seamlessly, but it would be great if done right. See Wil Shipley's comments for why it's hard. Merging is not easy.

Blossom is a great idea to link productivity nannying with emotional appeal - work hard and your desktop plant grows! It needs a new name, but with the right design and art, I think it'd be a hit. If they took a cue from the Nike+iPod multiplayer game, we could compete on productivity - make work a game and compete to see who can build the biggest, healthiest desktop plant. Brilliant!

Today's Windows User Fun Fact: I had to use Internet Explorer to see the entries on the MyDreamApp page - Firefox didn't show the entry details when I clicked on the pictures.

Windows User

I've been a Windows User for two weeks now. Since my powerbook disk died, I've been working on a loaner Dell Lattitude with XP.

I get by. It's not as hard as I would have guessed. Most of my work these days just needs a decent terminal, and SSH from (who knew there was a commercial SSH?) works OK. I don't understand why it doesn't save my keyboard preferences, but I muddle through.

I'm using pine for email, and the lack of SpamSieve is shockingly obvious. Sometimes control characters in spam emails garble the screen so badly that I can't tell what I'm deleting.

Otherwise, I use Firefox, and Google calendar keeps my schedule. So far, I haven't missed my iCal integration applescripts (new-todo-in-quicksilver, new-event-from-email, and new-todo-from-email), but I'm sure I will if this goes on longer.

I miss the years of notes I had inVoodooPad. In its place, I've been using emacs org-mode in a terminal. I like the outlining and todo/agenda gathering it can do, and wouldn't be opposed to similar features in VoodooPad. I'd actually been using it in the weeks before my forced switch, but just for the todos.

I've accidentally tried to print about a hundred times when editing text, expecting the old emacs shortcuts to be there. This is OK, though, because I can't configure printers.

There is a similar story for almost every key shortcut I want to use - I have no idea how to switch between tabs in Firefox, and when I did find out how to minimize windows from the keyboard (Alt-space, 'n'), I wished I hadn't, because it only works in half of the programs I use.

One thing Windows does is make me want to give up earlier. I actually just don't care if I can't figure out a good way to do something. This is an exciting new feeling - I just give up and get back to work, and each time, I feel a little more like a real grownup. You know, how you feel after all your youthful dreams have died.

I get more done at work with Windows. I no longer install programs just to try them out (scary!), or hack on minor projects (no friendly tools built-in). Even just surfing around is just not as tempting, and I do less of it, both at work and at home. In moments of weakness, I have considered swapping my PowerBook for three pound Windows laptop, just so I can finish sooner and with better posture. I know I could never do it, but the thought lingers.

Step back: why laptops?

I've been thinking of how I'd work if I didn't have a laptop. One thing's for sure: I wouldn't spend as much time rubbing my neck while waiting for builds, for a couple of reasons.

I'm beginning to wonder if a laptop is really any good at all, let alone necessary. Wouldn't I rather not carry that thing around all the time? Should my hands really sweat when my computer is working hard? Doesn't having a laptop just give me an excuse to pretend I'll be able keep working "later", even though that never really works? Does anyone really gain more productivity from working at a coffee shop than they would using a fast desktop computer?

I think the one thing my laptop is best at is blurring the line between fun and work, and not in a good way for either.

With the heat, weight, relatively poor performance, and especially the paranoia from carrying such an expensive thing around, I've grown to seriously resent my Powerbook, despite being at quite a loss without it.

Has anyone found a good way around the extreme dependence on heavy, fragile, hot-plate laptops, and still been able to get things done in more than one place? Anyone carrying Mac minis around between monitors? Carrying a disk between minis? Strict work/home division? What's your scheme?

Portable gut check

My Powerbook's disk died yesterday around 1pm. Since then I've been thinking about how to get by without it. After taking far too long to realize it was beyond my troubleshooting ability, I took it to the UTC Apple store at 5 or so, and got an appointment for 7:45, then went for a drink. When I came back, the Apple store guy referred me to drivesavers - I'll have to see what their estimate is when they call me back. For now, I assume I'll have to pay whatever they ask - I was careful about backing up notes and papers to my iDisk and keeping important work files in repositories on other machines, but there is still a lot of stuff on that machine that I can't replace, and my full-backup scheme was just inconvenient enough (and my target disk just small enough) that my full backup is pretty stale.

I can keep working for now on lab computers and my old home Mac - I'm pretty impressed with how much I can just transfer over to another mac and keep running - I can check all my mail (IMAP or GMail), read my NetNewsWire subscriptions (Newsgator Online), go through my bookmarks (iSync & .mac). Pretty impressive, even without any effort to stay flexible.

Here's hoping I can be back on track soon - for now I'm only a Mac user at nights, and I'm going to miss a few things during the days, to be sure.

AutoFill: BibDesk and DC-HTML

For my first contribution to BibDesk in a while, I've added the ability to read Dublin Core metadata when it is encoded in HTML META tags on a web page.

What this means is that when using the "New Publications from Web" feature, some sites you browse to will have the publication's information filled in for you, so you don't have to type anything at all. The open archive software does this, so check out their list of archives for examples to test it out on.

It'll be in the next version, which isn't scheduled yet, so if you'd like to try it out sooner, see the nightly builds page and heed its warnings.

If you publish web sites with one-page-per-pubcation and want info about embedding DC terms in your meta tags, see the Dublin Core recommendation.

If you want to support AutoFill for a site that doesn't have one page per publication, or would like to provide more metadata, I suggest waiting for the citation microformat. Feel free to ask why...

Update: I made a 12-second movie of how it works - BibDesk, EPrints and Dublin Core

TextStructure screencast

In case you had no idea what I was talking about in the last post, I've put together a hasty screencast with no script, no soundtrack, and no editing.

In the video you can see a few things:

  • 0:00: I have found a way to show you the palm trees on my street without showing you the rest of my street. Lucky you!
  • 0:08: After I press control-z (not my final choice for a keystroke), we get a nice little display of sections in your file. Right now I have it set up to show me ObjC methods.
  • 0:13: It shows you where you are.
  • 0:16: ... and where you've been.
  • 0:18: you can jump around by clicking on the section header list.
  • 0:30: you can search through the contents and it shows which sections match.
  • 0:38: you can also choose from a list of other ways to slice your text - described by a bunch of regexps in a file you can tweak without recompiling the plugin.
  • 0:49: I use Snapz Pro X and Quicksilver, which you know because I also tried to use iMovie HD and couldn't find the "Don't make my imported movies look like crap" option, so I just uploaded the original movie with no funny titles. Bummer.

Also, the leverage-discuss mailing list is up now, if you want to ask how to use it.