Links for June 27, 2014

  • OpenTuner by jansel "OpenTuner is a new framework for building domain-specific multi-objective program autotuners. OpenTuner supports fully customizable configuration representations, an extensible technique representation to allow for domain-specific techniques, and an easy to use interface for communicating with the tuned program. A key capability inside OpenTuner is the use of ensembles of disparate search techniques simultaneously, techniques which perform well will receive larger testing budgets and techniques which perform poorly will be disabled." -- This looks really useful. I started on something like this long ago at LLNL, but since I was so young I focused mostly on fancy plots and designing a language for describing experiments.

  • Paper to Plants on Vimeo A really cute video about a game that seems pretty charming. I'm still not sold on little kids using iPads so much, I'm told it's bad for their eye development.

  • Ergonomic Workspace Planner Tool | ComputingComfort.org Use this to figure out the optimal height of your standing desk

  • pollen Write web based books in racket

  • Finding the perfect house using open data — dealloc.me A guy builds a map of available houses in Portland that match his desires. Seems like a good real estate agent should do this for you - but how do you know you've got a good one? I guess you have to write some code.

  • Drawing Presentable Trees "When I needed to draw some trees for a project I was doing, I assumed that there would be a classic, easy algorithm for drawing neat trees. What I found instead was much more interesting: not only is tree layout an NP-complete problem1, but there is a long and interesting history behind tree-drawing algorithms. I will use the history of tree drawing algorithms to introduce central concepts one at a time, using each of them to build up to a complete O(n) algorithm for drawing attractive diagrams of trees." -- I've always wondered what a good way to do this would be. Knew it had to be a solved problem.

  • Grain-Free Oatmeal Raisin Cookies | Against All Grain - Delectable paleo recipes to eat & feel great If you want oatmeal-raisin but you're trying to avoid oats and sugar, these are really quite good. The coconut makes a great texture.

  • Lev Grossman on his daughter, Lily: How being a father ruined my life and made me a better writer. A really heartfelt story about becoming a dad and getting your act together. Now that I have kids I'm a total sucker for this kind of article.

  • HAL :: [inria-00555588, version 1] A comprehensive study of Convergent and Commutative Replicated Data Types Formal exploration of sync-able data types. Abstract: "Eventual consistency aims to ensure that replicas of some mutable shared object converge without foreground synchronisation. Previous approaches to eventual consistency are ad-hoc and error-prone. We study a principled approach: to base the design of shared data types on some simple formal conditions that are sufficient to guarantee eventual consistency. We call these types Convergent or Commutative Replicated Data Types (CRDTs). This paper formalises asynchronous object replication, either state based or operation based, and provides a sufficient condition appropriate for each case. It describes several useful CRDTs, including container data types supporting both \add and \remove operations with clean semantics, and more complex types such as graphs, montonic DAGs, and sequences. It discusses some properties needed to implement non-trivial CRDTs."

  • What I Wish I Knew When Learning Haskell 2.1 ( Stephen Diehl ) Nice quick article with practical tips for beginners.

  • Not So Much 'New York Poor' as 'Pittsburgh Rich' - Pacific Standard: The Science of Society You can get a lot for your money in Pittsburgh, and lots of other places throughout the US. I'm from Pgh, and have fond memories. I'd consider moving back there if they could fix the weather.

  • Call me maybe: Elasticsearch Part of a series of irreverent but thorough explorations of various popular distributed systems.

  • What Do We Save When We Save the Internet So as you proceed with your protests, I wonder if you might also ask, quietly, to yourself even, what new growth might erupt if we let the Internet as we know it burn. Shouldn't we at least ponder the question? Perhaps we’d be better off tolerating the venial regret of having lost something than suffering the mortal regret of enduring it."

  • Music Animation Machine — "Music Worth Watching" Old stuff, but great. Couldn't believe I hadn't bookmarked it long ago.

  • Moving from Octopress to Nikola | serialized.net This post and a few tweaks got me into nikola without much hassle.

  • Flashing a Nexus device with Ubuntu Touch from Mac OS X

    I've been working with Ubuntu Touch recently, using VirtualBox to run Ubuntu Saucy as a guest on my Mac, and while it does work to connect to the device once it's installed, the process of reflashing is difficult. The manual steps on the Ubuntu Wiki Touch/Install page work for the old way, via phablet-flash cdimage-touch, with the caveat that you may have to manually reboot the device and/or re-connect the USB in between steps.

    However, as of this email last week, the official way to flash phones is using the system images via phablet-flash ubuntu-system, which has a different set of files.

    Instead of just muddling through and updating the manual instructions, I decided to see if phablet-flash would run natively on the Mac. After a bit of python dependency resolution, it does. Here's what I did:

    1. Download the Android SDK tools. You only really need the "SDK Tools", not the whole ADT Bundle, so I clicked on "Use an existing IDE" at the bottom of that page, to get the smaller installer.
    2. Unpack the archive and run ./android. Install the platform-tools (and nothing else).
    3. Add the platform-tools/ directory to your $PATH for convenience.

    4. (Optional) create a virtualenv for the python dependencies, and activate it.

    5. pip install each of configobj pyxdg, and pyliblzma
    6. get bazaar if you don't already have it.
    7. branch phablet-tools: bzr branch lp:phablet-tools
    8. cd phablet-tools && python setup.py build
    9. At this point, phablet-flash ubuntu-system should work smoothly, with all transfers and reboots and timeouts going as expected.

    The only minor hiccup is that it apparently doesn't get ~/Downloads/ from pyxdg, but it just means that you'll get the images saved to ~/phablet-flash instead of ~/Downloads/phablet-flash. No big deal.

    Links: Simulation, Programming, Crab Cakes and Hockey Player Usage Charts

  • DYNAMO Someone has rewritten one of the earliest simulation systems in JavaScript (the fate of all interesting software). Also includes a link to an article about the history of simulation software that sounds very interesting.
  • The Food Lab: The Crabbiest Crab Cakes I love crab cakes, but I'm not sure I really want to try to make them at home. If I do, I'll use these tips. The Food Lab is fun stuff.
  • Debug It! A review of a book on debugging, which is a topic that I think should be taught right alongside programming. See also "Why Programs Fail"
  • GiveDirectly: introducing a radical new way to give! | GiveDirectly Send cash straight to poor people. If their assertions are true, it's a really interesting idea, and I can't believe it hasn't been done before. It also seems transparently better than microloans.
  • ContinuumIO/Bokeh · GitHub Something to look out for -- a Python ggplot that works with HTML5 is a great idea. "Bokeh (pronounced boh-Kay) is an implementation of Grammar of Graphics for Python, that also supports the customized rendering flexibility of Protovis and d3. Although it is a Python library, its primary output backend is HTML5 Canvas. There are many excellent plotting packages for Python, but they generally do not optimize for the particular needs of statistical plotting (easy faceting, bulk application of aesthetic and visual parameters across categorical variables, pleasing default color palettes for categorical data, etc.). The goal of Bokeh is to provide a compelling Python equivalent of ggplot in R."
  • FitDesk X1 Level up from a standing desk? I'd love to try this for a day.
  • Concurrent Revisions DVCS-like concurrent programming. Interesting sounding research - I haven't read it yet...
  • Many thanks to @robvollmannhl and the good folks at Hockey Abstract for these great interactive Player Usage Charts: hockeyabstract.com/playerusagecha… Player Usage Charts are fascinating, but I can never figure out why people always change the axes so that the dots fill the space. It makes it impossible to compare two charts, and it's not obvious, so you end up comparing charts without realizing that it's meaningless.
  • Mock nested properties with python-mock

    Python's mock library (part of stdlib in 3.3+) is a great tool for writing concise tests. Its documentation is very good, and rewards multiple reads - but I found one thing that wasn't totally clear, even after looking through a few times. I wanted to use PropertyMock to mock nested Properties. Specifically, I had patched the python Gnome-introspection wrapper for libsoup at the top level Soup objcet, and I also wanted to replace one of its nested constant properties, Soup.MemoryUse.COPY with a sentinel that I controlled, for later comparison.

    The idiom for PropertyMock is to assign a PropertyMock to the type object of the Mock object whose property you want control of. What I found is that because Mocks auto-create properties, it's possible to do nested mocking in one line, like this:

    import mock
    m = mock.Mock()
    type(m.a.b.c).d = mock.PropertyMock(return_value = mock.sentinel.my_value)
    assert(m.a.b.c.d == mock.sentinel.my_value)
    

    So my soup example looks roughly like this (mixing testing and tested code, and repeating literals for brevity):

    json_body = "{}"
    with patch(gi.repository.Soup) as mock_soup:
        from gi.repository import Soup
        type(mock_soup.MemoryUse).COPY = mock.PropertyMock(return_value=mock.sentinel.COPY)
    
        # tested code:
        message = Soup.Message.new("POST", "http://fake.com/api")
        message.set_request('application/json', Soup.MemoryUse.COPY, json_body, len(json_body))
    
        # checking:
        assert(mock_soup.mock_calls == [mock.call.Message.new("POST", "http://fake.com/api"), 
                                        mock.call.Message.new().set_request('application/json', 
                                                                            sentinel.COPY # <--- this was the point
                                                                            "{}", 2)])
    

    It's often possible to think of a shorter, clearer use of the mock library after revisiting a problem, but so far this still seems good. Let me know in the comments if you have a suggestion for improvements.

    Don't Snooze

    Mailbox is a new iPhone email app that grabbed a lot of attention recently, partly because of their waiting list for using it. It's a gmail-only system that stores copies of your email on Mailbox's servers, so they can do new features beyond the standard mail client. Even though I got through the waiting list a week ago, I haven't tried it myself, because I wasn't

    I have a few religious beliefs about email. One is that processing incoming email is different from

    that you should only process a message once - when you first see it, you should decide what to do next with it, even if that's "read it again later",

    I call it a religion because I believe it, but I don't always practice it. I have an inbox full of things I've looked at once and am procrastinating on, partly because it's hard to move things to the todo list that I use from Gmail. It annoys me every time I look at my email, and I've forgotten important things that I left sitting there. This is exactly why I think it's important that our tools encourage good habits, instead of encoding bad ones.

    Mac IRC clients

    At Canonical, we are spread all over and keep in touch via IRC. I've tried out a bunch of IRC clients, but nothing is quite right. I'll explain. Here's a list of some things I want from a client - it's probably not complete, but it'll do for now:

    1. Don't crash. A bouncer makes crashes recoverable, but I shouldn't have to use one just to keep the backlog around.

    2. I need to search the backlog. Incremental search is best.

    3. Show multiple channels at once, in one window.

    4. Highlight mentions of my nick so I can glance at a window and see if anyone's asking me something. Use Growl or notification center, too.

    5. Make it obvious when I've disconnected, and reconnect robustly.

    6. Let me hide/fade out join/leave notifications and other admin messages.

    7. Keep my place so I know what messages are new.

    8. Make it obvious which channel I'm looking at, and even more obvious which channel I'm about to type in. (Nothing is great at this)

    9. Spell-check is nice, but autocorrect is awful for IRC and technical discussions. Needs a global switch.

    10. Don't show unread counts. Some channels are chatty and I'm just in them listening for my name.

    Here are the clients I've tried and why they aren't quite perfect:

    Colloquy

    Colloquy is a nice full-featured open source client. I've used it the most. However - it doesn't show multiple channels in one window at the same time, and doesn't keep your place. I've had some problems with reconnection, where I thought IRC was quiet when I was actually just disconnected and didn't know it.

    I have to turn autocorrect off for each room I'm in, which I only remember after the first time it really annoys me by mangling some jargon.

    There's also some general buggy behavior, and I have a short list of bugs I'd like to report someday, but I don't have time to fix them.

    Years ago I hacked my own version of a place marker that I liked, but I can't remember why it didn't make it in, and of course the code is lost now.

    The backlog search looks like it should be powerful, but in my experience it just doesn't work - it never displays any results. I'm not sure what the deal is there.

    (As a side note, Colloquy's iPhone app is the only iPhone IRC client I've used. It seems to work fine, and I don't use it enough to have specific gripes. Just having to have IRC open on my phone is guaranteed to annoy me more than the software itself could.)

    Emacs ERC

    My current setup is just to run ERC in an emacs window alongside my code editing. This has super flexible window splitting and size management and unbeatable search for your backlog, since it's just another emacs buffer. There's another nice benefit - if you're writing about code, emacs' tab-completion for strings across all buffers means that you can easily autocomplete SomeReallyLongFunctionName in chats. This saves a lot of typing and cut n' pasting. When you do need to cut n' paste, it's nicer to do it between emacs buffers than between apps.

    The defaults are not ideal, as usual with emacs you need to dig around to find a good setup.

    There's a module to colorize nicknames according to a hash of the name, so it's easier to tell people apart. Other clients have similar features but I think I like emacs' the best for some reason - maybe the palette is nicer.

    There's a module to mark your place that only works if you left the cursor in the backlog, which is a little weird. It will highlight my nick when mentioned, but not the whole line.

    Since it's open source, I have the same problem as Colloquy, it's hackable, but I don't really have time for that. And emacs is fun to hack on, so it's a potentially dangerous time sink.

    Linkinus

    This is a nice client with some minor flaws and a little flakiness. I liked it but moved on because of minor annoyances.

    It has a kind-of nifty "conversation tracking" feature that tries to highlight messages in a back-and-forth when you hover over it with the mouse (but only in the currently selected channel). It's not really that useful, but I think it could be if pushed further. I'd like to see more clients try this kind of thing, since separating multiple concurrent conversations is hard on IRC, and it can be hard to avoid them.

    Lets you display multiple channels in one window, but that needs some improvement. There's just one text-entry field so it's easy to type something to the wrong channel.

    The split display has some issues. The split channel views don't show you the channel name anywhere, and in some cases don't even show you the channel topic, which may or may not be helpful anyway. So it can take some effort to figure out what channel you're looking at, especially if there are many of the same people talking on two channels.

    It can save a set of channels as a group, but it doesn't save the relative sizes of the splits in the group, so if you want to flip between groups, you'd better be OK with even splits.

    Finally, it only supports vertical splits, so with more than two channels, you just get a couple unreadably long lines per channel.

    It has a very barebones backlog search, but it does work.

    They've also clearly tried to re-design the account setup / prefs experience, but I'm not a big fan of an inspector palette - I wanted to compare settings between two servers and it will only show me one at a time. This was also the biggest point of flakiness, where some edits wouldn't commit and I couldn't tell if the settings were being changed.

    Textual

    Textual is an open-source app (github.com/Codeux/Textual) that is for sale on the Mac app store. This is an interesting approach that I think has some merit - people may still contribute code or bug fixes, but it's also nice to know that if the app ever gets abandoned, it could be resurrected. That said, I tried building from source to enable a quick hack once but the master branch didn't build for me, so they're not trying to make it easy.

    It lets me hide some admin messages, has automatic & manual scrollback markers to keep track of what I've missed.

    This has a setting to "Track conversations using nickname highlighting" but I couldn't really tell what it was doing.

    This also has a pretty barebones search, but again, it works.

    I tried this for a few days and liked it, but ultimately switched away, I think mostly because you can't display multiple channels - there's just one window and it doesn't split up.

    Mango

    I haven't tried Mango. The screenshots in the App Store and feature listings don't lead me to believe that it's anything significantly different from the others. I'd be glad to hear from fans, though.

    So, what do you use? Have I missed anything great?

    More out of date links

  • Programming Languages · Hammer Principle Very interesting survey site to answer the question "what language is right for what task?"
  • One Div Zero: Types à la Chart A chart of languages according to type power/safety. Seems reasonable. Made me curious about the ones in the top right.
  • Home · philsquared/Catch Wiki C++ unit testing framework, all in headers, looks nice.
  • ferd.ca -> An Open Letter to the Erlang Beginner (or Onlooker) Interesting, balanced article about what makes erlang nice.
  • "Research is a blind date with knowledge." worrydream.com/oatmeal/blind.…

  • Splitting RPython (the language) and PyPy (a python interpreter) plan: mail.python.org/pipermail/pypy… Sounds interesting. I wonder if RPython will grow in popularity separately.

  • Double-corn tortillas I'd love to try this sometime.

  • clips/pattern · GitHub Web-mining in python. Lots of functionality here, BSD license.

  • The Art of Coffee: A Mad Men Era Short Film | Brain Pickings I loved "This is Coffee!". 12 minute promotional short film from 1961 about coffee. It's wonderful to imagine the worn old mugs of my grandparents' generation new again, filled with something better than the electric-percolated canned junk I know they drank. Also great to see the Chemex in there, exactly the same as I use it today.

  • Learnable Programming Very clear and well-argued essay about how to make a more-understandable programming environment. I agreed with everything, and was left wondering how to apply this line of thought to more complex software. (Everyone has seen this by the time I post this list.)
  • @objc.signature(‘v@:@@o^@’) | Times New Rohan List of objc.signature codes
  • Links

  • What's so bad about Template Haskell? - Stack Overflow

    A bunch of good answers to why Template Haskell is occasionally maligned.

  • Which Haskell (GHC) extensions should users use/avoid? - Stack Overflow

    Interesting list of aesthetic judgements of haskell extensions. At press time I don't know what most of these words mean.

  • Yesod and Template Haskell

    Short post on why Yesod uses template haskell (which has a bad rep)

  • deck.js » Modern HTML Presentations

    Best "Write your presentations in HTML" thing I've seen yet, although I'm not sure how great it'd be at complex visuals or builds. Yes sometimes builds are good! SOMETIMES.

  • REDbot

    RED is a robot that checks HTTP resources to see how they'll behave, pointing out common problems and suggesting improvements. Although it is not a HTTP conformance tester, it can find a number of HTTP-related issues.

  • .@dpp’s Visi looks to be exactly the thing when Soulver isn’t enough and spreadsheets are rigid github.com/visi-lang/visi #emerginglangs

    Should I look at this before posting a link? Nah... But I will look at it, eventually! It looks interesting, I love this kind of stuff - although as usual I wonder if this is something you can already get in Mathematica but no one knows because it's so expensive.

  • High Scalability - Prismatic Architecture - Using Machine Learning on Social Networks to Figure Out What You Should Read on the Web 

    Probably interesting? I skimmed this, I'm not gonna lie, I don't remember much.

  • Deslidefied HTML

    Really useful bookmark to make slideshow news articles readable on one page. I am using it all the time!

  • Thinking Functionally with Haskell

    ""In which we explore what modern type systems bring to the table. Imagine an approach to programming where you write down some description of what your code should do, then before running your code you run some automatic tool to see if the code matches the description. That’s Test-driven development, you say! Actually, this is what you are doing when you use static types in most languages too. Types are a description of the code’s inputs and outputs, and the check ensures that inputs and outputs match up and are used consistently. Modern type systems—such as in Haskell or above—are very flexible, and allow these descriptions to be quite detailed; plus they are not too obtrusive in use and often very helpful. One point I’ll investigate here is how advances in types are converging with new ideas on testing, to the point where (I claim) the old distinctions are starting to blur and starting to open up exciting new possibilities—hence my suggestion that we need a new word to describe what we’re doing that is free from preconceptions and out-dated thinking. So put aside your bad experiences from Java, and prepare to be amazed! ""

  • So you want to hack on IMAP…

    "Well, first off, you’re horribly misinformed: you do not actually want to hack on IMAP." (I agree, IMAP bad.)

  • The Machine That Changed the World: Great Brains - Waxy.org

    Documentary about computing history: "It's a whirlwind tour of computing before the Web, with brilliant archival footage and interviews with key players — several of whom passed away since the filming. Jointly produced by WGBH Boston and the BBC, it originally aired in the UK as The Dream Machine before its U.S. premiere in January 1992. Its broadcast was accompanied by a book co-written by the documentary's producer Jon Palfreman."

  • A Testing Story

    Tests are great! This year I've started working on a codebase with a significant unit test suite for the first time in a while, and good coverage has definitely come in handy. Now I'm going to share a story where a single test did double duty.

    Our filesystem events listening daemon was crashing occasionally on my system, and I didn't know how to reproduce it. No one else seemed to be getting these crashes, either. The system was saving a backtrace, and it was always the same, but I wasn't sure I believed its line numbers. As a start, I made a guess at the lines that were really failing, and added a ton of debug dumps to inspect the state. (Since this daemon is run as root using launchd, it's still easiest to just use the old printf-n'-stare debug method.)

    Then I tried a lot of stuff to poke at it, even leaving Spotify on overnight in an attempt to recreate the conditions of the bug. No luck. It seemed like it would only crash when I wasn't trying to get it to crash - pretty frustrating.

    I finally found it, but only after giving up for a while. I checked in again after working for a while on another project, and hey, lots of new crashes! With all my extra debug info, I could see what was going on - a string that couldn't be encoded in UTF-8 was being handled by some code that assumed it could be. It was a filesystem path with invalid characters.

    What was the path that was killing my daemon? It was a temp file written by the test suite for the other project. It was a non-utf8 path, written to test the unicode handling of the GUI, and it had the wonderful (in retrospect) side effect of poking a bug in the daemon too. It's so satisfying when you find a bug's cause and it completely explains all the symptoms you were seeing.

    One test exercising the unicode handling of multiple projects, now that's coverage!

    disposable

    Computing gadgets don't last long enough. I wanted to write a lot about this, but since I don't have good answers, I figured I might as well just pose some questions.

    All the time, we're throwing out something that still works, because there's something new that works better.

    Working better is great! Faster chips and better screens and cameras make our lives easier and help us make cool new stuff.

    I used to work near people designing processors, so I know it's plenty hard just to keep that part of the system progressing. And I know that newer stuff will use less energy, which is great. I know that people designing data centers think hard about their efficiency, and upgrade ruthlessly to improve it.

    But how carefully are we considering the impact of getting rid of all that old equipment? How do we compare whether it's really, globally, ethically, better to improve speed & efficiency vs. just keep using old stuff? How do we factor in things like resources mined in conflict zones?

    I recently remodeled a house, and among all the advice I read, I remember this best: "The greenest building material is the one that's already there." (It was about floors. We put in new bamboo anyway. That old parquet was awful, but I hope the new stuff stays there for a hundred years or more.)

    I know that designing e.g., a great modern smartphone, is extremely difficult, even 'only' considering functionality, space and power constraints. Designing something that small and integrated that could still be upgraded partially, keeping around parts that still work, sounds near impossible, but wouldn't it be great? It sounds like a fun challenge.