#021

Open Source

I built Adjoint’s libraft tests against dejafu-2.0.0.0, with a minimal diff to the package.yaml and stack.yaml, and the tests compiled and ran just fine. So despite changing the fundamental abstraction in dejafu and the types of all the testing functions, I managed to avoid any breaking changes in the common case. Nice!
I wrote a migration guide for dejafu 1.x to 2.x. All that remains now is to review the rest of the documentation.

Ph.D

I decided to do the optional correction the examiners gave me, which was to talk about related work by Kostis Sagonas. I looked through his recent work, and found two papers which looked pretty relevant:
- Systematic Testing for Detecting Concurrency Errors in Erlang Programs, which I suspect is the related work they wanted me to talk about, which describes a tool for systematically testing concurrent Erlang programs. It’s similar to Pulse, another Erlang concurrency testing tool, but uses iterative pre-emption bounding to explore schedules, rather than random exploration. Unlike dejafu, it doesn’t use dynamic partial-order reduction, so in principle it will explore more executions than necessary.
  
  Interestingly, this and dejafu both use the same optimisation: avoiding scheduling threads which will immediately block without updating any shared state. If a thread does that, then there’ll be a context switch immediately, so there’s no point in scheduling the thread. Just schedule the thread which will be context switched to instead. I don’t think I read about this idea elsewhere (and I didn’t consider it interesting enough to mention in my thesis until I read about it here), so it’s a case of independent discovery.
- Optimal Dynamic Partial Order Reduction with Observers, which really reminds me of some other related work I’ve talked about in my thesis called “Maximal Causality Reduction”. Maximal Causality Reduction and Optimal DPOR with Observers are both based on the insight that, if you have bunch of writes to the same shared state, their order only actually matters if there exists a read which will observe the ordering difference. This lets you prune a bunch of additional schedules on top of regular DPOR.
  
  It would be really nice to implement one of these algorithms in dejafu, but they both have quite strong restrictions on how threads can interact, and these restrictions don’t hold in Haskell. Maybe in the future!

Work

I set up a new non-manged elasticsearch 5 cluster in our integration environment, fixed a bunch of configuration issues which that revealed, and verified that updates from publishing applications are now making it across. Then I started looking at importing data from the elasticsearch 2 cluster—I expected that to just work, but it didn’t, so next week I’ll be trying to figure out why a few tens of thousands of documents didn’t get imported.
We continued working out the costs and benefits of a managed cluster vs a non-managed one. Initially we were in favour of a managed cluster, but the balance has shifted somewhat as we’ve looked into it a bit more. A managed cluster limits what you can do (you have to do things in the way the provider allows), and using a managed cluster still requires knowledge of elasticsearch in GOV.UK, to fix our apps when a new elasticsearch version makes a breaking API change. Our existing non-managed elasticsearch 2 cluster just works, and people barely need to touch it, so I have to wonder what exactly the benefits of a managed cluster are. In addition, we also have all the puppet and terraform written to provision a non-managed cluster, but we’d need to figure out how to set up a managed one and connect it to our apps.

Miscellaneous

I renewed my passport using the online service. I got my replacement passport in the same week I submitted the form, which was impressively fast.
I flavoured and bottled some pear and ginger kombucha I’ve had fermenting for a while. This one is using a mix of green and black tea, rather than puerh, and fermented a lot more slowly. I’m hoping that this will avoid the bad taste of my two previous batches.
My Call of Cthulhu campaign reached the thrilling conclusion of the current chapter. This was the first direct confrontation with the mythos the players have had since the prologue chapter, and the session involved using dynamite to blow up a bunch of magical zombies and a sorcerer. Their sanity greatly damaged, the investigators have set sail to London to follow the trail of clues.
I’ve been playing a lot of Runescape.

Open Source

Ph.D

Work

Miscellaneous

Link Roundup