don't worry, it's probably fine

Notes from the Week #17

03 Feb 2019


Oh, it’s a long one. I’m trying another new format, breaking down by day - I often forget highlights in trying to limit myself to 2 or 3 things.


Going faster

I had a two hour session with the rest of the Team Leads about ways to help us go faster, within the constraints of keeping the Unruly culture that makes us unique and not over-egging the process pudding (so to speak).

It feels a lot like a linear/combinatorial optimisation problem that I learned about in school and uni respectively (I’m shuddering at the memory of manually running the Simplex algorithm during exams). There are a bunch of different levers we are able to pull but every action has an effect on everything else.

This probably falls somewhere in what Cynefin calls the complex domain, and we improve by Probing -> Sensing -> Responding.

Demo time

Shift have also started doing huddles to demonstrate our little pet projects since we’ve been trialling a flexible working system. I’m a self-identified morning person and I like to tinker when I’m in the office on my own.

This week I demo’d a custom Terraform module to wrap up different resources between GitHub, AWS, and other sources. Modules support multiple providers being passed in as attributes since 0.11, so this is no longer problematic to wire up.

Stephen demo’d a spike of a Slack app to administer resources on AWS, as well as communicate to users when they have stale assets, building on his 20% from last week



I’m deliberately very free-and-easy with our team’s process - when someone wants to try something new, like a facilitation technique or a way of doing things, I try my best to take a leaf from performance improv and respond with “Yes, and …” (unless the idea would potentially have drastic negative consequences, of course).

The idea came from Stephen listening to the podcast Reply All which has a segment called “Yes/Yes/No”. The podcasters look at a particular tweet, and answer “Yes” or “No” to whether they understand what it means or not, before finally explaining it to each-other so that everyone can answer Yes.

We tried using this format to really dig into our network security model, the technologies we use, the way we provision it and team feedback was a unanimous “let’s do this again”.

My gut reaction to why this worked for us was that actively engaging with the content and explaining bits of it to each-other rather than someone doing a demo made our brains work differently and absorb the information better.


My team and I have 1-to-1s every other Tuesday after lunch. I genuinely love these because I’m able to talk with my team members on a different level to when we’re in a group, and I get a lot of pleasure from engaging ith their thoughts and ideas.

It’s a great time to give and receive feedback, outside of our fortnightly “feedback and cake” sessions which focuses more on group feedback, so 1-to-1 feedback tends to be a lot more personal.


Coffee with Steve

Sometimes I’m a bit rubbish with times, but now that I’m in the office early every morning I no longer have an excuse for being late for my wednesday chat with Steve!

This week we talked about the differences between the Unruly and the GDS models for infrastructure and SREs. Our SREs have a team of their own but also act as enablers and pseudo-coaches to raise ProDev’s skill level across the board.

The world of SRE-ness is one that Shift has been dipping its feet into a lot over the last couple of months, so it’s great to hear how places other than e.g. Google, Facebook do SRE stuff.

20% Time

Wednesday is traditionally (but not always) the day I take my 20% time. It’s normally the day with fewest meetings and it breaks the week up quite nicely. I spiked the terraform module for managing ProtonMail DNS records that I spoke about in my last weeknotes, which will be released on GitHub and the Terraform Registry very soon.

(Is it obvious yet that I really like Terraform? I want to build a custom provider next, probably for some obscure web service)


DIY Team Lunch

Sarah had the great idea to hold a spontaneous team lunch in our office - she wrote about it in her own weekly reflections. She brought a picnic blanket to one of our meeting rooms, we all brought along our own lunches, and we shared a bottle of Appletizer.

I hadn’t tasted Appletizer for years, and I felt undeniably classier drinking it out of champagne glasses.

There was plenty of non-shop talk but we had a brilliant idea over the course of the hour - we provide a bit of tooling to deploy and run our puppet code, but we still use a manual mutex (in the form of a slightly grubby android plush).

Could we steal a leaf from Hashicorp’s book and replace our manual lock with something like Terraform’s state locking?

This discussion escalated into how we could push events to DynamoDB and profile the workflow much like we would any other system, and show our slightly unkempt Python script a bit of love.

Tech talks

Ina and I presented the findings and progress that Shift had made towards implementing SLx and Error Budgets for the first of our mission critical systems, graphite.

We got some really great feedback on both the presentation and the content, and there were some great questions about how we might be using our Error Budget when we’ve finished making the calculations.

Farewell to Jahed

We said goodbye to Jahed this week who is, or … was :(, one of our developers - he’s been here long enough to feel part of the furniture and we’ll certainly miss what he brought to Unruly and ProDev as a whole.

He’s a big supporter of open source like myself so I personally will miss his voice in the blogging and open source group.


Team lunch followup

As the last Shift member in the office, I took 15 minutes and spiked a quick-and-dirty attempt at event pushing functionality in our Puppet workflow.

  • Terraform’d a DynamoDB instance on AWS.
  • Python’d pushing { session_uuid, date, commit_hash, event, hostname } at specific points in the workflow.

I’m excited for this to start accreting data over the next week - we’ll have better insight onto how many runs start but are abandoned, a better look at how long runs take, and many more.

This weekend I baked some easy Fork Biscuits as I slowly build up my baking rep, and I’ve not made biscuits for absolutely YONKS.

I’ve also been ill, but this felt more like a physical cold than the last one I had which seriously affected my usual levels of reasoning. Grotty, but mostly still able to operate at normal capacity.