For the past few years, we’ve been quietly building a holistic, malleable notebook we call PlayBook. The goal is to make something that feels every bit as good as paper & pencil for sketching and writing in your own hand. But unlike paper, PlayBook is imbued with dynamic behaviour, built out of composable pieces that let you reshape the notebook and add your own augmentations. Members of our team have been using PlayBook continuously for over two years, living inside the tool for their daily note taking, brainstorming, music composition, puzzle solving, collaborative whiteboarding, PDF marking, tutoring, and more. In this newsletter, and throughout the coming year, we’ll share more about how PlayBook was designed, how we built it, and what it now enables us to do.

Portemine

First up: Marcel Goethals is in the midst of an exploration. He suspects that propagator networks will be a useful computational substrate for PlayBook. You can use them to implement SAT and constraint solving, foundational pieces of our past Ink research. Propagators lend themselves quite naturally to visual/spatial representation, with execution visualized over time. They’d likely work well as a kind of “assembly language” that other visual programming systems could be built atop. As his experiments progress, Marcel has been publishing a series of small lab notes with motivations, findings, video clips, open questions, and other tidbits. Read more about Portemine, an exploration into propagator networks.

Gestures

Another area of ongoing work in PlayBook is our systems for handling user input. PlayBook is designed to feel like paper. It doesn’t have any on-screen GUI elements that you can accidentally tap with your finger, because paper doesn’t have that. You can comfortably rest your hands anywhere on the screen, or grip the screen with one hand while writing with the other like a clipboard. To switch between tools (such as drawing and selecting), we’ve created a special “firm press” gesture that you perform with the pen. Our focus on designing these gestures, and our desire to rapidly prototype new gestures, has led us to develop our own approach to dispatching and acting on user input events. In a recent lab note, Ivan Reese walked through the technical design of the PlayBook gesture system step by step, with comparisons to other popular approaches.

DrawDeck

Finally, we have DrawDeck. What is DrawDeck? Why is DrawDeck? Very mysterious. Perhaps there are “rune stones” you could pick up and set down, and scraps of paper you could set the stones beside. Perhaps a computational process would unfold in space and time. You could draw a cat. You could perceive the cat, or not. You might instead reach for other stones. Divination. Imagination. Perhaps the scraps of paper remember something. And from the mouth of Marcel Goethals, I directly quote, “Stones can communicate with each other through ‘dreams’.” For the sake of argument, let’s say you read more about DrawDeck, looked at some videos. Good, good. Now you never know.

What’s one more open tab?

After implementing image dithering for the new Automerge website, Ivan Reese created a little sandbox where you can play with error diffusion dither kernels in a way that feels like a board game.

Tickets are on sale now for Local-First Conf 2026. This year, in addition to two days of conference talks, there will be a special “Lab Day” hosted by Ink & Switch featuring live demos, creative experiments, and community projects — part unconference, part showcase, and shaped by the ideas that animate our community. We hope to see you there!

That brings us to the end of our special Inkstravaganza newsletter. Stay tuned for next time to hear about the flurry of activity happening in Patchwork, and reach out if you’d like to work more closely together.

GAIOS

In the previous dispatch we announced that Ink & Switch has joined the ARIA Safeguarded AI Programme (SGAI). Put plainly, the programme’s goal is to develop a suite of tools for modelling complex systems to determine how they might be augmented with AI. To that end we are building GAIOS, a malleable, collaborative software platform derived from our Patchwork project. GAIOS will allow other teams participating in the SGAI programme to unite their modelling and simulation tools, building atop a common foundation with interoperability and version control powered by Automerge. We’ll share more of GAIOS, including its source code, throughout the coming year.

The team working on GAIOS includes many longtime Ink & Switch researchers, and a few new faces:

• The mononymous grjte will lend her expertise in zero-knowledge proofs and programmable cryptography to the GAIOS team. Previously at Bain Capital Crypto, grjte recently shared Groundmist, a series of experiments building local-first tools that leverage the AT Protocol and Automerge.
• Maciek Sakrejda worked on Postgres tooling for most of his career but has now seen the local-first light. He’s already an active contributor to Automerge Repo, and since joining the lab has helped land presence indicators for our in-house instance of Patchwork. His full-stack skills will be instrumental as we prepare GAIOS for real-world use.
• Fresh from presenting at LIVE 2025, Orion Reed brings his knack for quick-and-compelling prototypes, programmable canvas tools, and malleable substrates to the lab. Orion’s work spans disciplines, leading him to collaborate with varied groups like FolkJS, the mysterious Tentpole Collective (with researcher-in-residence alumni Elliot Evans and Lu Wilson), and the Liberatory Computing collective.

Keyhive

We’ve accelerated our work on Keyhive, a capabilities-based auth system specially designed for local-first software, to prepare it for GAIOS. Adding access control is an important step for collaborative systems like GAIOS and Patchwork: it defines who can connect, who can request documents, and lays the groundwork for richer permissions in the future. What follows is a lightly-technical summary of the most recent work on Keyhive, written by Brooklyn Zelenka and John Mumm.

Integration into GAIOS

Automerge Repo started with no ability to do auth. We added integration points to Automerge Repo for authorization APIs, and then plugged in Keyhive as the auth provider. This gives us a natural point to enforce capabilities whenever a document is accessed while maintaining the existing Automerge Repo APIs. This integration provides the following:

• All messages sent or received are signed with the local non-extractable signing key
• Messages for a particular document are authorized by checking the access rights of the authenticated sender of the message against the local Keyhive instance
• Keyhive’s ops are synchronized over the same network stack as the rest of Automerge Repo, but currently with a naive diffing mechanism which we plan to replace with the new sync system (Subduction — more on that below)

The end result is that all documents in GAIOS are protected at the network boundary by the Keyhive access control CRDT and GAIOS guest applications are passed an interface to manage this access control state without needing to know about storage or synchronization. This is an initial integration, and ongoing work includes adding E2EE and enforcing revocations on backdated Automerge content.

“Share Menu” Graphical Interface

In parallel, we began prototyping a “share” (permissions) dialog for users. This menu provides a clear UI to invite collaborators, adjust permissions, and revoke access. The goal is to make capability management feel familiar — more like adding a collaborator in Google Docs, less like managing cryptographic keys. Behind the scenes, these actions map to issuing or revoking Keyhive capabilities. This design aims to connect the user mental model (“sharing a document”) with the technical enforcement layer (capabilities).

We’ve recorded a short video demonstrating this early UI prototype if you’d like to see it in action.

Subduction

One recurring challenge in CRDT-based systems is synchronization at scale. As more users join, and as documents grow larger, our reference sync server implementation has shown stress points:

• It maintains a fully materialized version of every document in memory.
• It computes hashes and related states eagerly for each request.
• Being written in Node.js — which being 32-bit has a 4GB memory limit — it can run into memory pressure.

The result: servers sometimes time out under heavy load, particularly when handling large diffs or very active documents.

To address this, we’ve been developing an alternative sync strategy we call Subduction. Instead of materializing entire documents, Subduction deterministically chunks documents and streams only the minimal state needed to reconcile peers.

We believe that this should result in:

• Improved scalability: less memory overhead, much less CPU use per request.
• Better performance: faster diffs, quicker round-trips.
• Set us up for E2EE documents since it doesn’t need to know anything about the underlying document other than some hash identifiers.

We are finalizing the integration of Subduction with Automerge Repo, and hardening the integration of Keyhive in GAIOS. Subduction and Keyhive are nearly ready for their public debut — after just a little bit more internal testing and dogfooding.

LIVE 2025

In September we participated in LIVE 2025, the 11th Workshop on Live Programming. Here are three of the presentations from folks affiliated with the lab.

A spreadsheet for exploring scenarios

In Ambsheets: A spreadsheet for exploring scenarios, Alex Warth, Geoffrey Litt, and Avi Bryant introduce Ambsheets, a spreadsheet that lets you model uncertainty and explore multiple scenarios simultaneously. The paper builds on their earlier explorations, discusses limitations they discovered, and presents a new version of the language that addresses those limitations. Also make sure to check out the video.

Nova

June Gardner, a researcher-in-residence at the lab, presented her work on Nova, a lightweight model of computing that uses rule-based modelling and physical metaphor to bridge the gap between physical media (pen/pencil, notecards/paper) and the programming of digital, electronic computers. The talk covers the basics of Nova as well as its advantages and current development status, as well as some hints at what the future might hold for Nova as a programming language.

Live Programming in Hostile Territory

Orion Reed, who recently joined the lab to help build GAIOS, collaborated with Christopher Shank on Live Programming in Hostile Territory. The paper and talk argue that live programming research should broaden its purview from the creation of new environments to the augmenting of existing ones and, through a selection of prototypes, explore three adversarial strategies for introducing programmatic capabilities into existing environments which are unfriendly or antagonistic to modification.

Version Control for Space and Structure

Beckett — our exploration of version control inside the Godot game engine — continues apace. We’re pleased to share that Lilith Duncan has joined the team. Lilith is a gamedev and toolmaker, most recently at Unity focused on worldbuilding and environment tools. Lilith and the rest of the team were in-person at GodotFest Munich, and they received a glowing response to various demos of their work-in-progress.

Speaking of work-in-progress, we’ve just published the first Beckett lab note, Version Control for Space and Structure. The note describes our latest findings after working to integrate diff visualizations into the Godot editor viewport and handling various structural changes within Godot scene files. These scene files are stored as text, making it possible to track them with traditional version-control systems (like git), but these systems fail to uphold important structural relationships within the files, leading to broken, unopenable projects. Read the lab note to learn how Beckett overcomes these issues.

And Points Beyond

We’re also pleased to spotlight two more new people who have joined Ink & Switch as researchers-in-residence. June Gardner, as we mentioned above, is the creator of the Nova programming language. She plans to spend the residency enhancing Nova and improving the tooling and documentation around it. Previously, June created Modal and founded the Nouveau creative computing collective.

Mimi Reyburn is a design-focused engineer and AI researcher, currently working at the NHS to improve information for patients and visualizing their needs. A few of her playfully inspiring personal projects are isthetubef.uk and Inky Calendar (not to be confused with Sketchy Calendar).

Finally, after months of internal brainstorming and iteration, we unveiled a brand-new website for Automerge with a striking visual design created by Todd Matthews and a whizzy interactive demo by Ivan Reese. The new website uses punchy language and illustration to nail down a few key points: what Automerge is for, how you can adopt it, and why it’s worth using. Behind the scenes, the website uses a custom static site generator that Ivan previously created for the Ink & Switch website, designed for rapid iteration and ongoing tailoring to our exact needs. This will increase how quickly and readily we can update the Automerge website and unblock our effort to improve the docs, tutorials, and other resources.

That’s all for this mega-sized Ink & Switch Dispatch. If you’d like to collaborate with us or share your thoughts, we’re all ears. See you next time!

Ink & Switch supporting the ARIA Safeguarded AI Programme

I’m pleased to share that Ink & Switch has joined the ARIA Safeguarded AI Programme. It’s an ambitious research programme dedicated to making it possible to employ AI systems in large-scale safety critical applications, such as optimizing national power grid allocation or developing biomedical supply chains. The programme is funded by the UK Government’s Advanced Research + Innovation Agency.

This might seem far afield from our usual area of work, primarily focused on “tools for thought”, but in fact that’s precisely what we’re offering to the programme. We’ll be working with our fellow Creators in the programme to develop the interfaces that help people and AI agents collaborate on building accurate models of interesting real-world systems.

While it’s still early days for our collaboration, and we don’t want to presume what the best approach for this project will be, our work on Patchwork is what brought us into the programme, and is a starting point for further exploration.

The SGAI programme is significant, including too many teams to name across many disciplines, but we are joined in our little sub-group by our friends at HASH (who some of you might recognize from the Block Protocol), the Future of Programming lab building Hazel under Cyrus Omar’s direction at the University of Michigan, and the Topos Institute who are working on CatColab.

Throughout the next two-and-a-half years, we’ll be spending lots of time in the UK working closely with the research group and with the many people and institutions who might need these tools. We held our first UK-based meetup recently at the tldraw offices (thanks to Lu and Steve for hosting us), which featured demos of Patchwork, Hazel, Tonk, DXOS, and Cursor for Canvas.

I’m excited and honored to be a part of an ambitious project like this, and I’m grateful to David Dalrymple and the ARIA team for bringing us on board. We are just one small part of a huge team, but we are delighted at the chance to continue to develop tools for thought in a new context of serious use.

As always, if your organization could use our help, you can always reach out. We’re available both informally and in partnership.

Oh, one more thing: one of the requirements of the programme is the open-sourcing of all code participating in the project, so we’ll be releasing Patchwork in some form soon. It needs a little more time to bake in the oven before we turn you loose on it but… watch this space.

Two new faces joining the lab

We’re pleased to welcome a pair of new researchers to our team. They’re joining, in part, to lend a hand with our work on the ARIA Safeguarded AI Programme, each bringing a particular area of expertise.

You may remember John Mumm (he/him) from his work on Keyhive. After a brief stint away, he’s rejoined the lab to help implement a version of Keyhive suitable for Patchwork, and will also be contributing to Automerge. John also has a knack for cooking up surprising little local-first demos, so don’t be surprised if some of those make an appearance in future Dispatches.

We also have chee rabbits (they/them or just chee), who joined just weeks ago and has already rebuilt the core of Patchwork in a much more modular form. While Patchwork started as a research prototype, we’re now using it for much of our collaboration within the lab, and it’ll be up to chee to steadily increase its dependability and malleability.

Exploring ink deformation

Several of our projects — Keyhive and Patchwork, for instance — have “notebooks” chronicling their evolution. For Programmable Ink, however, we’re doing something different.

The ink work is fluid, and has been advancing in surges over the life of the lab. In that time we’ve amassed a sizable collection of lab notes internally, but they aren’t organized in any sort of coherent sequence or structure. Lately, we’ve been collecting some of these existing notes, and writing new ones, to grow a wiki (of a sort) tying these notes together.

It’s not meant to be the sort of thing you sit down and read front-to-back, or follow along with as each new entry is added (though if you’d like to, subscribe to our RSS feed). Rather, we’ll occasionally point you to something new—a good starting point—and let you explore from there. Follow your curiosity.

This month, we’re sharing an exposition of ink deformation. In short, it’s our goal to imbue the ink and paper of your digital notebook with a sense of material physics, behaviour informed by but diverging from their real world analogs. This includes systemic behaviours like undo/redo—naturally—but also more specialized behaviour. In the case of ink, we’re looking for ways to make it bend, stretch, and mold. For instance, if you draw an arrow, you should be able to grab the arrow and point it at something else — yet it should still look like it was drawn by your hand.

To that end, here is Ink Deformation — A Review, and if you’d like to explore deeper, see this growing collection of all our ink deformation notes.

What’s another open tab?

• Join us this Wednesday for the September Automerge Community Call, featuring updates about the core libraries, a couple demos, and time for Q&A. These community calls are a new monthly opportunity for the Automerge community to share their projects and hear the latest news from the core team. Sign up for the calendar, and simply RSVP if you’d like to attend.
• Fans of the Local-first Conf, take note of Sync Conf happening on November 12th in San Francisco, brought to you by the same organizing team (and with substantial interest overlap).

That’s all for this month. As ever, we’d love to hear your feedback and ideas. Simply reply to this email, or use one of the many other ways to connect with us. See you next time!

Local First: the Secret Master Plan

The recording is up from lab director Peter van Hardenberg’s talk Local-first: the secret master plan, which he gave at Local-First Conf in May.

In the talk, Peter shares our vision for building collaboration tools on top of local-first infrastructure. You’ll see prototypes of universal version control and malleable software, and some new demos of the Patchwork collaboration environment that we use to do our own work inside the lab.

Automerge 3 is out!

Automerge is a local-first data sync engine that makes it easy to build collaborative apps. Today we’re excited to announce version 3.0 of Automerge.

The main update is a dramatic reduction in memory usage. Automerge stores the full history of documents to facilitate offline collaboration and version control workflows; in previous versions of the library, this could lead to gigabytes of memory usage when working with documents that had long histories. Now, in Automerge 3.0, we’ve cut that down memory usage by over 10x, sometimes dramatically more, making Automerge feasible to use in a much wider range of scenarios.

In addition to the memory usage improvements, we’ve also cleaned up some redundant APIs, particularly around the dealing with text strings.

If you’re using Automerge already, you should upgrade today. Automerge 3.0 uses the same file format as Automerge 2 and the API is nearly fully backwards compatible. See the migration guide for more details. If you’re not using Automerge yet, now’s a great time to take another look, as things have come a long way in performance and reliability.

To learn more about how we achieved these improvements, read our launch blog post.

Safe in the Keyhive

For the past year, our Automerge team has also been working on access control and end-to-end encryption. Recently, at Local-First Conf, Brooklyn Zelenka and Alex Good gave live updates on our progress. Check them out:

• Safe in the Keyhive: Local-first access control with E2EE and capabilities: Brooklyn Zelenka presented on building end-to-end encrypted access control for local-first applications.
• Beelay, a (reasonably) generic encrypted sync protocol for CRDTs: Alex Good discussed approaches for synchronizing end-to-end encrypted CRDTs, explaining the Beelay protocol developed as part of the Keyhive project.

Scribble on your Google Calendar

Here’s an update from our Sketchy Calendar project by Marcel Goethals and Paul Sonnentag. (You can also read this update on our website.)

Calendar apps and physical notebooks have very different vibes. Calendar apps are formal and visually uniform — just colored rectangles with titles that snap to a fixed grid. Notebooks have texture — you can use different pens and highlighters to mark and annotate however you want.

In our first prototype, we tried bringing both worlds together by starting with a digital notebook. We wanted to test whether the expressive freedom of pen and paper could coexist with synced digital events. The approach was to pull in your Google Calendar events as a static background layer, so you could sketch directly on top of them—circling important meetings, drawing arrows between tasks, or blocking out rough time estimates. Our goal was to preserve the iterative, flexible nature of notebook planning while still being able to see your scheduled events.

Here’s what we learned:

Sketching on your calendar feels liberating. Calendar apps force everything into uniform colored blocks, making it hard to distinguish critical meetings from your partner’s events that you just need to avoid scheduling over. Pen and highlighter are far more expressive - sketch tentative plans in gray, highlight time blocks, or jot quick notes without popups. When using sketchy calendar we found ourselves adding much more personal detail than in Google Calendar, capturing not just meetings but what we actually spent our time on.

It’s difficult to translate between the two worlds. When building this prototype we struggled with how formal events and scribbles should interact. Should sketched events sync back to Google Calendar? Sure, we can use text recognition on handwriting, but you’d lose the implicit meaning of color and style. And how do you translate a roughly sketched time range into a fixed start and end?

The reverse problem is equally tricky — if you draw an arrow pointing to an imported event that gets rescheduled, should the arrow automatically move? A simpler solution might be stopping auto-updates once you’ve drawn on the paper, and just indicating when underlying events have changed so users can update manually.

It’s useful to have a distinction between formal events and rough scribbles. In our prototype, Google Calendar events and hand-drawn scribbles remain quite separate. Initially we thought this was a problem, but there’s actually value in distinguishing between formal events with fixed times and loose sketchy plans. When meeting others, everyone needs to show up on time. But when planning for yourself, it doesn’t matter if you use personal shorthand that only makes sense to you in the moment. Our current prototype solves this by keeping scheduled events in Google Calendar and personal plans as scribbles. Even in a world everyone had automatically synchronized sketchy calendars, it’d still be useful to have distinction between formal and informal events.

It’s nice to have a dedicated device for daily planning. Sketchy calendar feels like a notebook open next to your computer — you can see upcoming events alongside daily todos at a single glance. Having that context in your physical space keeps you aware of your priorities without switching away from your work. Unlike a physical notebook, the events stay synced, so when someone reschedules a meeting in Google Calendar, it automatically appears in your sketchy calendar.

Even without solving all the complex interactions between sketches and formal events, keeping them clearly separate proved surprisingly effective. The simple model of importing calendar events as a background layer while keeping all annotations as personal scribbles felt natural and useful. Next, we want to explore how this approach scales—what happens when you need to flip between days, weeks, or plan across an entire year?

What’s another open tab?

• Designing Calm Technology: a short, classic read on ubiquitous computing, by Mark Weiser and John Seely Brown
• Enough AI copilots! We need AI HUDs: lab researcher Geoffrey Litt draws our attention to ambient user interfaces — think spellcheck — and implores us to explore beyond chat-based AI.
• Practical Local-First Software with Automerge: Lab director Peter van Hardenberg gave some demos of how to use the latest Automerge tools to build local-first software.

That’s all for now—until next time.

New Automerge releases

We’ve recently released some major updates for Automerge, the open source CRDT-based data synchronization library that we develop at Ink & Switch.

Automerge 3.0 Beta: Automerge 3.0 is an architectural overhaul that radically improves performance. Historically, Automerge compressed large histories on disk, but didn’t use the compressed format in memory. For this release, Orion Henry re-implemented Automerge to use the on-disk columnar compression format in memory; the result is memory savings of 100x in some cases, and much faster document load times. We’re seeing huge benefits in our internal deployments!

Alongside the performance improvement, we’ve also taken the opportunity to do a major version bump and fix some awkward parts of the API which were a result of maintaining backwards compatibility.

This is a big change, so we’re rolling it out slowly. We’ve run fuzz tests and have been successfully using it internally; we’ve also been working with the Automerge community to make sure it’s stable and fast in their usage. We’re confident enough now to release a beta. Please kick the tires and report back to us!

Automerge Repo 2.0: We have also released a new major version of Automerge Repo, the networking and storage adapter library for Automerge. We’ve cleaned up APIs to require less boilerplate, improved handling of asynchronous loading states, made progress reporting better, added more convenient helpers for history and diffing, and more. See the announcement blog post for more details.

Sketchy Calendar

When it comes to calendars, you can choose between using a digital calendar app or getting a paper calendar. They both allow you to keep track of things like doctor appointments, work meetings or birthdays, so you can keep a clear head and be sure that you won’t forget anything. But while the two approaches may seem similar on the surface, they’re radically different in the kinds of trade-offs they make.

Lab researchers Marcel Goethals and Paul Sonnentag have begun exploring what it would mean to have a calendar that combines the convenience of a digital calendar with the simplicity and expressivity you get from pen & paper. Find out more in their introductory lab note.

Seeking Godot game devs

Over the past few months, we’ve been working with the Endless Foundation to prototype a new approach to collaboration in game development: live and asynchronous collaboration, with branches and diffs, all built right into the open-source Godot game engine.

Here’s a sneak preview:

We’re now seeking people with Godot game development experience to test out an early prototype. Sign up here if you’re interested. Or, if you know anyone who does Godot development, we’d appreciate you sharing this link with them!

Website updates

We recently shipped a major expansion to the Ink & Switch website.

In addition to our essays and lab notebooks, you’ll now find landing pages for our four primary research areas—local-first software, programmable ink, malleable software, and universal version control—each sporting a newly designed logo. On these landing pages you can learn more about the ongoing themes present across our work, and then dig deeper into summary pages for each of our projects, some of which are being shared publicly for the first time.

On our home page, you’ll also find a sampling of upcoming events and appearances by our researchers, and — answering one of our most frequently-asked questions — information about how our research is funded and how you can get involved.

This update goes a bit deeper than just the launch of new pages. We’ve also replaced our old site engine, Hugo, with a new system developed in-house that makes it easier for us to experiment with new ideas for the form and content of the site. Our notebook for Programmable Ink, for instance, is continually evolving as we find new ways to share this work as it unfolds.

So set aside some time to browse around and see what’s new, and subscribe to our RSS feed to be notified whenever anything is published to the site. Of course, we’ll also keep sharing updates on our work right here in the Dispatch newsletter.

How to Do Ambitious Research in the Modern Era

We don’t often talk about how we work: we try to let our work speak for itself.

Still, if you’re the kind of person who finds the “behind-the-scenes” view of research interesting, you might enjoy this talk at the Nemertes [next] conference by our lab director, Peter van Hardenberg. He talks through the mechanics of operating an independent research lab, connecting academia and industry, recruiting, and thinking about research like a revolutionary.

If you’re interested less in the “how” and more in the “what” of cutting edge research, consider starting with Itai Cohen’s talk from the Nemertes conference about his group’s work on Electrically Integrated Autonomous Microscopic Robots.

That’s it for this month—until next time!

Open-sourcing our local-first access control work

We’ve previously introduced Keyhive: a project focused on local-first access control. Today we have two exciting updates on that work.

First, we have open sourced a pre-alpha version of Keyhive and Beelay for those curious to see the code. Keyhive is the core signing, encryption, and delegation system, and Beelay provides auth-enabled sync over end-to-end encrypted data. This is for interested readers and experimenters, but not for production yet.

We’ve also written a lab note about how two peers synchronise their collections of documents. The underlying problem here is one called “set reconciliation” and it is surprisingly common For example, fetching changes from git has to deal with this. We’ve adopted some exciting new set reconciliation algorithms that make it possible to synchronize very large collections with a small amount of communication, which has become important as we see Automerge-backed applications growing in scale.

Filtering scenarios in ambsheets

When you’re choosing from a long list with many options, it’s helpful to have filters to narrow down your search. This is a familiar concept on many websites—for instance, on the flight search website Kayak, you can start with a list of thousands of flights, and then filter down to the ones that meet your needs: “a nonstop flight, leaving before 11am, in economy class”.

What if you could apply this same concept of filtering to any problem in your life? Whether it’s deciding on a budget or planning a project, we all encounter situations where we’re considering a large number of options, and need to narrow down our search.

In our latest lab note , we show how to do just that, using filters in Ambsheet, our research prototype of a spreadsheet for modeling multiple scenarios. Filters in Ambsheets are powerful because you can filter on any cell, including the outputs of computations with formulas. Read the note for more demos and details.

Sneak peek: Reconstructing Sketchpad

Ivan Sutherland’s Sketchpad is a seminal work in computer science. Besides being one of the first computer graphics systems, it also pioneered the use of objects (via its “masters” and “instances”) and nonprocedural programming (via geometric constraints). It ran on the TX-2, an experimental computer the size of a football field that unfortunately no longer exists.

To give people a chance to experience Sketchpad firsthand, lab researcher Alex Warth has been working on a re-implementation that runs on the iPad. It’s not quite ready for prime time yet, but he has been documenting his progress on social media.

What’s another open tab?

• We’re admiring these modular MIDI controllers.
• We’re excited about the book WASM from the Ground Up, recently published by friends of the lab Mariano Guerra and Patrick Dubroy: “Learn Wasm by building a simple compiler in JavaScript.”
• Several lab members are speaking at Local First Conf 2025 in Berlin, May 26-28th. See you there?

First, a couple announcements:

The Automerge team will be hosting a community call on February 20. Come hear about progress on Automerge 3! A new multi-document sync system, end-to-end encryption, read-only document support, and massive improvements in memory usage and load/save performance.

Local-First Conf is coming back after a successful inaugural event last year. This year’s conference will be May 27-28 in Berlin; tickets are still available.

Pulling the thread on Inkling

There’s an early exploratory stage of every project where ideas are soft, where you’re guided by hunches and reckons. Maybe you have a concrete problem to solve and you’re not sure where to start. Or maybe you’re being pulled along by the desire for how things ought to be. This is a time of loose thinking, where the notes you write are scrawled and the drawings you make are sketchy.

On the ink track, we often find ourselves in this mode — we cherish it. After all, we’re trying to make tools that support this kind of thinking. In the fall of 2023, we spent a few months chasing down some rogue ideas about how this sort of tool ought to feel. We figured out how to build a constraint system powered by relaxation that performed well and wouldn’t blow up in our faces (like all the previous ones had). We figured out how to turn these constraints into tangible things you could touch and toggle and tug at with all your fingers. We revived some old ideas from Crosscut that felt amenable to relaxation-based computation. This all came together in a little prototype we called Inkling.

In the past few months, Inkling was demoed at the LIVE conference and then open-sourced. We’ve now put together a little page on our website — not a polished essay; something more exploratory — that pulls back the curtain on the Inkling project. Take a look.

Ambsheets: spreadsheets for exploring scenarios

From calculating monthly budgets to planning dream vacations, spreadsheets are our trusted companions as we navigate life’s complex decisions—a playground for thinking, where possibilities come to life through numbers.

Spreadsheets are particularly useful for thinking through financial models, budgets, or any situation that involves considering lots of possible scenarios. Instead of laboriously redoing a bunch of math, you can quickly ask “what if?” questions and see the effects.

But, as great as spreadsheets already are for this task, we think they could be even better. In our Ambsheets project, we are exploring a small extension to the spreadsheet: what if a single cell could hold multiple values at once?

Suppose you’re moving to a new city, and you want to figure out how much you can afford to spend on your car, apartment, and other essentials each month. You start out by making a spreadsheet summing up the various line items.

Now, you want to consider two scenarios for possible cars you might lease.How might you compare those scenarios? In traditional spreadsheets you might need to temporarily try entering different values, or restructure the sheet to accommodate the scenarios.

But in an ambsheet, you can simply enter two possible values for the price of the car:

The new value of B3, {500,1200}, means either 500 or 1200. We call this an amb value, and it represents one dimension in our possibility space. Amb values flow through the computation just like regular values and don’t require any special treatment in formulas. Cell B7, whose formula is SUM(B3:B5), now evaluates to either $3,318 or $4,018.

We think this simple idea could make spreadsheets more powerful for thinking through problems that involve multiple scenarios. To learn more about how amb values work and how they compare to existing spreadsheets, check out the lab note introducing the project: A spreadsheet for exploring scenarios

A new protocol for local-first key agreement

For many documents, you want to collaborate with a trusted group and no others. The default for Automerge today is if someone knows the URL, they can read and write to that document. We needed a way to encrypt documents for groups of collaborators that change over time, and importantly concurrently and without a single source of truth. How do we transparently keep a group’s read access up to date?

Of course, we only want folks with at least read access to be able to view our documents, but not sync servers and other infrastructure. This means we need a way to encrypt and decrypt our documents, including when they are shared via semi-trusted sync servers. This in turn requires a way for our dynamic group to agree on keys for encryption and decryption over time. If a key is compromised, we also want to limit the amount of past and future data an attacker can gain access to.

Messaging Layer Security (MLS) has an answer for this called TreeKEM. But it’s not a fit for our local-first use case since it requires a central server to order operations and choose winners. There are concurrent solutions that are closer to our requirements, but still not the fit we were looking for.

To adapt these techniques to our use case, we designed our own variant of TreeKEM called BeeKEM. It’s designed for local-first applications that require end-to-end encryption for groups on the order of thousands of members. Decryption in BeeKEM is very efficient in the case where one device updates at a time, and scales linearly with the number of concurrent updates at any given time. This means that the usual case scales logorithmically like TreeKEM, but gains the ability to merge concurrently updated trees unlike TreeKEM in exchange for some extra computation.

To read more details about the underlying protocol, read our lab note: Group Key Agreement with BeeKEM

What’s another open tab?

• Friend of the lab Alexander Obenauer is writing and publishing a book called Bootstrapping Computing: “a short, technical book for curious, non-technical people, exploring how we built modern computing out of simpler parts.” Preorders are available now.

• Frinklang: a programming language for physical calculations with units.

• A year-end note from our director
• A recap of a recent unconference in Los Angeles
• An unusual programming environment: Droste’s Lair
• Sneak preview: version control for game development

A letter from our director

Ink & Switch’s dream is of computers that help us think, that help us feel more human. We long for tools that support our creativity, instead of encouraging consumption. A decade into our research, we’ve made some meaningful progress. Local-first software has grown from an idea into a global movement. In service of this goal, we’ve run dozens of projects exploring everything from gestural interfaces to digital ink down to the finer points of merge algorithms for rich text.

Our work has resulted in many more questions than answers, and so much work remains to be done. Of course we cannot do it alone, and I want to thank everyone in this growing community for their kindness and generosity of spirit. Changing the world is a group project, and before we meet our goals we will welcome many more people into the fold: scientists and artists, entrepreneurs and sponsors.

I hope everyone reading this is able to take some time to relax and unwind at the end of a long year. Looking back, we’ve earned a rest: we’ve worked with astronomers on their scientific papers, turned up wrapped in flagger’s tape to talk about programmable ink, and traipsed around the world in search of old friends and new ideas. I won’t try to give a full accounting – looking back at past dispatches will probably give an impression – but I can say that we have a lot more work we’re looking forward to sharing.

And so: thank you. One and all. See you in the New Year.

Peter van Hardenberg

2024 Unconf Los Angeles

As part of a recent visit to Los Angeles for a team summit, we hosted an unconference for friends of the lab at the Preserve. It was a great day of meeting people, having inspiring conversations, and experiencing folks’ projects live and in person. We have assembled a memento that aims to share some of the energy of the experience:

Unconf 2024 Los Angeles

Thanks to everyone who has contributed notes, recollections, and photos. If you have something to add, please send us a note at hello@inkandswitch.com.

Programming in a dungeon

Lab researcher-in-residence Elliott Evans and lab collaborator Josh Horowitz recently embarked on a collaboration to explore their shared interests in visualizing mathematics. Here’s a summary, in their own words:

Droste’s Lair is an unusual programming environment for building and counting mathematical structures, built in a two-week sprint. In Droste’s Lair, the user manipulates mathematical structures through direct interactions: dropping dominoes on chessboards and dragging items between lists. The system’s power comes from two forms of abstraction on top of this foundation: an “amb” mechanism that allows strands of execution to branch from one another, and a procedure-calling mechanism that enables recursion.

Intrigued? Descend now into Droste’s Lair.

Sneak preview: version control for game developers

We’re starting a new project next year at the lab about version control for game development. Here’s a sneak preview…

Imagine you’re a student building a video game, and you want to collaborate on it with some friends. What’s the first step: diving into designing a level? Coding a script for how a player moves?

Often, the answer is none of the above. Instead, it’s: “learn git”. Before you can even share a single scene or line of code, you need to become fluent with files, CLI commands, pushing and pulling, and managing merge conflicts.

Version control systems like git and Perforce are powerful tools for large teams of professionals. But for hobbyists and kids just getting started, the learning curve can stand in the way of even basic collaboration.

Imagine a different world where none of these barriers get in your way. You start playing your game, and invite your friend to start editing the level. All their changes show up live. No files, no CLI commands—just shared editing:

Over time, you’ll want to add more sophisticated and formal patterns, like working on separate copies (“branches”) to avoid stepping on each others’ toes. But these patterns should only be there when you want them to be; they shouldn’t get in the way at first.

For the past year, we’ve been exploring tools for achieving exactly this kind of workflow: starting with lightweight synchronous collaboration (“Google Docs” style), and gradually adding more formal workflows: space for private asynchronous work, tools for reviewing suggested changes, and more. We’ve tried dozens of prototypes for this workflow across many use cases: writing Markdown blog posts, writing empirical science papers, and drawing diagrams. Along the way, we’ve been building up a collaboration platform called Patchwork which bakes in version control as a foundational primitive.

Next up, we’re excited to take this exploration to the domain of game development, which poses a bunch of unique challenges:

Multimedia support: How can a version control environment support a variety of media types, from art to music to code? What does it look like to review changesets across media types? How do specialists like artists and programmers collaborate in ergonomic ways?

Integrating collaboration: Today, version control is often a separate system from the main engine or IDE, forcing users to switch tools and also adding more things to learn. What would it look like to make collaboration a more integrated part of the game dev experience?

Working in parallel: How can different users work together on a game without getting in each others’ way? How might we make it easier to explore different variants of game mechanics in parallel?

History: How can you visualize the evolution of a game over time?

One context we’re particularly interested in is education. In conversations with educators at the Endless Foundation (who are sponsoring this work), we’ve heard that introducing kids to files and Git CLI commands before they can do any collaboration is a serious barrier. We wonder if it’s possible to create a smoother ramp that starts simple for students but also scales up to the needs of larger collaborations and even professional game developers.

Concretely, our plan is to extend Godot—an open source game engine used by the Endless Foundation for their teaching activities—with some of the ideas from Patchwork. The prototype of live collaboration shown above is just one example of the kinds of experiments we’re trying. But we hope that this process will yield general insights into version control that apply beyond just the Godot environment.

If you have experience with the problems of collaboration in game development, we’d love to hear from you! We’re especially interested in hearing from hobbyists or indie game devs using Godot. Please send us a note at hello@inkandswitch.com.

Fine-grained provenance

In a prior Dispatch we introduced Jacquard, a project exploring collaborative editing interfaces for authoring science papers.

Lately, we’ve been exploring the theme of fine-grained provenance: connecting specific parts of one file with corresponding parts of another file. We’ve been trying out some new UI interactions that take advantage of this provenance information, such as:

• mirroring comments between a PDF and source text file
• allowing users to edit underlying source text directly from a rendered PDF
• using provenance information as a “map” of a project

Based on our own tests and some conversations with scientists, we’re feeling that there’s a promising path here towards more fluid editor interactions. We’re also thinking about how it might work to have fine-grained provenance information available as a core primitive in an operating system.

To see some video demos, take a look at our lab note on fine-grained provenance.

Automerge updates

It’s been a summer of change for Automerge, the local-first data library developed at Ink & Switch. Recently the team shared updates on three ongoing projects at a community call:

• Memory usage: Orion Henry is working on drastically reducing memory usage. He’s taking the columnar encoding already used to store Automerge documents on disk, and using it at runtime. This project is at the correct-but-slow stage right now.
• Multi-document sync: Alex Good is working to build a new sync protocol which doesn’t require the document to be in memory, works over encrypted documents, and enables syncing collections of documents more efficiently.
• Auth: Brooke Zelenka is leading the Beehive project, which implements end-to-end encryption and local first authentication + authorization for Automerge (and hopefully other systems as well).

For a deeper dive, you can watch the video recording of the call.

LIVE 2024 presentations

Last week, many of us attended LIVE 2024, one of the main academic workshops focused on live programming environments that make programming feel tangible and concrete. Several lab-affiliated folks presented work there, and you can watch video recordings of their talks:

• Ivan Reese and Alex Warth presented Inkling, Sketching Dynamic Systems, a system for sketching with constraints developed at Ink & Switch together with Marcel Goethals.
• Marcel Goethals presented Subsequently: Telling stories with pictures makes programs, a visual programming environment inspired by comics.
• Researcher-in-residence Elliott Evans presented Live Programming a Live Programming Environment: An Experience Report, work done together with Philippa Markovics, Martin Kavalar, Andrea Amantini, and Jack Rusher on developing a live programming environment using the Clerk live programming environment.
• Researcher-in-residence Lu Wilson presented Arroost: Unblocking creation with friends. They’ve also given several other talks in the past month; check out What it means to be open which has been provoking some good discussions about process within the lab.

We also recommend the keynote presentation by Jonathan Edwards: The Meaning of LIVE.

You can see all the talks from LIVE in the full video recording of the day. (You may find the talk schedule helpful for navigation.)

Malleable Software in the Age of AI

Lab researcher Geoffrey Litt recently presented Malleable Software in the Age of AI at the Nemertes Next conference, exploring what qualities make software malleable, and how we might thoughtfully incorporate AI.

In addition to demos of the dynamic document environments Potluck and Embark, the talk includes some more recent findings from Patchwork on using version control to support collaboration on documents—and some early glimpses of applying those same ideas to collaboratively editing the software itself.

What’s a few more open tabs?

• We’ve been enjoying digging through the Dynamicland archive from Bret Victor and team.
• Jamie Brandon’s HYTRADBOI conference is happening again on Feb 28, 2025. We enjoyed the last one!
• Gridfinity: for the organization nerd in your life.

That’s it for now — until next time.

Jacquard: Version control and provenance for empirical research

We believe one of the most important capabilities in creative tools is version control: helping people collaborate, review suggestions, and see what’s changed, in both synchronous and asynchronous editing situations. Last year we published Upwelling, a prototype of “draft layers” for asynchronous collaborative writing. Next, Patchwork built on that work to explore “universal version control”: powerful diffing and branching tools built not just for writing, but also drawings, spreadsheets, and other kinds of media.

This summer, Paul Sonnentag and Geoffrey Litt from the Patchwork team have teamed up with Josh Horowitz to explore universal version control in a new domain: scientific research papers.

We’ve heard from scientists in a variety of fields that their digital tools make it cumbersome to collaborate on data analyses and writing papers. One problem is that limited version control makes it difficult to review collaborators’ edits. Another issue is that writing and data analysis are managed in separate environments, which leads to tedious manual work stitching together data across tools.

On this project, we’re prototyping Jacquard: a collaborative environment for writing empirical research papers. The goal is to free up researchers to focus more on their core work of science and communication, and less on tedious bookkeeping. (The name “Jacquard” comes from the automated loom that was an important step in the history of computing.)

Our first demo of Jacquard shows a collaborative editor that supports editing LaTeX files and Python files. It tracks a full provenance chain in order to help build all the source files needed to build an astronomy paper.

For more details on the demo and our goals for the project, check out the Jacquard lab notebook, where we’ll post further updates. And if you’re a scientist who works with data and struggles with collaboration, we’d love to hear from you—email geoffrey@inkandswitch.com.

Beehive: Local-first access control

Cloud based services provide excellent access control features allowing users fine grained control over who has read and write access to a document, as well as features like user groups and folders which implicitly grant access to their members and contents respectively. For local-first software to be successful we’ll need to be able to provide similar features without relying on a central server to enforce access control at the network boundary. In fact, we want servers to become simple interchangeable relays which only operate over encrypted data. The goal of the Beehive project is to design and build a production ready instance of such a system which is general enough for most applications.

To date the the local-first ecosystem has mostly used a purely pull-based model, which is often sufficient for personal projects: each user can manually decide which peers to connect to and which changes should be applied. On the other hand, many production contexts (i.e. corporatations, journalists, or even planning a surprise party) are lower trust, require higher alignment, and are ideally low touch enough so that it’s not up to each person in a large organization to separately and manually infer who to trust.

This naturally leads to questions like:

• Who should the group members accept new edits from?
• What is a specific user able to do to this document?
• How to only share documents with some people but not others?
• What to do if a previously trusted peer starts behaving badly?
• What if an admin’s device is lost or stolen?

These are especially challenging in a local-first setting since there is no network boundary or central server to guard reads and writes. By its nature, local-first requires that any access control mechanism used must travel with the data itself and work without a central guard. There are also some tricky edge cases due to causal consistency. What should happen with content that’s later discovered to be malicious but honest ops depend on it causally? What is the best strategy to handle ops from an agent that was revoked concurrently (especially given that “backdating” operations is possible). If a document has exactly two admins (and many non-admin users), what should happen if the admins concurrently revoke each other (for instance, one is malicious)?

Recently, Brooklyn Zelenka and Alex Good (with significant input from Martin Kleppmann) have been hard at work building Beehive: a local-first access control system that seeks to address the above concerns. At a very high level, the current approach in Beehive is composed of three layers:

1. End-to-End Encryption (E2EE): With post-compromise security (PCS) and key management
2. A Group Management CRDT: Self-certifying, concurrent group management complete with coordination-free revocation
3. Convergent Capabilities: A new capability model appropriate for CRDTs, and sits between object- and certificate-capabilities

We’ve made substantial progress in designing the core data structures and algorithms, though a few open questions remain. We are currently refining our approach to address revocation edge cases, ensure causal stability under E2EE, balance forward security in operation-based CRDTs, and minimize trust in sync servers. As always, usability, space and performance are also top-of-mind.

It’s also worth mentioning another ongoing project at the lab focused on data synchronization for peer-to-peer and via sync servers that’s been headed up by Alex Good. We’ve realized that sync and secrecy strongly interact. Broadly speaking, sync protocols benefit from more metadata (to efficiently calculate deltas), whereas cryptographic protocols aim to minimize metadata exposure. This tension extends across related systems, including merging E2EE compressed chunks, and determining if a peer has already received specific operations when a sync server cannot access them in cleartext.

Fortunately, combining these systems can sometimes result in more than the sum of their parts. For instance, convergent capabilities help facilitate the calculation of which documents are of interest to particular agent, helping the sync system know which documents to send deltas of. For these reasons, we’re treating synchronization and authorization as part of a larger, unified project, even though each will yield distinct artifacts.

Automerge Anywhere

The Automerge team has made some big improvements to the WASM packaging setup for the library, which makes it usable in more contexts, including vanilla JS applications with no bundler, in React-Native applications on mobile devices, within cloud services like Cloudflare Workers or Val.town, and more.

For more details, see the full writeup on the Automerge blog.

Researchers-in-residence

Elliot joined as a researcher-in-residence this month. Elliot is working on researching tools that build bridges between diverse ways of specifying programs, from the esoteric to the mundane. While in residence, Elliot will be working on programming language prototypes to explore variations on concepts like destructuring and evaluation direction. Elliot is here to strengthen his research fundamentals, collaborate, and get feedback.

Lu Wilson published their essay about Arroost, a music-making tool. In the essay, they make the case for tackling emotional blockers when building creative tools.

What’s a few more open tabs?

• From lab researcher Ivan Reese: 2222, a mysterious programming game
• We wrote on the Patchwork notebook about Universal Comments: a general abstraction for working with annotations on any kind of document.
• You might enjoy this excellent study on file system design by Dominic Giampaolo, lead creator of Be File System who now works on file systems at Apple.

That’s all for now, until next time.

In this Dispatch: we have a report from the inaugural Local-First Conference in Berlin, a deep dive from our ink team on a new selection model, and some updates on our researcher-in-residence program.

Trip report: Local-First Conf

The first local-first conference was held in Berlin last month and several lab members were in attendance. For those who aren’t up-to-speed on what’s been happening with the local-first movement, Martin Kleppmann gave the opening keynote and described the Past, Present, and Future of Local-First, and Alex Good presented on his recent work shipping rich-text support in Automerge. There’s now a playlist of videos from the event so you can catch up on the talks you missed. We all particularly enjoyed Maggie Appleton’s closing talk introducing the concept of the Barefoot Developer.

After the main conference there was a second “expo” day of presentations by technology builders:

• Jazz.Tools
• PowerSync
• Fireproof
• Automerge
• DXOS
• ElectricSQL
• and of course, Berlin’s own: Yjs.

It’s incredible to see a thriving community growing around these ideas. Huge thanks to Johannes Schickling, Adam Wiggins, James Arthur, and the Scéal team for putting on a great show.

One last thing: you don’t need to wait a whole year to jump in with the local-first community. Why not check out Johannes’ local-first podcast or join the Local-First Discord?

Deep dive: dynamic selection with flux

Over on the Ink team, we just wrapped up a project designing our next programmable ink system. One of the interesting developments in this project was a new approach to selecting objects on the canvas.

Imagine a typical drawing canvas, like tldraw, Cuttle, or Figma. When you make a selection, you’re always selecting something specific. You might directly click to select a single object. Or you may drag a selection marquee to cover a region of space, selecting the objects that occupy it. The former is incredibly precise. You’re saying “I want this one object”. The latter is ever so slightly more approximate. You’re saying “I want any objects in here.”

A programmable system allows a wider range of specificity. You might have some deeply nested data, and make a selection by reaching for a known place within the structure: myObject.nested.element.to.select. But you can also create a selection dynamically by matching a pattern, evaluating some criteria, or issuing a query:

dotfile = /(^|[\/\\])\../
if filename.match(dotfile) then log("unix strikes again")

These later approaches make selection very flexible. You empower a program to do the selection for you at some point in the future, rather than selecting something yourself right this instant. You don’t know what exactly will be selected, but you do know some qualities it should possess.

When writing the code to perform a dynamic selection, you might have a few concrete examples to generalize from. But it’s up to you to identify the pattern and express that in programming terms. You have to take a step away from concrete, into the abstract. This is the cost of code — you gain flexibility by sacrificing tangibility. Each of the myriad approaches to dynamism feel different, and offer different tradeoffs at this step.

For our programmable ink system, we want an approach to dynamic selection that feels like a natural progression from the concrete selection of objects on a canvas. Not something like regex or a query DSL, where the way you describe a selection doesn’t much resemble the stuff being selected, especially if that stuff is graphical in nature. If our programs are going to be querying and effecting stuff on a canvas, they should be programmed right on that same canvas using that same stuff.

For a few years now, we’ve been exploring an approach to dynamic selection called spatial queries. In Inkbase, every object would keep track of the objects around it, so you could for instance ask any object for the things to its right. But “to the right” is a bit vague, and clarifying “50 pixels to the right” feels like coding, not like drawing. It’s also a bit too rigid — “50 pixels to the right” is less sketchy than “the rest of the sentence” or “this entire doodle”, and in our programmable ink system we prize sketchiness.

Over the past few months, while designing our new programmable ink system, we’ve been playing with a new idea for dynamic selection. We call it flux. It’s spatial queries as an object you can see and touch.

You draw flux on the canvas just like ink, and then it spreads out from where it was drawn, flowing and pooling like a liquid. It can spread through space, or through ink — your choice. When flux floods the inside of a region encircled by ink, it resists flowing through any small gaps. When it spreads along ink strokes, it can jump across small gaps.

Ink that is engulfed by flux is treated as a selection, just as though you’d selected the ink by hand. And you can use any of these selections (from flux or by hand) as the input to a programmed behaviour.

Flux will continue to flow and retract in reaction to changes in the ink around it. If you pull some flux-covered ink away, the flux withdraws. If you write or draw close to some flux, the flux will spread to encompass your newly made strokes. It’s a dynamic query that you can draw, see, and manipulate right on the canvas. It exists somewhere in between a marquee or lasso, which encompasses a region of space, and a reactive query, which continually feeds values into a computation.

We tried to make the experience of working with flux feel very much like other types of drawing and selection on the canvas. You can even tug some flux to guide how it will spread, which is helpful for selecting (say) all the text to the right of a checkbox.

Behind the scenes, flux works a bit like a hybrid of a particle system and a cellular automata. There are rules governing how the bits of flux relate to their neighbouring bits of flux, and how they react to other elements on the canvas.

We’re still exploring how flux interacts with other parts of our programmable ink system. How exactly should flux confer effects upon the things it selects? Do we want flux to eventually cool into a more static state? Or do we want flux to be even more dynamic, to spread and retract on command by a script? We don’t yet know what exactly flux is, how it works, or what it means, so you can look forward to hearing more about it in the coming months.

Researchers-in-residence

Alexander Obenauer has wrapped up his term as a researcher-in-residence with the lab. We’re excited to keep following his work on personal health informatics and more.

We’re thrilled to welcome a new researcher-in-residence: Maggie Appleton! She’ll be exploring ideas at the intersection of tools for thought and language models—more details to come in future dispatches.

What’s a few more open tabs?

• a video by Freya Holmér on the Continuity of Splines
• a 2010 paper on the design of Scratch

That’s all for now, until next time.

We’ve had a busy spring at Ink & Switch, including a lab summit in Denver and a hack week where some new projects sprung up. In this dispatch we have some updates on our ongoing work on version control, notes on an exciting new release of Automerge, and news from our researchers-in-residence.

If you’re in the neighborhood of Berlin at the end of May, several lab members will be participating in the Local-First Conference. Tickets are sadly sold out, but if you’d like to connect when we’re in town, drop us a line!

Patchwork: universal version control

In our last Dispatch in Febuary, we introduced Patchwork: a research project by Geoffrey Litt, Paul Sonnentag, Max Schoening, Adam Wiggins, Peter van Hardenberg, and Orion Henry exploring universal version control. Our goal was to design conceptual primitives for version control that were simple enough for all computer users; powerful enough to handle a variety of sophisticated workflows like branching changes and editing history; and could span a wide variety of types of media and collaboration workflows.

We started by exploring a variety of versioning concepts for helping small teams of writers collaborate better. The lab notebook for the project logs some of those experiments:

• Dynamic history: flexibly querying document history at different zoom levels and with different groupings
• Diff visualizations: experiments with many ways of visualizing diffs in writing at small and large scale
• Edit groups: grouping sets of edits into reviewable units
• Lightweight branching: making alternate versions of a document on separate branches, using simple light interactions
• Version history as chat: making history legible and discussable

All of these ideas were prototyped in Tiny Essay Editor—the Automerge-based Markdown editor we use at the lab—and used to write our lab notebook posts. We’ve found that even simple versions of branching, edit groups, and history timelines were quite powerful in that context. But we also discovered tricky challenges we’re still working on, including some new ways of representing changes in Automerge to make it easier to cherry-pick edits across branches.

We’ve also lightly explored two more areas beyond: AI bots that make suggestions on branches, and first steps towards version control on other media types like drawings and spreadsheets. More work is needed to flesh these out, but they feel like promising directions.

We’d love to hear feedback via email on this work! We’re especially interested in hearing from you if you need a better collaborative versioned text editor for writing blog posts or papers, or you might want to add better versioning to an existing application you’ve already built.

Rich text in Automerge

Automerge, the CRDT-based collaboration library developed by Ink & Switch, now has official support for rich text and a fully supported Prosemirror binding!

For those who’ve been following along: our Peritext essay from 2021 described a model for representing text with inline annotations like bold, italic, or underline. But that work didn’t address a critical part of rich text: block elements like paragraphs and bulleted lists, where any span of text can only belong to one block at a time. Martin Kleppmann then proposed some extensions to Peritext to handle these cases.

Since then, the Automerge team, particularly maintainer Alex Good, have been hard at work implementing all of these ideas in Automerge. We’re thrilled to announce that the results are now available: the new version of Automerge includes a native and portable representation for rich text with inline marks and blocks, and a reference binding for the ProseMirror rich text editor library for the web. There is also work underway on iOS bindings.

This work has been supported by our corporate sponsors, particularly including our partners at GoodNotes. Thanks also to Marijn Haverbeke for his input on the project.

If you’d like to learn more about the theory behind this, Alex will be speaking at Local-First Conf and we’ll plan to write up a more detailed account in the future.

For more details on this work, check out the post on the Automerge blog!

Researching-in-Residence

Researcher-in-residence Alexander Obenauer is working on Heron: an applied research project investigating concepts in personal health informatics for individuals with chronic conditions.

Researcher-in-residence Lu Wilson has concluded a collaboration with Dave Ackley, introducing and exploring the concept of natural code, and has also been researching autocomplete for canvas, building on some work from Patchwork.

Mary Rose Cook has finished her residency and taken a job with Void, a slightly mysterious new game engine company started by GitHub founder Chris Wanstrath. We’re all looking forward to seeing what they come out with.

Till next time!

New projects are in full swing and we’re getting ready for our first lab summit of this year at the end of next month! In this dispatch, we wanted to spotlight the work of Alexander Obenauer who’s been a Researcher-in-Residence with us since last year and introduce our new research project, in case you’ve missed Adam and Geoffrey’s posts on X.

OLLOS and Workbench by Alexander Obenauer

Alexander Obenauer is exploring how the future of personal computing could better serve people’s lives. You might remember Alexander from the Embark project, where he joined the malleable software team to address the limitations of modern siloed apps. In his independent research, Alexander is questioning today’s operating systems and exploring the OS of the future.

Adopting a new app is the only way for most people to get new interfaces and features into their personal computing environments. This presents users with a funny challenge: surveying pre-made apps for the ones that, hopefully, match the way they think and work. But this is a surprisingly arduous process because apps often operate on their own silo of data, and they present a new, separate environment which must be learned and adopted. When that doesn’t work, people are stuck trying to use software that doesn’t think or work the way they do.

Alexander has been exploring the concept of an “itemized” OS, one that allows users to construct their own environment. In such an OS all of your digital things — emails, todos, notes, reminders, webpages, podcast episodes, and so forth — are “items” that sit in one, local graph. Users are then able to create new interfaces with the items that matter in their lives and work, or introduce items that bring new functionality to their overall environment (such as having a reminder item which can be added to any other item).

OLLOS, one of Alexander’s recent experiments, is an interface that uses time as an organizing principle. Having all of your items organized on one timeline gives you a new dimension of context, as much of our digital things are related to each other by time. A meeting with a colleague involves a calendar event, a related note and maybe a follow-up email. In OLLOS, the temporal association is a benefit of the system. He recently published an essay in which he discusses designing, building, and living in OLLOS for several years, with details on how the design and concept evolved, what future work it points to for timeline interfaces and the OS of the future, and some reflection on personal interfaces like it. Read the essay here.

Workbench, his next project, allows anyone to build personal interfaces on top of the item store that powered OLLOS. This project aims to improve our understanding of the kinds of personal interfaces people want to build and the impact that living in personal interfaces has on people’s lives and work.

Universal version control

The creative process is all about iteration: trying out alternatives, seeing a progression over time, or discussing changes with our collaborators.

Different disciplines use different tools to explore variations, for example:

• Coders use Git to work in branches and see diffs
• Writers use Track Changes in Microsoft Word to suggest and review edits
• Designers duplicate artboards in Figma to see variations side-by-side

Last year, we took inspiration from these versioning systems and built an experimental text editor for writers. We think the core concepts in Upwelling are promising and transferable to other creative domains.

So now, we’re continuing this line of work with version control for writing and beyond. We believe that simple, powerful, universal version control tools could help creative people in many fields work and collaborate more effectively. Perhaps this version control could one day be built into the storage layer of your OS!

Geoffrey Litt, Paul Sonnetag, Max Schoening, Peter van Hardenberg, and Adam Wiggins have started building prototypes to test this idea in our new research project, codenamed Patchwork. Follow the work-in-progress posts in the Patchwork lab notebook.

What’s a few more open tabs?

• Local-First Conf 2024 is the world’s first local-first conference, happening on May 30 in Berlin, Germany. Early bird tickets are going on sale today. Martin Kleppmann, Alex Good, Adam Wiggins and others from the lab will be there, come say hi if you are able to attend.
• For a reminder or quick primer on what local-first is and why it’s important, listen to Peter talk about it in this localfirst.fm episode.
• Geoffrey gave a talk on Tiny Essay Editor, a collaborative markdown editor built on Automerge. We’ve been using it around the lab for all sorts of writing including this dispatch. Here’s a quick summary in slides.
• Lu has recently joined us as a Researcher-in-Residence. They have a channel of slightly-surreal videos on creative coding. The latest one is on Arroost — a playful, mind-boggling music-making tool.

Till next time!

We’re finding ourselves in a reflective mood as the end of 2023 approaches. This year, we published 3 essays, communed at a couple events, and organized several workshops. It’s always nice to celebrate publications and presenting our research in public, but much of our work are ongoing journeys of continuous exploration, prototyping, and findings. So, we wanted to start this end of the year dispatch by talking a bit about one of our longest standing research tracks: programmable ink.

Making a conversational medium

The ink team at the lab is dedicated to making something that combines what we love about sketching on paper with the power of computers. This new tool we envision will allow you to add dynamic behaviour to imperfect and often fuzzy sketches. More importantly, this tool should be intuitive to use; working with it should feel conversational like working with pen and paper or a trusted collaborator.

The lab has been working on this idea through different projects with various team members (see Untangle, Inkbase, or Crosscut). Making the experience of the prototype feel conversational and intuitive has been an ongoing research problem and one of our (many) obsessions. This year, Marcel Goethals, Alex Warth and Ivan Reese have made a breakthrough with their current prototype. They’ve built a system that’s bringing us closer to the level of conversational and intuitive feel we want.

Part of what makes pen and paper a versatile medium for thinking is its flexibility and informality. You can start scribbling anywhere and use any set of symbols or words to represent your thinking. You can decide on the relationship between things you scribble as you go, and you can go back and change parts of what you’re scribbling. The meaning of anything you put on the paper can easily transform and evolve as you think, and there’s no friction between your thought process and the medium. The ongoing conversation between what’s on the paper and what’s in your head is what helps you make sense of your ideas and push them further.

Computers are a dynamic medium that allow us to perform calculations and actions like copying, reusing things, and simulating ideas. But computational tools lack the kind of flexibility and informality inherent to pen and paper. They add too much structure and formality (think structured tables or vector shapes) too soon for informal, often hazy thinking. They also typically rely on you knowing the input(s) so that you can arrive at a desired output. A spreadsheet can compute your net profit if you add or make changes to your losses and expenses. We can use CAD to create an accurate floor plan if we know the measurements and positions of items.

Tools like spreadsheets use unidirectional computation where computation can only flow forward in one direction. A spreadsheet formula can compute a result using numbers from the cells but it can’t give you changes to the numbers if you edit the result. This is good for answering questions like: what will my monthly payment be if the interest rates go up? But sometimes, we want to work backwards and ask: what interest rates do I need if I want to keep my monthly payment at a certain amount? For these kinds of questions we’d have to do extra work, and this distracts us from the thinking work that truly matters. We can’t have a conversational experience with a tool that adds too much structure too soon and requires us to think for it. So how do we create a new computational medium that can give us the kind of conversational experience pen and paper does?

One of the key starting points for solving this problem was making bidirectional computation work in our prototype. Any point, symbol, or line you draw on paper are like outputs that are simultaneously inputs that you can do more with. We want the tool we’re making to give you this kind of freedom—the freedom to manipulate any object and add dynamic behavior to it. We implemented a version of bidirectional computation in Crosscut, a tool for drawing vector-based dynamic models. It’s what allowed users to wire a relation between two points and experience the relationship immediately by wiggling any of the points, like in the example below.

When designing, we’re constantly thinking about materiality — what is the material that things are made out of? If you’re a designer you may have thought about this when it comes to UI elements (remember brushed metal?), but we also think about the materiality of our programming constructs. In Crosscut, the materiality of the programming model had a feeling of grain, like wood, where the model behaves well when data flowed back-and-forth in certain directions, but poorly in others. As a user you would encounter this grain when using the tool and so you couldn’t manipulate objects as freely as we’d like.

The challenge of bidirectional computation is that sometimes there’s more than one way to get to a result. If you change the result of an addition, which input should you update? In our latest prototypes, we’ve developed a computational model that’s powered by a new constraint system. This model can find solutions that satisfy your constraints but it feels grainless to interact with. We’ve started calling this approach omnidirectional computation. We call it omnidirectional because there aren’t really inputs or outputs to it. There are only relationships between objects. You can edit any – or all – of the pieces of the system simultaneously and still get a result even for complex problems. By default, our system will change every object a little bit instead of choosing a single input to change a whole lot. We like to think of this as “spreading the change”.

This model of computation opens up new opportunities for how our system can evolve both computationally and interactionally—it’s bringing us closer to the conversational medium we want to make. For example, it provides robust support for multitouch interactions and problems like this pulley system:

We’ve made several attempts at implementing this constraints-based computation but weren’t successful at making the constraint system stable enough until recently. Some of our early prototypes felt like you were working with jelly. Our new constraint system feels fast and solid, and feels much more intuitive and conversational. It’s been a journey but we’re excited to continue improving it and look forward to sharing more of our findings in the near future.

Out and about

It’s the end of the year, and so I (Peter) wanted to share some recaps from a few events we attended this year. As a distributed research group, occasional in-person meetups help us keep in touch with each other as well as the rest of you. There’s simply no substitute for meeting people in person, sharing a meal, and getting into the weeds on our shared interests.

This September, many of us attended the last Strangeloop in St. Louis, MO where Martin talked about new algorithms for collaborative text editing. It was bittersweet: Strangeloop was my personal favorite conference and brought together a mix of industry and academic folks with an eclectic programme that combined useful information with feats of “stunt computing.” Any attempt to pick favorite talks would be impossible, but here are our own talks from the previous two years on Programmable Ink by Szymon Kaliski and Backchannel by Rae McKelvey.

After the main event, we co-hosted a one day Local-First Unconference with our friends at Fission and DXOS. It filled up quickly, so we expanded the capacity and then it filled up again. Local-first software is on the move! I’m still hoping to edit something together from the community notes one of these days but there’s plenty of raw material to peruse here.

More recently, we participated in the LIVE Programming and Programming Local-First academic workshops in Cascais, Portugal as part of SPLASH 2023 and then hosted another unconference at a beautiful resort. Together these three events capture the breadth of our interests: LIVE is full of wild open-ended programming language experiments, PLF is a mixture of field reports and crunchy algorithms talks, and the unconference attracted roughly fifty renegade developers from across Europe to discuss optimistic visions of a new kind of computing.

What’s a few more open tabs?

• A gallery of photos from our unconference in Portugal by Todd Matthews.
• Alex Good has written a round-up of Automerge updates. Automerge CRDT lives at Ink & Switch and has had a lot of performance work and new features in recent months.
• An interesting article on creating a lab notebook on top of Notion by biotech lab Eleven Therapeutics.

Till next time

We’re looking forward to lots of new projects next year. Thanks for following along with our work. Happy holidays!

Some of you have expressed an interest in knowing more about what’s going on at the lab—wish granted!

First, a quick re-introduction in case you’re not sure how you wound up here. Ink & Switch is an independent research lab exploring the future of tools for thought. Our goal is to invent a computer that helps its users think more clearly, collaborate more effectively, and never gets in your way.

As of today we have three active research tracks: local first software, programmable ink, and malleable software. We’re about a dozen researchers around the world, mostly spread across North America and Europe. We have a mixture of industry veterans and academic researchers and we view our output primarily as knowledge: a clearer understanding of the problems and opportunities in tools for thought.

In this first dispatch, we want to share some of our recent work on Malleable Software with Embark, talk a bit about our Researchers-in-Residence program, and introduce you to Mary Rose Cook.

Embark - Dynamic documents for making plans

On the Malleable Software track our goal is to design software environments that people can mold to their unique needs by pushing personal computing beyond the boundaries of siloed apps.

Recently, we’ve been exploring a potential solution to this problem: dynamic documents that let you gradually enrich text with interactive behavior. Last year we shared Potluck, a dynamic document environment for text notes and recipes. It felt like we had found a valuable thread worth pursuing for making software that’s more personal, but we also found challenges with some aspects of Potluck’s design, and we also wanted to find deeper, more authentic uses for this new kind of document.

So, this year, Paul Sonnentag and Geoffrey Litt from the Malleable Software track teamed up with independent researcher Alexander Obenauer to design another take on dynamic documents. This time around, they focused on travel planning as a use case. Planning a trip typically requires juggling many apps and manually coordinating between them. It can involve booking flights and hotels, researching restaurants, scheduling meetings, planning routes, etc. Different pieces of information are spread across multiple web applications, and it’s left to us to collect and keep track of them all in one place. How could we simplify and improve this process with a new kind of dynamic document?

The result of this exploration is a research prototype called Embark. It lets you write a text outline augmented with three features: mentions of structured data like places and dates, spreadsheet-like formulas to compute weather forecasts and routes, and rich views like maps and calendars. Together these primitives enrich a planning document into a live, interactive surface that goes beyond text content.

In our new essay, we share what we learned from using Embark to plan many of our own real trips. We also delve deeper into why travel planning with apps is difficult in the first place, and the design thinking behind Embark. We’re now more convinced than ever that dynamic documents are a powerful foundation for personal software. We hope you find the essay thought-provoking and look forward to hearing your thoughts and feedback.

• Read the essay!
• Watch Paul’s presentation at the LIVE workshop.

Researchers-in-Residence

Solo independent researchers we know often discuss the challenge of working alone. Being part of a research group can create community, social accountability, provide feedback, and just give you someone to talk to when you’re feeling stuck or bored. This is why we started our Researchers-in-Residence program.

We invited a few independent researchers to come and join our virtual lab and work on their own projects being a part of our little community. They come to our weekly meetings to talk about their plans, give or get feedback on their work, and get their own “office” (read: Discord channel) where they can post what they like.

In the current rotation, we have Mary Rose Cook who has been working on Lude and Alexander Obenauer who is exploring the future of computing with the Itemized OS.

As Mary just presented Lude at LIVE 2023, we wanted to introduce you to her and what she’s been working on.

Lude by Mary Rose Cook

Building software takes an awful lot of time; this is partially because we don’t have better structures for expressing software other than code. Code remains the best way to express software but writing code is incredibly laborious. As a software maker, testing a simple idea can take longer than it should. Mary’s research aims to address these limitations—she’s making tools for building software quickly.

Lude is a tool for building video games quickly which materialized during Mary’s residency at Ink & Switch. Video games are an interesting use case because “building games faster” is basically the hardest version of the problem Mary is tackling—games often require defining complex behavior. So, how does Lude make it quicker to build games?

1—Direct manipulation of live objects in the running game. Traditionally, level editors are not in sync with the running game. The game designer makes edits in one window, then runs the game in a separate window for testing. The magic in Lude comes from syncing the editor with the running game, so that all changes are reflected in both environments. In other words, when you’re making changes in the editor, you’re actually directly manipulating the live game objects.

2—Expressive, fast code generation. Generating code via prompts is often faster than coding by hand and straightforward, but how can this experience be improved for the game designer? The key is generating code that matches what the designer intended it to do. Lude is designed to capture the expression of the designer’s intent implied in the prompts. (If you’re curious of how exactly this is achieved, read more in Mary’s research blog.) And what makes the expressive code generation fast? Lude allows fragments of code to be generated and integrated into the codebase, as opposed to regenerating an entire code base following a change.

Where next?

The goal of Lude wasn’t necessarily to invent something new, it was to discover and test useful concepts for faster software development. Following Lude, Mary is looking to take these learnings and direct her research focus onto better prototyping tools.

What’s a few more open tabs?

Sorry, we couldn’t let you go without a few more links to things.

• The Workshop on LIVE Programming 2023 was a blast, we especially recommend Lu Wilson’s talk on CellPond: Spatial Programming Without Escape.
• Alex Obenauer is thinking about Gestural View Construction.
• Engraft by erstwhile collaborator Joshua Horowitz appeared at UIST.
• The OpenDoc Programmer’s Guide weighs in at roughly 600 pages, but if you’re interested in Malleable Software, there’s a ton of great information in there.

That’s it for now

We have so much more to talk about: conference recaps, ongoing ink work, new experiments with local-first… but it will all have to wait for another day.

Is there anything in particular you’d like to know more about via our dispatches? Let us know.