Stack literals are declared by wrapping a comma separated list of values in square brackets ([ / ]).

VAR my_stack = ["hello", "stacks"]

There's no particular restrictions on what a stack can hold, except when declaring a global variable for the first time when the normal restrictions on not referencing other variables apply.

VAR my_stack = ["hello", 22, false, -> bananas]

== bananas
Hmmm. Bananas.
-> END

Mixing types up like this in a stack is generally not recommended, but you be you if you want to deal with the headaches it would cause down the line or you know exactly what you're doing and why.

Like most data in Ink, stacks are "immutable". That means if you want to add or remove something from a stack, you create a new stack with items added or removed instead. That sounds complex, but this is already exactly how things like strings and numbers work in Ink.

VAR my_number = 5

Who is Number 1?
You are Number {my_number + 1}!

// my_number is still 5, not 6

VAR my_stack = ["strawberries"]
You eat your {my_stack + "cream"}.

// my_stack is still ["strawberries"]

We say two stacks are equal if they have the same number of items, and all of the items in the two stacks are equal to each other when compared in order.

VAR stack1 = ["one", "two"]
VAR stack2 = ["one", "two"] // equal to stack1
VAR stack3 = ["two", "one"] // not equal to stack1
VAR stack4 = ["one"] // not equal to stack1

And we can 'add' or 'substract' with stacks, although with a slightly different meaning to numerical addition and substraction. Addition concatinates the two stacks (i.e. sticks them together, in order).

VAR stack1 = ["one", "two"]
// ["one", "two", "three", "four"]
{stack1 + ["three", "four"]}

While subtraction looks for the first time each value in the subtrahend (the stack on the right) appears in the minuend and removes it if it exists. It is not an error to 'subtract' a value that isn't in the original stack, it just results in an unchanged stack.

VAR stack1 = ["three", "one", "one", "two"]
// In order, we remove the first "one" (leaving the second) then the "three"
// ["one", "two"]
{stack1 - ["one", "three"]}

For the other mathematicians in the audience: these operations are not inverses, and addition is not commutative. I apologize for the brain hurty.

In the example visual novel Tournament you play a martial artist sent out into the world by your sensei. If you want to try it, you can play it here (it's a couple of minutes long). The novel opens by establishing what is important to your sensei, where you flashback to your training and select what he taught you as foundational, as being important, and as final polish.

These are the qualities in question:

LIST qualities =
  strength, skill, control, 
  speed, honour

Ideally, it would be trivial to just add the qualities to a variable of some kind in order, at which point you'd immediately know which is which. Instead, we end up having to create multiple variables and keep track of which one it is we're filling.

VAR most_important = honour
VAR important = honour
VAR least_important = honour

We discussed in one of the lessons the idea of a 'card battler' mechanic within our visual novels, and quickly realized that keeping track of a deck of cards within Ink that could have cards added to it during the story (especially duplicate cards) would be very hard if not impossible.

Fortunately, all is not lost. Ink allows compiler plugins, and VisualInk now implements one for STACKs.

A stack definition consists of a name, a maximum size, and the "empty" value for the stack - the value you'll be given if you ask for a stack slot that isn't filled, or "pop" a value from an empty stack. It allows us to do things like this:

LIST qualities =
  strength, skill, control, 
  speed, honour

STACK sensei_values 3 strength

~sensei_values_push(strength)
~sensei_values_push(skill)
~sensei_values_push(honour)

// Oops, sensei had a change of heart
~sensei_values_set(3, speed)

Sensei values {sensei_values_index(1)}, {sensei_values_index(2)}, <>
{sensei_values_index(3)}.

// Outputs: Sensei values strength, skill, and speed.

As you can see, the stack tracks values being pushed into it, and can change and read values at specific indexes (the first value in at 1, not 0, for the programmers among you).

The push and pop functions also let you know if you're successfully pushing or popping if that's important. Push returns true for success, and false if the stack is full, while pop will return the "empty" value you specified if the stack has no more values.

STACK cards 2 false

-> draw_card

=== draw_card
~temp did_draw = cards_push(RANDOM(1, 10))
{ did_draw:
  You now have {cards_last} cards in your hand
  -> draw_card
- else:
  Your hand is full
  -> play_card
}

=== play_card
~temp next_card = cards_pop()
{ next_card == false:
  You're out of cards!
  -> END
- else:
  You play a {next_card}.
  -> play_card
}

// Draws two cards and then plays two cards

We have you covered! The plugin is just generating Ink code to include under the hood (see below), and you can get the same code by browsing to https://visualink.mavnn.eu/inkTools/cards/2/false and copying and pasting the code there into a file to include in your project manually. Just change the last three parts of the address to match what you need.

Things get immediately technical from here on in, so if you just want to use stacks you can feel free to leave at this point! If, on the other hand, you're interested in extending Ink in a larger coding project, here's how I did it.

The implementation of this is two fold. Firstly, we have a IPlugin for the Ink compiler that does a simple search and replace on the script provided. It looks for lines in the correct format, and replaces them with INCLUDE lines with, er, slightly odd filenames.

module VisualInk.Server.VisualInkPlugin

open Ink
open System.Text.RegularExpressions

let private StackDefinitionRegex =
  Regex(
    "^STACK\s+(?<label>\w+)\s+(?<size>\d+)\s+(?<nil>.*)$",
    RegexOptions.Multiline ||| RegexOptions.Compiled
  )

let private StackIncludeRegex =
  Regex(
    "^VisualInkStack (?<label>\w+) (?<size>\d+) (?<nil>.*)$",
    RegexOptions.Multiline ||| RegexOptions.Compiled
  )

type PluginInclude =
    | StackInclude of string * int * string

type VisualInkPlugin() =
  interface IPlugin with
    member _.PreParse(storyContent: byref<string>) : unit =
      storyContent <-
        StackDefinitionRegex.Replace(
          storyContent,
          "INCLUDE VisualInkStack ${label} ${size} ${nil}"
        )

    member _.PostExport
      (parsedStory: Parsed.Story, runtimeStory: byref<Runtime.Story>)
      : unit =
      ()

    member _.PostParse(parsedStory: byref<Parsed.Story>) : unit = ()

This means we replace lines on a 1-1 basis, preserving lint error locations and similar without having to do any complex mapping work.

In our implementation of Ink.IFileHandler (responsible for finding included Ink files for the compiler), we then add a check for filenames matching the include regular expression above. If it matches, we provide generated content instead of the content of an other actual file.

The actual Ink code driving the stack is simple, but very verbose. Here's the F# code for generating it:

let generatePluginInclude (StackInclude (label, size, nil)) =
    let last = $"{label}_last"
    let content = seq {
      // Set up storage
      yield $"VAR {label}_size = {size}"
      yield $"VAR {last} = 0"
      for i in 1 .. size do
        yield $"VAR {label}_{i} = {nil}"
      yield ""

      // Access via index
      yield $"=== function {label}_index(index)"
      yield "{ index:"
      for i in 1 .. size do
        yield $"  - {i}:"
        yield $"    ~return {label}_{i}"
      yield "}"
      yield ""

      // Set via index
      yield $"=== function {label}_set(index, value)"
      yield "{ index:"
      for i in 1 .. size do
        yield $"  - {i}:"
        yield $"    ~{label}_{i} = value"
      yield "}"
      yield ""

      // Push
      yield $"=== function {label}_push(value)"
      yield $"{{ {last} >= {size}:"
      yield "  ~return false"
      yield "- else:"
      yield $"  ~{last}++"
      yield $"  ~{label}_set({last}, value)"
      yield "  ~return true"
      yield "}"

      // Pop
      yield $"=== function {label}_pop()"
      yield $"{{ {last} <= 0:"
      yield $"  ~return {nil}"
      yield "- else:"
      yield $"  ~temp result = {label}_index({last})"
      yield $"  ~{label}_set({last}, {nil})"
      yield $"  ~{last}--"
      yield "  ~return result"
      yield "}"
    }
    content |> String.concat "\n"

I think that's about it for now; at the moment I'm only really expecting this to be a VisualInk specific plugin, so it isn't independently available. If you're an Ink user and you think you'd find it hopeful, I can always package up a sharable version of it. Let me know!

Comments or thoughts? The blog announcement post is a good place to leave them!

The very first thing you need to understand about Effect is that it is built around a specific type. Appropriately enough, the Effect type. This type functions as a replacement for Promise, allowing for asynchronous code, but also covers a few other needs.

Let's get started, in the traditional way:

import { Effect } from "effect"

const hello = (name?: string): Effect.Effect<string> =>
  Effect.gen(function* () {
    return `Hello, ${name || "world"}!`
  })

This code is spiritually the same as writing:

const helloAsync = async (name?: string): Promise<string> => {
  return `Hello, ${name || "world"}!`
}

You can think of the Effect.gen(function* () { ... }) as a slightly more verbose version of the async prefix to a function.

What about calling the function? Again, we have a similar concept but different syntax to async/await.

// Effect version
const sayHello: Effect.Effect<void> =
  Effect.gen(function* () {
    const greeting = yield* hello("world")
    yield* sendGreetings(greeting)
  })

// Async/await version
const sayHelloAsync: Promise<void> = {
  const greeting = await helloAsync("world")
  await sendGreetingsAsync(greeting)
}

Given that we're using a cooperative scheduling system for asynchronous code, you can think of yield* as saying "hey runtime - I'm yielding the thread now, start me back up when the effect on the right has finished."

There is, however one big difference between these two which will catch you out if you don't know about it.

Promise is what is sometimes referred to as a "hot" or "immediate" asynchronous construct. What does that mean?

Well, in the example above sayHelloAsync is a constant value. As soon as the value is created, the logic within the promise will be scheduled for execution and we will start the process of asynchronously running the helloAsync function. If we await the result of sayHelloAsync somewhere else, we will block until the function has finished.

This also means that if we await sayHelloAsync in two different places, we will only send greetings once. After all, the Promise type that we are handing around represents the running execution of an asynchronous process; it has already started, and awaiting it multiple times won't start it again.

Effect, by contrast, is a "cold" or "thunked" asynchronous construct. It represents a series of steps that will be executed if the result is requested.

So if we run:

const camelot: Effect.Effect<void> =
  Effect.gen(function* () {
    yield* sayHello
    yield* sayHello
    yield* sayHello
  })

We will send the greetings three times, but only when somebody requests the result of running camelot. Until then, nothing will happen at all.

If you remember nothing else from this post, remember that an Effect that nobody executes never runs. This will catch you out with logging.

// logs nothing
const camelot: Effect.Effect<void> =
  Effect.gen(function* () {
    Effect.log("Bass voice: spam a looooooot")
    yield* sayHello
    yield* sayHello
    yield* sayHello
  })

// logs the bass voice
const camelot: Effect.Effect<void> =
  Effect.gen(function* () {
    yield* Effect.log("Bass voice: spam a looooooot")
    yield* sayHello
    yield* sayHello
    yield* sayHello
  })

Although the ability to create Effects without executing them can be extremely useful, leaving yield* out of your code when you actually need it is probably the most frustrating element of learning Effect, and the one that the compiler is least able to warn you about.

Effect makes heavy usage of a concept it calls "pipes", which is a way of passing a value through a series of functions.

The main place you'll see this is that we can wrap Effect values in a wide variety of ways. These are very powerful, but quickly start suffering from the "lisp effect" of a pyramid of brackets when we start combining them.

Let's rewrite the camelot function above using the built in Effect.repeatN method:

const camelot: Effect.Effect<void> =
  Effect.gen(function* () {
    yield* Effect.repeatN(2)(sayHello)
  })

Now let's add a timeout on finishing the 3 calls to sayHello; greetings are time critical after all:

const camelot: Effect.Effect<void> = Effect.gen(function* () {
  yield* Effect.timeout("10 seconds")(Effect.repeatN(2)(sayHello))
})

You can see this is already becoming quite hard to follow. This is where pipes come in. We can rewrite the above to become:

const camelot: Effect.Effect<void, TimeoutException> = Effect.gen(function* () {
  yield* sayHello.pipe(
    Effect.repeatN(2),
    Effect.timeout("10 seconds")
  )
})

You'll see this a lot in most code bases as the Effect library contains "pipeable" methods for everything from error handling to naming pieces of code for telemetry.

In that last example, Effect suddenly grew its second generic parameter. The timeout wrapper adds the explicit possibility that camelot can fail with a TimeoutException.

Do not get caught out! This doesn't mean that camelot cannot throw other exceptions; it means that we do not consider camelot timing out to be a "defect" but a known behavior which we can take action on.

This is worth spending a moment on, as it can get confusing. Effect executions can "finish early" in three main ways:

• An exception is thrown. This is referred to as a "defect", and the Effect interrupted is said to "die".
• A specific, known, failure type is explicitly returned. The type will be part of the type signature of the Effect, and if this happens the Effect is said to "fail".
• The Effect runtime can interrupt an Effect that is in flight because the result is no longer needed. These Effects are "interrupted".

For example, if the timeout is triggered in the code above the camelot function will "fail" with a TimeoutException while the call to sayHello which has taken too long will be "interrupted". If, instead, sayHello throws an exception because the network is down then it will "die" and then in turn cause the camelot Effect to "die" as well.

With the addition of the "empty" case (nothing went wrong!) these four categories are used to define the Cause of an Effect completing execution.

Here we hit a piece of (in my opinion) terrible naming on the part of the library authors. The Effect.catch wrapper that as a TypeScript developer you would expect to, you know, catch thrown exceptions only catches "failures" (i.e. the known, type safe, errors deliberately returned). This is very useful in allowing you to apply logic in code that depends on known failure routes, but if you're actually looking for what you thought Effect.catch did you're really looking for Effect.catchAllCause.

If you want to signal that a function should return early due to an error, you can either use Effect.fail or you can use any error that is a subtype of YieldableError (part of the Effect library). This introduces the slightly weird construct of return yield*, as you can see below.

const canGoWrong = (input: number) => Effect.gen(function* () {
  if(input < 0) {
    return yield* Effect.fail("Input must be positive")
  }

  if (input > 10) {
    return yield* new IllegalArgumentException("Input must be not too big")
  }

  yield* sendGreeting(" Moderate Number Inputter")
  return "Success!"
})

Why can't you just call return? The answer is it messes up the types; the function above has the type:

(input: number) => Effect.Effect<string, string | IllegalArgumentException>

But if we were to write:

const canGoWrong = (input: number) => Effect.gen(function* () {
  if(input < 0) {
    return Effect.fail("Input must be positive")
  }

  if (input > 10) {
    return new IllegalArgumentException("Input must be not too big")
  }

  yield* sendGreeting(" Moderate Number Inputter")
})

...then it would end up with the type:

(
  input: number
): Effect.Effect<string | IllegalArgumentException | Effect.Effect<never, string>>

What's happening is that without the yield* we are returning the failures as possible success values. That probably isn't what you want!

In general, if you ever find yourself with an Effect of an Effect, you're probably missing a yield* somewhere.

One of the most powerful features of Effect in day to day usage is the built in, type safe, dependency management. Let's apply some inversion of control to our sendGreetings service.

Effect allows us to build "services", which are classes that extend Context.Tag to specify what interface their implementations will provide.

For example, we can specify a service for sending greetings that looks like this:

import { Effect, Context } from "effect"

interface ISendGreetings {
  sendGreetings(name: string): Effect.Effect<void>
}

export class SendGreetings extends
  Context.Tag("SendGreetings")<SendGreetings, ISendGreetings>() {}

And now we can rewrite camelot to use the service:

const sayHello: Effect.Effect<void, never, SendGreetings> = Effect.gen(function* () {
  const sender = yield* SendGreetings
  const greeting = yield* hello("world")
  yield* sender.sendGreetings(greeting)
})

const camelot: Effect.Effect<void, TimeoutException, SendGreetings> = Effect.gen(function* () {
  yield* sayHello.pipe(Effect.repeatN(2), Effect.timeout("10 seconds"))
})

This is where we see Effect's third and final generic parameter, which tracks which dependencies your code requires.

"But I thought we could only yield* Effects?": well, it turns out this isn't quite true. It turns out that as you can make your own Awaitable types in TypeScript that are not Promises, you can also implement alternative Yieldable types in Effect. And that's exactly what the pre-provided Context.Tag class does as a static interface. Which means, slightly bizarrely, you can yield* the name of the class and it will then run all of the dependency injection logic needed to go and get you an actual implementation at run time.

In general, you just want to let Effect build up required dependencies itself; if we add a second service then the type system will capture that "automatically". If possible, leave the types of your Effect functions inferred as then it will automatically pick up changes in their dependencies. That said, I'll carry on adding them here so you can see what's happening.

const sayHello: Effect.Effect<void, never, SendGreetings | TranslateGreeting> = Effect.gen(
  function* () {
    const sender = yield* SendGreetings
    const translator = yield* TranslateGreeting
    const greeting = yield* hello("world")
    const translated = yield* translator.translate(greeting)
    yield* sender.sendGreetings(translated)
  }
)

Note that the types only capture direct dependencies. Here, the TranslateGreeting service almost certainly depends in turn on some kind of user or session service, but we don't need to worry about that. This makes providing alternative service implementations for tests, or for running on the client versus the server, exceedingly straight forward and safe. If you can provide exactly what your code needs you're good, and it won't ask you for any more than that. If your Effect code is starting to build up a huge list of dependencies, that's normally a good indicator that you want to start wrapping it in an interface of its own - that way, the things that call your code don't inherit a huge dependency tree, and they in turn become more maintainable.

It is the responsibility of the entry point to the runtime to make sure that all services are provided, which is done via the Layer type which provides facilities to manage service implementations with caching and life cycle management. But that, unfortunately, is definitely the subject of a complete write up of its own.

"How to ADHD" has lots of material on what ADHD is, how it manifests, and what you can do about it: https://howtoadhd.com/

A paper reviewing research on working memory in adults with ADHD: https://psycnet.apa.org/record/2013-16996-001

Mads Torgersen (lead designer of C# at Microsoft) talks about his own diagnosis on the No Dogma Podcast: https://nodogmapodcast.bryanhogan.net/165-mads-torgersen-adhd/

(Bonus extra: that last link has more links to more ADHD resources)

Finally, I mentioned at one point an organization tool that happens to mesh fairly well with my own variant of ADHD called SkedPal: https://skedpal.com/. You prioritize tasks in advance, and assign them to "time maps" (i.e. this times are work times, these times are home times). Then you hit the button and it suggests a calendar of tasks. I've found it useful because it allows things like "this task is important to me but it doesn't matter when it is done" and "this task isn't very important but if I'm going to do it, it needs happen by Friday" and it will suggest a sane next thing to do. And when you (inevitably) fail to actually follow the plan, you just hit the button again and it suggests a new plan based on the things you actually did rather than the things it thought you were going to do. Caveat: it's not free software, and it does charge a monthly fee.

Dan Luu's tour de force review of research into whether or not strong typing leads to more reliable code: https://danluu.com/empirical-pl/

Brian Marick has a well thought out post on why he's not fully convinced by property based testing: https://www.crustofcode.com/a-reluctant-rebuttal/. Most importantly he links to a paper written in 1990 that partition testing has some issues, all of which would also apply to property based testing: https://www.site.uottawa.ca/~gvj/papers/Software%20Engineering%20IEEE%20Transactions%20on%201990%20Hamlet.pdf. It is worth noting that both Marick (about PBT), and Hamlet and Taylor (about partition testing) do state that they see use cases for these testing methods, but that they do also have concerns.

The arguments I've heard against domain driven design have tended to be more anecdotal but mostly boil down to: "you're making the code harder to understand by forcing developers to understand both the code and the specialist terminology of the users at the same time."

I happen to be a fan of Scott Wlaschin's video and blog posts at https://fsharpforfunandprofit.com/pbt/ titled "The lazy programmer's guide to writing 1000's of tests".

Otherwise, a quick google search for "John Hughes" will net you many talks from the author of the first property based testing framework.

Fuzz testing is a related topic which somewhat overlaps, but with a different focus.

I cannot over stress how amazing a book "Domain Modeling Made Functional" is (again, by Scott Wlaschin) https://pragprog.com/titles/swdddf/domain-modeling-made-functional/. There is also a talk covering the basics available at https://www.youtube.com/watch?v=MlPQ0FsPxPY.

Probably the place to start is the Idris programming language https://www.idris-lang.org/

You may remember from part 1 that HTMX and Falco's markup library are both tools I'm trying out for the first time. This means that while I'm happy with the results I achieved in this post, I'm not all that happy with the resulting code. Yet. There will be a refactoring follow up.

Which translates to: don't take anything as an active recommendation of how to do things, but a chance to follow along as I learn a new tool.

My first attempt at nice server side UI building hinges on two key ideas. 1) each domain module should be responsible for its own UI requirements and 2) the overall UI should look coherent.

This sounds like a place for a style guide, so I created a StyleGuide directory and started hacking. We ended up with four files in here, each with their own little area of responsibility.

With our helpers constructed, we can start using them. Simple full page endpoints are quite simple; we just swap in the new Layout functions and we're good to go. For example, the view for the home page now looks like this:

Layout.page
  "Home"
  [ Layout.containerSection
      [ Layout.title
          Layout.T1
          (match user with
            | Some u -> $"Hi {u.username}!"
            | None -> "You should go log in!")
        Layout.paragraphX
          []
          [ Text.raw "Would you like to "
            Layout.link
              (greeting.greetingLink "Bob")
              "greet Bob?" ] ] ]

As soon as we get to adding things like navigation bars to the page template they will all just appear.

The magic, again, begins in the User.fs module. Let's have a think about the request life cycle with HTMX.

Commit diff

The first commit eliminates a surprisingly common source of bugs in a project: mismatched dependency versions between what you test, and what you deploy.

To help combat this, in 2022 the NuGet introduced "central package management" which is a mechanism to allow each of your project files to specify which packages it depends on, while managing the versions of all packages across your whole repository in one central location.

Given that a new major version of Marten was released recently and I wanted to upgrade to use it, it seemed an ideal moment to put in a top level Directory.Packages.props file and remove the version numbers of dependencies from our fsproj files. An entire category of bugs eliminated permanently.

The only code changes in this commit are to account for changes to how custom logging is implemented in Marten 7.0.

Commit diff

Talking about logging, our second commit enhances the logging we added to Marten to make sure that we carry through the RequestId assigned by AspNetCore to any Marten operations. We also add an environment variable switch to change over to structured JSON logging in our packaged docker container; this is considerably more verbose but means that we always know which component is logging and which request the log relates to when we start feeding the logs through to a log aggregator in production.

Commit diff

I've been lax on testing so far, and the place where that bothered me most was that I wasn't completely certain that the type safe route definition library I'd built would actually construct links that would "round trip" correctly through the AspNetCore machinery.

The idea is that I can define a route definition like so:

let private greetingRoute =
  literalSection "/greetings/" ./+ stringSection "name"

And using that route definition I should be able to create links to an endpoint that receives any path parameters without them being changed.

// Any path I create with this function...
let link yourName = greetingRoute.link yourName

// ...should get handled by this endpoint, and
// ~greetingHandler~ should receive ~yourName~ as an input
let greetingEndpoint =
  Handler.toEndpoint get greetingRoute greetingHandler

I wasn't sure how strings requiring URL escaping would be handled, so I added unit tests that actually call the underlying AspNet libraries to make sure I wasn't going to have any unpleasant surprises.

It was a good thing I did, too, because the code did in fact do the wrong thing with strings that needed escaping. So this PR also includes the fixes.

You can see the resulting test code in the new RouteDef file in Server.Test, which also shows just how easy it is to create a set of parameterized tests in Expecto.

Commit diff

In the previous post I said that I was a bit unhappy with how much of its internals the user domain module was exposing, and maybe I should give a standard way of a domain context to define itself - but it would be premature to do so with only one context.

Then I realized that in some ways I already had a couple of other contexts; the home page, which you could claim is a bit borderline to call a context, and the greetings functionality which allows you to greet somebody.

In a way this is smoke and mirrors; I'm well aware that these are not really bounded contexts within a domain in the way that we mean when talking about domain driven design. But at the same time, the point of writing this sort of "hello world" code is precisely because it starts telling you enough about the system you're building to be able to start designing based on reality rather than a set of assumptions.

Looking at the code in question, it became clear that one thing would definitely already be helpful: an interface defining what endpoints a domain context provides and what config it needed to add to Marten.

That led to the DomainSetup module:

module Mavnn.CalDance.DomainSetup

open Falco
open Marten

type IConstructedContext =
  abstract member endpoints: HttpEndpoint list
  abstract member martenConfig: StoreOptions -> unit

A bit of rearranging later, and we now have three domain modules all which export a context class that both implements the interface above and is also a convenient place to expose any link builders that the module wants to expose. A lot of other code could then immediately become private to each module.

If you're an F# developer (or interested in becoming one) I hope the details of the commits are helpful. But there's a bigger take away here: names don't just matter in our code; talking to people with terminology that is easy for them to grasp and which highlights the areas of shared importance on all sides is an enormously valuable skill. You may well struggle to explain why you want to spend time refactoring ("you want to spend time making changes to the routing module that don't change what the code does?"), but "we need to improve the internal quality of the routing module so that we can write new features more quickly and correctly" is probably much easier to get agreement about.

I hope you're enjoying this journey of discovery with me - as always, if you have questions or comments all of the code is in the CalDance repository on GitLab. And if you'd like someone to help you keep the internal quality of your code base high then reach out about my short term consultancy services.

Next time: starting to shape up our actual user interface.

To borrow a term from domain driven design, this sounds like a "bounded context" within our system. Other parts of the code may care about certain events happening related to users (users being created, that kind of thing), but they probably shouldn't know or care about how the internals of "a user" work or what it takes to authenticate a user.

There are as many ways of organizing your code as there are grains of sand on the beach, but fundamentally all of the ones that help are about choosing where to have boundaries in your code base.

We are going to have three horizontal slices; shared library code, domain logic (our "business" code), and execution environment. Vertically we're going to slice the domain logic by bounded context - of which, admittedly, we only have one at the moment.

We end up with something like (things further down the table depend on the things above):

You'll notice that this doesn't group the code by the technical task the code is trying to achieve, a pattern you'll often find in example project templates where you'll end up with a "Controllers" directory and a "Views" directory. It's also not an organization along "clean/hexagon/ports and adapters" lines with a strict demarcation between code that speaks to the outside world achieved via interfaces and abstractions.

It's not that I feel that either of those patterns has no merit (although I feel like the main driver of the first pattern is that you can suggest it even for projects you know nothing about which is a useful property when writing templates and dispensing nuggets of wisdom at conferences about the one true way to organize code). But I do feel that for the vast majority of code bases, it is a far bigger gain to productivity to be able to co-locate code by purpose than by type.

Let's face it: while you sometimes pick up a story/card/work ticket that requires you to go and change all the controllers (normally during dependency upgrades), or replace all the database interface implementations (you're about to have a long few months), it is much more likely on a day to day basis that you're trying to add a new field to the data we store about users, and you want to update the data store, business logic, and UI of users to be able to do that. Taking this logic to its logical extremes leads you towards microservices - but that starts to bring in a different type of complexity of its own.

All of this to say: there's now a folder called Domain which holds our new, shiny, user management code in a file called: drumroll, please User.fs. Let's have a look at it in detail.

module Mavnn.CalDance.Domain.User

open Falco
open Falco.Routing
open Falco.Markup
open Falco.Security
open Marten
open Marten.Events.Aggregation
open Marten.Events.Projections
open Mavnn.CalDance
open Mavnn.CalDance.Routing
open System.Security.Claims
open Microsoft.AspNetCore.Identity

As just mentioned, this module is going to be responsible for the whole vertical slice of the application for user management, so we start by including everything we need from the data store (Marten) through to the UI (Falco.Markup). We could have created sub modules within a Users folder if needed, but the module is only ~300 lines long so I haven't split it up (yet).

type User = { id: System.Guid; username: string }

type UserState =
  | Active
  | Disabled

[<CLIMutable>]
type UserRecord =
  { Id: System.Guid
    Username: string
    PasswordHash: string
    State: UserState }

type UserEvent =
  | Created of UserRecord
  | PasswordChanged of passwordHash: string
  | Disabled

Next we define a few data types that represent our users, and the events that can happen to them over time. This is important because we are "event sourcing" the state of our users, meaning that the golden source of truth for what state a user is in is defined by what events have happened to them so far. The two representations of the user represent what we care about in the running system (the main User type) and what we need to store about them on disk (the UserRecord type); in general we would expect that other modules might make use of the User type but in general they should not make use of the UserRecord type. Its an open question in my mind whether it should actually be marked as a private type declaration, but I've erred on the side of leaving it available for now.

A minor implementation detail: to try and keep the incremental steps of the project manageable I'm using the default (de)serializers for Marten, which require the object to be deserialized from the data base has a default constructor and mutable fields, which we get from the [<CLIMutable>] attribute. We'll probably remove that going forwards by switching to a serialization strategy that works with immutable F# records.

The life cycle of our users is very simple at the moment; a Created event signals that a new, active, user was created. That user can change their password, or they can be marked disabled which effectively ends the lifecycle of the user. There's no way to reactivate a user now, although we could always add one later.

type UserRecordProjection() =
  inherit SingleStreamProjection<UserRecord>()

  member _.Create(userEvent, metadata: Events.IEvent) =
    match userEvent with
    | Created user -> user
    | _ ->
      // We should always receive a created event
      // first so this shouldn't ever happen...
      // ...but it might, and we don't want to throw
      // in projections.
      { Id = metadata.Id
        Username = ""
        PasswordHash = ""
        State = UserState.Disabled }


  member _.Apply(userEvent, userRecord: UserRecord) =
    task {

      match userEvent with
      | Created _ ->
        // Should never occur after the first event in the stream
        // so we ignore duplicates
        return userRecord
      | PasswordChanged passwordHash ->
        match userRecord with
        | { State = UserState.Disabled } ->
          // Don't update password of disabled users
          return userRecord
        | user ->
          return
            { user with
                PasswordHash = passwordHash }
      | Disabled ->
        match userRecord with
        | { State = UserState.Disabled } ->
          return userRecord
        | { State = Active } ->
          return
            { userRecord with
                State = UserState.Disabled }
    }

Marten leans heavily into the code reflection capabilities of the dotnet framework, allowing us to configure our data store in terms of the in program types we want it to store. A "projection" in event sourcing is the logic which takes a list of events (our base line source of truth) and turns it into a current state, so this class defines a projection that will create and/or update UserRecord data in Marten's document store (we know it does this because it implements the SingleStreamProjection<UserRecord> interface). It will project from events of the UserEvent type, because that is the type of the first argument of the Create and Apply methods we have supplied.

There are a few conventions we need to follow here to allow for this minimalist a configuration. Our current state type must have an Id (or id) field of type string, uuid, or integer. And when an event matching the signature of our projection is pushed to a stream with an ID, the resulting update to the current status type must produce a document with the same ID as the stream ID.

We're treating our records as immutable objects (because we're planning to make them immutable going forward), so our create and apply methods return a Task<UserRecord>; if the document type was mutable we would also have the options of mutating it in place and returning void.

With that explanation out of the way, hopefully the state machine that represents our user life cycle is clear in the code above.

Now that we can store information about our users, and update them based on what is happening to them, it's time to start implementing the actual responsibilities of the module. We're keeping things minimal to get started, so we'll implement only the three things we really need: sign up, log in, and log out.

type LoginFormData = { username: string; password: string }

let findUserRecord (username: string) =
  Marten.withMarten (fun marten ->
    marten
      .Query<UserRecord>()
      .SingleOrDefaultAsync(fun ur ->
        ur.Username = username))
  |> Handler.map Marten.returnOption

let loginRoute = RouteDef.literalSection "/login"
let logoutRoute = RouteDef.literalSection "/logout"
let signupRoute = RouteDef.literalSection "/signup"

let getSessionUser: Handler<User option> =
  Handler.fromCtx (fun ctx ->
    match ctx.User with
    | null -> None
    | principal ->
      match
        (System.Guid.TryParse(
          principal.FindFirstValue("userId")
         ),
         principal.FindFirstValue("name"))
      with
      | ((false, _), _)
      | (_, null) -> None
      | ((true, id), username) ->
        Some { id = id; username = username })

A few definitions and helpers start us off; what data a form needs to capture for someone to sign up/log on, what urls exist and are managed by this module, and a couple of helper functions for obtaining a user record and a user session from the current HTTP context (using the Handler type we talked about in the last post).

let loginGetEndpoint =
  Handler.toEndpoint get loginRoute (fun () ->
    Handler.return' (
      Response.ofHtmlCsrf (fun csrfToken ->
        Elem.html
          []
          [ Elem.body
              []
              [ Elem.form
                  [ Attr.method "post" ]
                  [ Elem.input [ Attr.name "username" ]
                    Elem.input [ Attr.name "password" ]
                    Xss.antiforgeryInput csrfToken
                    Elem.input
                      [ Attr.type' "submit"
                        Attr.value "Submit" ] ] ] ])
    ))

Our first end point is straight forward. When we receive a get request to the login path, we reply with a form containing a token to prevent cross site vulnerabilities and username and password fields.

let private makePrincipal userRecord =
  let claims =
    [ new Claim("name", userRecord.Username)
      new Claim("userId", userRecord.Id.ToString()) ]

  let identity = new ClaimsIdentity(claims, "Cookies")

  new ClaimsPrincipal(identity)

let passwordHasher = PasswordHasher()

let updateUser (id: System.Guid, events: seq<UserEvent>) =
  handler {
    do!
      Marten.withMarten (fun marten ->
        task {
          // explicitly assign this as an array of objects
          // so that Marten chooses the correct method
          // overload for `Append`
          let eventObjs: obj[] =
            Array.ofSeq events |> Array.map box

          marten.Events.Append(id, eventObjs) |> ignore
          return! marten.SaveChangesAsync()
        })

    return!
      Marten.withMarten (fun marten ->
        marten.LoadAsync<UserRecord>(id))
  }

Our next end point is going to actually handle the form coming in, so it requires a few more helpers. The web framework we're using will handle things like sessions for us, but only if we "buy into" the .NET standard ways of representing a user, in this case using the ClaimsPrincipal type - so we have a helper to map from one of our user records to a claims principal. We initialize a password hasher which will salt and hash our passwords for us (don't roll your own crypto, folks, especially when your language ecosystem has a decent implementation ready for you). And finally we add an other method that works within our HTTP context expressions - updateUser takes the ID of a user and a list of events and returns the updated UserRecord.

With all of that in place, we can write the loginPostEndpoint.

let loginPostEndpoint =
  Handler.toEndpoint post loginRoute (fun () ->
    handler {
      let! loginData =
        Handler.formDataOrFail
          (Response.withStatusCode 400 >> Response.ofEmpty)
          (fun f ->
            Option.map2
              (fun username password ->
                { username = username
                  password = password })
              (f.TryGetStringNonEmpty "username")
              (f.TryGetStringNonEmpty "password"))

      let! userRecord =
        findUserRecord loginData.username
        |> Handler.ofOption (
          Response.withStatusCode 403 >> Response.ofEmpty
        )

      let verificationResult =
        passwordHasher.VerifyHashedPassword(
          userRecord,
          userRecord.PasswordHash,
          loginData.password
        )

      match verificationResult with
      | PasswordVerificationResult.Failed ->
        return
          (Response.withStatusCode 403 >> Response.ofEmpty)
      | PasswordVerificationResult.Success ->
        return
          Response.signInAndRedirect
            "Cookies"
            (makePrincipal userRecord)
            "/"
      | PasswordVerificationResult.SuccessRehashNeeded ->
        let! _ =
          updateUser (
            userRecord.Id,
            [ PasswordChanged(
                passwordHasher.HashPassword(
                  userRecord,
                  loginData.password
                )
              ) ]
          )

        return
          Response.signInAndRedirect
            "Cookies"
            (makePrincipal userRecord)
            "/"
      | _ ->
        return
          failwithf
            "Unknown password verification result type %O"
            verificationResult

    })

Time to actually use our handler expression in earnest! There is some personal preference in play here, but personally I really like the clear flow of the request we can see happening in this code. We either have the form data we need, or we return a 400 error. Then we either find a user record with a matching username, or we return a 403 error (we don't want to reveal whether a username exists or not, so we return the same code as for when the password is incorrect; security +1, helpful error messages to users -1). Then we check the password, and we either return 403 (if it is wrong) or log you in if it is correct. A minor piece of extra complexity is introduced by the fact that the password hasher may signal that the password is correct but the hash needs updating in storage, a background operation that the user does not need to know about.

I'll leave the other end points for the reader to read at their leisure on Gitlab, as they are either trivial (logoutEndpoint) or very similar to the log in end points (signupGetEndpoint and signupPostEndpoint).

Finally, we get to the end of the module where we export everything that the web server setup code (the bottom layer in my newly christened "julienned domain sandwich" architecture).

let endpoints =
  [ loginGetEndpoint
    loginPostEndpoint
    logoutEndpoint
    signupGetEndpoint
    signupPostEndpoint ]

let martenConfig (storeOptions: Marten.StoreOptions) =
  storeOptions.Projections.Add<UserRecordProjection>(
    ProjectionLifecycle.Inline
  )

At the moment, with only one domain, this is just an adhoc export of the end points we're wanting to add to the webserver and the projections we want to add to Marten. As the project grows, we'll probably add an interface that each of our domain modules will export which will provide to allow a standardized process for consuming the needed configuration. But there's little point trying to proactively create an abstraction over a single example of a pattern.

And there you have it; event sourced (basic) user management for our web application. If you have thoughts and questions, drop them as an issue on the CalDance repository. I'd love to see example repositories having in depth discussions of when the architecture they suggest is or isn't useful, even if (especially if!) that discussion includes comments critical of the architecture demonstrated.

Next up: a round of internal quality control.

We're about to add a database to our project, and this is an area where Nix really shines.

Adding services with pinned versions of dependencies to are development environment is as simple as adding them to the list in flake.nix:

devShells.default = pkgs.mkShell {
  buildInputs = [
    dnc.sdk_8_0
    pkgs.nixfmt
    pkgs.skopeo
    pkgs.overmind
    pkgs.tmux
    pkgs.postgresql
    fantomas
    format-all
    format-stdin
    local_postgres
  ];
};

The only clever thing we're doing here is also adding a local_postgres command which runs postgres with its data directory set to be a git ignored directory in the repository. This means that a simple git clean will reset the database along with everything else.

As a courtesy to developers who may work on code that isn't CalDance, we also set a non-standard port for postgres to use in our .envrc file so that we don't compete with any system wide installations that may already be running.

Overmind is a process runner that runs processes as defined in a Procfile, so we add one to the root of the project with the following:

server: dotnet watch --project Server/CalDance.Server.fsproj
postgres: local_postgres

Now we can run overmind s to start both postgres and a dotnet watcher to live recompile our server code as it changes.

We're adding dependencies to our server of Marten (document/event database library that sits on top of postgres) and Serilog (a nice structured log library).

Marten depends on a postgres library with native (i.e. non-dotnet) dlls, so to allow Nix to cache and link to the correct versions of the native code we have to specify which runtimes we expect to be building our code for. For the curious minded, you don't need to do this to be able to run dotnet build directly because the dotnet cli will dynamically download and add the required native libraries - which breaks Nix's caching strategy of a reproducible output from a fixed set of input files.

This isn't a huge issue once you know you need to do it; you just add a RuntimeIdentifiers node to your project files under the TargetFramework node like so:

<PropertyGroup>
  <OutputType>Exe</OutputType>
  <TargetFramework>net8.0</TargetFramework>
  <RuntimeIdentifiers>osx-arm64;linux-x64;linux-arm64</RuntimeIdentifiers>
</PropertyGroup>

Then we can add our nuget packages as normal and everything continues to work:

<ItemGroup>
  <PackageReference Include="Falco" Version="4.0.6" />
  <PackageReference Include="Marten" Version="6.4.1" />
  <PackageReference Include="Serilog" Version="3.1.1" />
  <PackageReference Include="Serilog.AspNetCore" Version="8.0.1" />
  <PackageReference Include="Serilog.Sinks.Console" Version="5.0.1" />
</ItemGroup>

In general, the code to handle an endpoint in an AspNet.Core application is a function from HttpContext to Task, where we mutate the HTTP context and then write the correct output stream.

Falco gives us an abstraction a little higher than that by giving us a set of composable functions for manipulating the HTTP context, which is already a step forward. But I was finding them harder to compose than I would like because in several cases the functions took two inputs and effectively "branched" the response that could be given - for example, do I have the form fields I expect in this POST request, or am I logged in.

I quickly realized that I'd be happier with some kind of "result" mechanism - a way to be able to declare during the specification of a handler that I wanted to short circuit from this point onwards with a failure response.

I also knew that I wanted a type safe way of writing handlers for paths with "place holder" sections.

Because of that, I added a Routing module in which I've defined a Handler type as below:

type Handler<'a> =
  HttpContext -> Task<HttpContext * Result<'a, HttpHandler>>

For the sharp eyed among you with functional programming experience you may have spotted this is the same shape as the monad type of a stateful either monad, and indeed we also define a computational expression called handler that allows us to now write our handlers in a more declarative style.

The revised indexEndpoint in the main program file gives a good example of what it looks like:

let indexRoute = literalSection "/"

let indexEndpoint =
  Handler.toEndpoint get indexRoute (fun () ->
    handler {
      let! user = User.getSessionUser

      return
        (Response.ofHtml (
          Elem.html
            []
            [ Elem.body
                []
                [ Elem.h1
                    []
                    [ match user with
                      | Some u ->
                        Text.raw $"Hi {u.username}!"
                      | None ->
                        Text.raw "You should go log in!" ]
                  Elem.p
                    []
                    [ Text.raw "Would you like to "
                      Elem.a
                        [ Attr.href (
                            greetingRoute.link "Bob"
                          ) ]
                        [ Text.raw "greet Bob?" ] ] ] ]
        ))
    })

Note the let! on the first line where we pull the user session out of the HTTP context which the computational expression is "invisibly" carrying along for us.

Having defined our handler type, it makes sense to make the rest of our tooling easy to use from within the abstraction.

The new Marten module contains some boiler plate to configure Marten and add Serilog logging to it, but most importantly it also adds:

let withMarten f =
  Handler.fromCtx (fun ctx ->
    ctx.GetService<IDocumentSession>())
  |> Handler.bind (f >> Handler.returnTask)

// Marten returns null if a record isn't found, but
// F# records declare they can't be null. This works
// around that to return an option instead
let returnOption v =
  if (v |> box |> isNull) then None else Some v

Now from within any HTTP handler we're writing, we can write code like:

let! user =
  Marten.withMarten (fun marten ->
    marten.LoadAsync<UserRecord>(id))

...and as if by magic the request specific Marten session will be pulled out of the HTTP context of the request and we can use it to connect to our data source.

I think that's about enough for this blog post, because I want to leave a whole post for the real meat of this set of changes: our first domain entity, the User.

If you want a sneak peak, you can check out the PR and see how we can define a neat vertical slice of responsibility in our code base. The module takes the responsibility for user management all the way from the domain object, the events that can happen to it, the Marten config to make sure those are tracked, through to the paths that it has responsibility for and the UI that will be displayed when they are requested. Lots of fun stuff for us to talk about in the next exciting installment of "Dev Journal": different time, multiple channels, next week.

Next up: Log in, log out (part 2)

TypeScript has one form of monad notation already - the await keyword. Unfortunately, there isn't any way to plug into the mechanism used and define your own alternative bind implementation without doing something dangerously hacky. And, frankly, the last thing your TypeScript code needs is an other sharp edge to cut yourself on.

But... what does binding a value in monad notation really do? It doesn't allow you to write code you couldn't have written anyway long hand. It allows you to give the result of a calculation in your code in name in the current scope.

So: if we consider the fact that a scope is really just a mapping from names to values, and that TypeScript allows function inputs to alter the type of their output... maybe we can do something with that?

A type that maps names to values is reasonably easy to define in TypeScript. It looks something like this:

export type Scope<Keys extends string> = {
  [K in Keys]: any;
};

We can say that anything we're willing to consider as a scope is a type that extends the type above: it will have some keys, which will all be strings, and they will map to some values, which will all be sub types of any.

Now we need a type safe way to add a value to the scope.

We start with a calculated type which works out what the result of adding a value with a name to a scope should be:

export type ExtendedScope<
  OldScope extends Scope<any>,
  NewField extends string,
  NewValue
> = OldScope extends any
  ? {
      [K in keyof OldScope | NewField]: K extends NewField
        ? NewValue
        : K extends keyof OldScope
        ? OldScope[K]
        : never;
    }
  : never;

ExtendedScope<ExtendedScope<{ aDate: DateTime }, "aNumber", number>, "aString", string>

{
    aDate: DateTime,
    aNumber: number,
    aString: string
}

The next question we need to answer is what is this going to be the scope of. Monad friendly syntax normally binds names in the scope of a function body, so let's generalize that idea slightly (and with a lot of hand waving) to the idea of the scope of an operation.

We want to be able to do two main things with this operation: we want to be able to add steps to it (using the scope accumulated so far), and we want to be able to execute it. The idea of building up an operation like this brings to mind the appropriately named "builder" pattern, where we use a class with some state to "build" up the thing we want. I'm going to call the "add step" method chain (because it feels a fairly intuitive name for what's happening) and the "execute" method execute (no reason needed). To keep the theme, we'll call the builder class a SomethingChain where something is a monad.

If such a builder class were to exist, it would allow us to write code like this:

export const monadTest = new MonadChain((name: string) =>
  Monad.pure({ initialInput: name }),
)
  .chain("punctuation", () => Monad.pure("!"))
  .chain("finalGreeting", (scope) =>
    Monad.pure(
      `Hello ${scope.initialInput}${scope.punctuation}`,
    ),
  ).execute;

The type of monadTest above is a function that takes a string, and returns a monad value of an object with three string fields: "initialInput", "punctuation", and "finalGreeting". As you can see, each step in the chain adds to or overwrites a value in the scope while leaving the rest of the scope available for use later.

The pattern of how we implement the builder for each type of monad is going to be pretty much the same (that's the point), but we can also play some neat tricks based on the specific monad we're implementing. The magic is in the chain method, which wraps a perfectly normal "bind" with a call to extendScope from above. Let's see the pattern first in its purest possible form by creating an identity monad chain, along with some comments in the chain method.

export type Monad<T> = T;

export namespace Monad {
  export const bind = <A, B>(
    prev: Monad<A>,
    func: (prev: A) => Monad<B>,
  ): Monad<B> => {
    return func(prev);
  };
  export const pure = <A,>(value: A): Monad<A> => value;
}

export class MonadChain<Input, OutputScope extends Scope<any>> {
  private readonly operation: (
    input: Input,
  ) => Monad<OutputScope>;
  constructor(
    starter: (input: Input) => Monad<OutputScope>,
  ) {
    this.operation = starter;
  }

  execute(input: Input): Monad<OutputScope> {
    return this.operation(input);
  }

  chain<NextOutput, ResultName extends string>(
    resultName: ResultName,
    next: (previous: OutputScope) => Monad<NextOutput>,
  ): MonadChain<
    Input,
    ExtendedScope<OutputScope, ResultName, NextOutput>
  > {
    // We need to build the operation to pass to
    // the constructor of the chain we're going
    // to return
    const chainedOperation = (
      input: Input,
    ): Monad<
      ExtendedScope<OutputScope, ResultName, NextOutput>
    > => {
      // Get the monadic value of the chain
      // up to this point.
      const previousResult = this.operation(input);
      return Monad.bind(
        previousResult,
        (output: OutputScope) => {
          // Use the output from the chain so far /twice/.
          // The first time to work out the new value that is
          // going to be added to the scope, and the second
          // time to as the starting scope passed to ~extendScope~
          return Monad.bind(next(output), (nextOutput) => {
            return Monad.pure(
              extendScope(output, resultName, nextOutput),
            );
          });
        },
      );
    };
    // Wrap the function back into the builder class
    // so we can carry on calling chain on it
    return new MonadChain(chainedOperation);
  }
}

For our last example we're going to both take things up a notch, and take a bit more advantage of TypeScript's type level programming.

We're going to create what I'm going to call, for brevity, the Solid monad. (Technically it's an asynchronous either state monad - mix and match order to taste - but that's a bit of a mouth full). This allows us to think about how to write our code in a slightly different way, separating out the logic of our "happy path" from error handling in a way that would be either very verbose or completely break type safety without some tooling like this builder.

Let's first look at the kind of use cases where this monad is useful. Imagine we've created an integration with the "Lax" messaging service, and now when a customer clicks a button in one of the messages we send out our code has to handle a call back from Lax. But we don't know anything about the incoming request initially. We don't know whether it actually comes from Lax, whether we can find the customer organization that the message was sent to, which user of Lax in that organization clicked the button, which user in our application that Lax user maps to...

This uncertainty means there's also numerous ways this process could go wrong. Maybe the message isn't actually from Lax (who cryptographically sign their callback payloads). Maybe our database is down. Or maybe Bob clicked the button 30 seconds after Fred did and you can't do that thing anymore. We also need to respond differently and to different people depending on what has failed.

To deal with this, each step in the process can either succeed or fail. What we want to do in the case of failure is often highly dependent on both what type of failure has occurred and what information we already have available. As such, it's common to see code that either returns a union type of Success | SemanticallyMeaningfulErrorType(s) or where each operation in turn is wrapped in try ... catch blocks to respond to exceptions in context.

Normally in TypeScript the resulting code will end up looking something like this:

export const withoutChain =
 ({ laxOperations, commands, localFunctions }: LaxCallbackDependencies) =>
 async (httpRequest: HttpRequest) => {
   const signatureOk = await laxOperations.checkSignature({ httpRequest });
   if (signatureOk.result.kind !== "success") {
     console.log("Error to internal logging service");
     return;
   }

   const laxContext = await laxOperations.parseRequest({
     laxSignatureCheck: signatureOk.result.value,
   });
   if (laxContext.result.kind !== "success") {
     console.log("Error to internal logging service");
     return;
   }

   const command = await commands.parseUntrustedCommand({
     untrustedCommand: laxContext.result.value.actionPayload,
   });
   if (command.result.kind !== "success") {
     console.log("Error to internal logging service");
     // Oh! We know how to contact the user now as well!
     await laxOperations.reply({
       laxContext: laxContext.result.value,
       reply: "A suitable error message" as any,
     });
     return;
   }

   // ... more code here ...
 };

This code isn't ideal. Admittedly, it has one big advantage: anyone who has written any TypeScript can immediately see how it works. But it also has a number of flaws. The most important one is that it is so verbose, and in such a repetitive way, that it is actually hard to follow the flow of what the code "wants" to do - the sequence of operations that will be carried out if everything works as expected. Trying to work out how a particular type of error will be handled, or even which types of errors might occur, is even harder.

What if we could separate out the straight forward intent of our code on the one hand, but also ensure we handle all of the possible failure states? What if we could compose together multiple operations which each, individually, know how they might fail and end up with a larger operation that still knows all the ways it can fail.

Business logic that looks like this:

export const processLaxCallback = ({
  laxOperations,
  commands,
  localFunctions,
}: LaxCallbackDependencies) =>
  SolidChain.start<HttpRequest, LaxCallBackState>()
    .chain("laxSignatureCheck", laxOperations.checkSignature)
    .chain("laxContext", laxOperations.parseRequest)
    .tap(({ laxContext }) => Solid.set("laxContext", laxContext))
    .chain("command", ({ laxContext }) =>
      commands.parseUntrustedCommand({
        untrustedCommand: laxContext.actionPayload,
      })
    )
    .chain("userInfo", laxOperations.findUserAndOrganization)
    .tap(({ userInfo }) => Solid.set("user", userInfo.user))
    .tap(({ userInfo }) => Solid.set("organization", userInfo.organization))
    .chain("eventsCaused", commands.executeCommand)
    .map("reply", localFunctions.createSuccessResponse)
    .tap(laxOperations.reply)
    .executeTarget("eventsCaused");

Once we've called our code, we can then exhaustively handle any errors it may have produced, safe in the knowledge that if any of the operations we're depending on add a new failure mode the compiler will force us to deal with them appropriately. This consolidates all of our error handling into an other clear and easy to read piece of code, generally looking something like the following:

switch (processResult.result.failure.type) {
  case "NotPermitted":
    await reportError("You can't do that");
    break;
  case "ValidationFailed":
    await reportError("You sent the wrong information");
    break;
  case "UnrecognizedCommand":
    await reportError(
      "Something is wrong with the message we sent you, sorry!"
    );
    break;
  case "OrganizationNotFound":
  case "UserNotFound":
    await reportError("You don't seem to be fully set up on Lax yet");
    break;
  case "UnableToParseLaxContext":
  case "InvalidLaxSignature":
    await reportError("Invalid callback");
    break;
  case "CouldNotContactLax":
    await reportError(
      "We tried to send you a message, but something went wrong at Lax's end."
    );
    break;
  case "LaxRefusedReply":
    await reportError(
      "We tried to send you a message but something went wrong on our end."
    );
    break;
  case "UnhandledException":
    await reportError(
      "Something went wrong, our support staff will look into it"
    );
    break;
  default:
    // this is just a function that takes and returns "never"
    // to enforce completeness
    return exhausted(processResult.result.failure);
}

Let's do it! We need to both build out our monad, and then also think about how to write the code that uses it.

The advantage of using a standard abstraction like a monad is that someone has done most of the hard thinking already for you. In TypeScript we don't have a nice way of defining code that works on "anything which is a monad", but in Haskell you can - and that means that there's collections of functions that you can refer to that are implemented on the basis of something being a monad. This means that if you have a working knowledge of Haskell (or Scala, or an other language where someone has done this work for you already) you can easily add features to your XChain class as you discover you need them.

For example, what do you do if you have an array of inputs that you want to map using a Solid returning function?

Well, if you check the link above you'll find the mapM function with the signature mapM :: (Traversable t, Monad m) => (a -> m b) -> t a -> m (t b). A Traversable is roughly an iterable in TypeScript speak, so we can check how mapM is implemented on a type that would be useful in TypeScript. It turns out it is an alias for the traverse function of whichever t is traversable, so we pick the list implementation as probably being close to what we'd want and it is:

instance Traversable [] where
  {-# INLINE traverse #-} -- so that traverse can fuse
  traverse f = List.foldr cons_f (pure [])
    where cons_f x ys = liftA2 (:) (f x) ys

Hmm. liftA2 is a bit weirdly named but it allows a binary operation to happen in our monadic context. We can in turn look up a default implementation of that in terms of what's available in a monad, and we end up with some new helper functions on monad.

export const Solid = {
  // All the existing monad operations like bind etc...
  map: <Previous, Next, Failure, State extends Scope<any>>(
    func: (previous: Previous) => Next,
    prev: Solid<Previous, Failure, State>
  ) => {
    return Solid.bind(prev, (success) => Solid.pure(func(success)));
  },
  apply: <Success, NextSuccess, Failure, NextFailure, State extends Scope<any>>(
    funcInMonad: Solid<(prev: Success) => NextSuccess, NextFailure, State>,
    prev: Solid<Success, Failure, State>
  ): Solid<NextSuccess, Failure | NextFailure, State> => {
    return Solid.bind(funcInMonad, (func) => Solid.map(func, prev));
  },
  lift2: <
    Left,
    Right,
    Result,
    LeftFailure,
    RightFailure,
    State extends Scope<any>
  >(
    operation: (left: Left, right: Right) => Result,
    left: Solid<Left, LeftFailure, State>,
    right: Solid<Right, RightFailure, State>
  ): Solid<Result, LeftFailure | RightFailure, State> => {
    return Solid.apply(
      Solid.map((left: Left) => (right: Right) => operation(left, right), left),
      right
    );
  },
  traverse: <Input, Success, Failure, State extends Scope<any>>(
    inputs: Input[],
    func: (input: Input) => Solid<Success, Failure, State>
  ): Solid<Success[], Failure, State> => {
    return inputs.reduce<Solid<Success[], Failure, State>>(
      (acc, next) =>
        Solid.lift2(
          (results: Success[], next: Success) => {
            results.push(next);
            return results;
          },
          acc,
          func(next)
        ),
      Solid.pure([])
    );
  },
};

Did translating that make my brain hurt a bit? Yes, but it made my brain hurt a lot less than working out that logic for myself. And now you are just a cut and paste away from being able to reuse this same code on any other monads you want to create, and in your solid chains you can right things like:

export const traverseExample =
  (commandOperations: Commands) => (untrustedInput: any[]) =>
    SolidChain.start<
      {
        untrustedInput: any[];
        userInfo: { user: User; organization: Organization };
      },
      {}
    >()
      .chain("parsedCommands", ({ untrustedInput }) =>
        Solid.traverse(untrustedInput, commandOperations.parseUntrustedCommand)
      )
      .chain("resultingEvents", ({ parsedCommands, userInfo }) =>
        Solid.traverse(parsedCommands, (command) =>
          commandOperations.executeCommand({ command, userInfo })
        )
      )
      .executeTarget("resultingEvents");

Warning: there's an argument for not adding too many of these helpers too quickly or to not make all of the intermediate abstractions publicly available. Developers new to functional programming will quickly see the point of a method like traverse when shown an example, but finding something like lift2 is going to leave a lot of people scratching their heads.