When you throw some Immer into MongoDB

9/8/2021

Once I needed to use Immer to do a specific task at work and liked it a lot! Since then, I install Immer in every front-end project that needs state management around objects.

One day at the shower, I had an idea: why don't use Immer with MongoDB to offer document versioning?

The rest of this text is just a simple example where I explain the whys and hows.

Before proceeding, consider the code below.

./index.js
const { MongoClient } = require("mongodb");
const client = new MongoClient("mongodb://localhost:27017/immer_mongo");

const coll = "users";
const data = {
  firstName: "John",
  lastName: "Doe",
};

async function main() {
  await client.connect();

  const { insertedId } = await client.db().collection(coll).insertOne(data);
  const insertedData = await client
    .db()
    .collection(coll)
    .findOne({ _id: insertedId });

  console.log(insertedData);
}

main()
  .then(() => {
    process.exit(0);
  })
  .catch((err) => {
    throw err;
  });

The main function acts as a wrapper to use async/await. The real interesting part is the body of main function, where the object data is stored in the database, then read using the document id.

The last line of main function will log the inserted data. If you run this program using node, this should be de following result:

$ node .\index.js
{
  "_id": "613818fd6ad2eb8d4ab48207",
  "firstName": "John",
  "lastName": "Doe"
}

Adding Immer

Immer is a very nice JavaScript library that uses Proxies to wrap an object, record changes into this object, then produce a new object using a recipe function.

I'll not write an introduction to Immer, but you need to know what the code below is doing.

import produce from "immer"
const result = produce({ data: 1 }, draft => {
  draft.data = 2
})

If you don't understand what the code above is doing, you will not understand the rest of this text. In this case, I'd recommend reading immer docs.

Ok, I'll go ahead and install Immer as a dependency and import it at the beginning of the file.

./index.js
const { MongoClient } = require("mongodb");
const { produceWithPatches, enablePatches } = require("immer");

enablePatches();

Yeah, I'm already importing the function produceWithPatches and enabling patch generation. This feature allows us to generate an array of patches to rebuild the object and an array of patches that makes the reverse.

What about the cost? There is a runtime cost, but remember that this is an experiment. We can handle the cost.

Next, I'll jump a lot of refactoring steps and organize the initial code and create something called useInsertWithPatches.

const client = new MongoClient("mongodb://localhost:27017/immer_mongo");

const useInsertWithPatches = (coll, patchesColl) => ({
  async insert(data) {
    const [_, patches, reversePatches] = produceWithPatches({}, (draft) => {
      Object.assign(draft, data);
    });

    await client.connect();
    const dataInsertionResult = await client
      .db()
      .collection(coll)
      .insertOne(data);

    const patchesPayload = {
      documentId: dataInsertionResult.insertedId,
      createdAt: new Date(),
      changes: [
        {
          patches,
          reversePatches,
          createdAt: new Date(),
          reason: "created",
        },
      ],
    };

    await client.db().collection(patchesColl).insertOne(patchesPayload);

    const insertedData = await client
      .db()
      .collection(coll)
      .findOne({ _id: dataInsertionResult.insertedId });

    return { data: insertedData, patches, reversePatches };
  },
});

Do you like mixins?

The code above defines something that I like to call a behavior. Why? Because Elixir and Phoenix have a lot of use SomeModule clauses. I like the mental model.

React has React Hooks that are closely related to this mental model. You can use a state behavior that will bring state capabilities into your components. I like this mental model, this is the reason that I called useInsertWithPatches.

This could sound strange, but let's proceed because I'm sure that you will understand!

This function exposes an object that contains the method insert. It is a lot easier to reason about this using a usage case. See the example below.

const inserter = useInsertWithPatches("collection", "collection_patches")
const {data} = await inserter.insert({ hello: "world" })

The prefix use makes sense when you use mixins to combine behaviors, as the User object in the next example.

const User = {
  ...useInsertWithPatches("users", "users_patches"),
  name(user) {
    return `${user.firstName} ${user.lastName}`
  }
}

const newUser = await User.insert({ firstName: "John", lastName: "Doe" })

It is all about the API.

I'll proceed and create the useUpdateWithPatches. This function will return an object with the update method. This method will receive a record, an Immer recipe function, and an optional reason for that change, that is a string.

const client = new MongoClient("mongodb://localhost:27017/immer_mongo");

const useUpdateWithPatches = (coll, patchesColl) => ({
  async update(doc, recipe, reason = "") {
    const [data, patches, reversePatches] = produceWithPatches(doc, recipe);
    await client
      .db()
      .collection(coll)
      .updateOne({ _id: doc._id }, { $set: data });

    const change = {
      patches,
      reversePatches,
      createdAt: new Date(),
      reason,
    };

    await client
      .db()
      .collection(patchesColl)
      .updateOne({ documentId: doc._id }, { $push: { changes: change } });

    return { data, patches, reversePatches };
  },
});

Again, it is all about the API. How can I use this?

const updater = useUpdateWithPatches("collection", "collection_patches")
const {data} = await updater.update(someRecordData, draft => {
  draft.hello = "world"
})

Using the same "user" example as viewed before:

const User = {
  ...useInsertWithPatches("users", "users_patches"),
  ...useUpdateWithPatches("users", "users_patches"),

  name(user) {
    return `${user.firstName} ${user.lastName}`
  }
}

const newUser = await User.insert({ firstName: "John", lastName: "Doe" })
const updatedUser = await User.update(newUser.data, draft => {
  draft.firstName = "Rebeca",
  draft.lastName = "Dogger"
})

No this, no classes, only mixings and object composition using closures πŸ’ͺ.

Ok, and what about the patches? We saved their in the auxiliar collection *_patches, now we can rebuild our objects.

I'll start creating the useDocumentHistory behavior that will expose the method historyOf. This method will receive a document and use this document patches to rebuild all versions of the document.

const useDocumentHistory = (patchesColl) => ({
  async historyOf(doc) {
    const documentPatches = await client
      .db()
      .collection(patchesColl)
      .findOne({ documentId: doc._id });

    const versions = [];
    let lastVersion = {};
    for (const changes of documentPatches.changes) {
      lastVersion = applyPatches(lastVersion, changes.patches);
      versions.push(lastVersion);
    }

    return versions;
  },
});

The method historyOf will find the document patches, then iterate over each change and apply those patches using Immer's function applyPatches.

Be informed that this is an experiment. The code quality isn't the main goal here, so please, don't use these ideas and put this code to run in production.

Let use the "user" example again:

const User = {
  ...useInsertWithPatches("users", "users_patches"),
  ...useUpdateWithPatches("users", "users_patches"),
  ...useDocumentHistory("users_patches"),

  name(user) {
    return `${user.firstName} ${user.lastName}`
  }
}

const newUser = await User.insert({
  firstName: "John",
  lastName: "Doe",
});
let updatedUser = await User.update(newUser.data, (draft) => {
  draft.firstName = "Rebeca";
  draft.lastName = "Dogger";
});
updatedUser = await User.update(updatedUser.data, (draft) => {
  draft.firstName = "John";
  draft.lastName = "Dogger";
});

const hist = await User.historyOf(newUser.data);
// [
//  { firstName: 'John', lastName: 'Doe' },
//  { firstName: 'Rebeca', lastName: 'Dogger' },
//  { firstName: 'John', lastName: 'Dogger' }
// ]

That's cool, right? πŸ”₯πŸ”₯πŸ”₯πŸ”₯

Using this approach, we can log every change into the User document and rebuild the User at any given time. Does someone do something wrong? Easy, roll back to the last version. Need to look at the document history to find something? Easy, build the entire version history.

Resources

Full example file

./index.js
const { MongoClient } = require("mongodb");
const { produceWithPatches, applyPatches, enablePatches } = require("immer");

enablePatches();
const client = new MongoClient("mongodb://localhost:27017/immer_mongo");

const useInsertWithPatches = (coll, patchesColl) => ({
  async insert(data) {
    const [_, patches, reversePatches] = produceWithPatches({}, (draft) => {
      Object.assign(draft, data);
    });

    await client.connect();
    const dataInsertionResult = await client
      .db()
      .collection(coll)
      .insertOne(data);

    const patchesPayload = {
      documentId: dataInsertionResult.insertedId,
      createdAt: new Date(),
      changes: [
        {
          patches,
          reversePatches,
          createdAt: new Date(),
          reason: "created",
        },
      ],
    };

    await client.db().collection(patchesColl).insertOne(patchesPayload);

    const insertedData = await client
      .db()
      .collection(coll)
      .findOne({ _id: dataInsertionResult.insertedId });

    return { data: insertedData, patches, reversePatches };
  },
});

const useUpdateWithPatches = (coll, patchesColl) => ({
  async update(doc, recipe, reason = "") {
    const [data, patches, reversePatches] = produceWithPatches(doc, recipe);
    await client
      .db()
      .collection(coll)
      .updateOne({ _id: doc._id }, { $set: data });

    const change = {
      patches,
      reversePatches,
      createdAt: new Date(),
      reason,
    };

    await client
      .db()
      .collection(patchesColl)
      .updateOne({ documentId: doc._id }, { $push: { changes: change } });

    return { data, patches, reversePatches };
  },
});

const useDocumentHistory = (patchesColl) => ({
  async historyOf(doc) {
    const documentPatches = await client
      .db()
      .collection(patchesColl)
      .findOne({ documentId: doc._id });

    const versions = [];
    let lastVersion = {};
    for (const changes of documentPatches.changes) {
      lastVersion = applyPatches(lastVersion, changes.patches);
      versions.push(lastVersion);
    }

    return versions;
  },
});

async function main() {
  await client.connect();

  await client.db().collection("users").deleteMany({});
  await client.db().collection("users_patches").deleteMany({});

  const User = {
    ...useInsertWithPatches("users", "users_patches"),
    ...useUpdateWithPatches("users", "users_patches"),
    ...useDocumentHistory("users_patches"),

    name(user) {
      return `${user.firstName} ${user.lastName}`;
    },
  };

  const newUser = await User.insert({
    firstName: "John",
    lastName: "Doe",
  });
  let updatedUser = await User.update(newUser.data, (draft) => {
    draft.firstName = "Rebeca";
    draft.lastName = "Dogger";
  });
  updatedUser = await User.update(updatedUser.data, (draft) => {
    draft.firstName = "John";
    draft.lastName = "Dogger";
  });

  const hist = await User.historyOf(newUser.data);
  console.log(hist);
}

main()
  .then(() => {
    process.exit(0);
  })
  .catch((err) => {
    console.error(err);
    process.exit(1);
  });
πŸ‘ˆ All blog postsπŸ“ Edit this page

🀟

Have a gorgeous day