Holochain package management

1 The Problem

• Code re-use is important
• Deterministic code re-use is hard: verifiably deterministic code re-use is harder
• Code must be verifiably deterministic for the DNA of dApps
• UX and DX (developer experience) are both critical for network effects

1.1 Code re-use is important

As app builders, developers need tools and packages. Software architecture has always been plagued by inconsitency, dependency conflicts, and build time errors.

1.2 Deterministic code re-use is hard

Devops engineers learned years ago that idempotency (re-running the build and deployment multiple times results in the same output) and determinism were keys to acheiving success.

1.3 Verifiably deterministic code re-use is even harder

The core of Holochain has an essential improvement of a DHT, and this can be only realized through distinctive functions which define the DNA of a given app. It is essential that these functions be verifiably identical on different nodes which are running the same DNA.

1.4 Code must be verifiably deterministic for the DNA of dApps

In order for the fuctions to be verifiably identical, the package management must be verifiably identical as well.

2 Code Re-use

2.1 The opportunity of software libraries

In order to quickly bootstrap an eosystem of quality software, it will be necessary to build on top of excellent libraries

• Cryptography: libuecc, pycryptodome, go-crypto, libp2p-crypto, Crypto++, libsodium, botan, Bouncy Castle, cryptlib, libgcrypt, softhsm, pycrypt, bcryp, scrypt, sops, tcpcrypt, perl-Crypt, perl-Math, perl-CryptX, olm, gnuradio-nacl, libmcrypt, nettle, ccrypt, cryptol,
• Data manipulation:
• API libraries:
• Dependancy management can be a nightmare

2.2 The opportunity of verifiably identical software environments

Nix makes sure that each software environment can be custom tailored to the app, while re-using all the supporting libraries which are shared with other apps on the system. The libraries are stored as derivations in the nix store

2.3 The limitation of using only runtime interpreters

Currently, we are building a software ecosystem from runtime environments implimented in Go (Otto and

3 What is nix?

Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. It provides atomic upgrades and rollbacks, side-by-side installation of multiple versions of a package, multi-user package management and easy setup of build environments. It is a purely functional language. Here is an example nix expression:

{ stdenv, fetchurl, perl }:

stdenv.mkDerivation {
  name = "hello-2.1.1";
  builder = ./builder.sh;
  src = fetchurl {
    url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
    sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
  };
  inherit perl;
}

3.1 Nixpkgs

Nixpkgs is a huge collection of nix expressions, giving developers access to just about every library in every language.

3.2 NixOps

A tool for provisioning network and compute resources declaratively using the functional nix language. Current VM support includes nix-shell, NixOS containers, Amazon containers, Goolge Cloud Compute and exporting to Dockerfile. If we defined Holochain DNA using nix expressions and added holochain as a NixOps target, it would be easy to create apps which could deploy across all these platforms seamlessly.

3.3 Nixos

NixOS is an operating system which is built entirely from the nix expressions found in nixpkgs.

4 Functional vs non-functional

4.1 Non-functional package management

• npm not designed to be deterministic
• gx-go
• CPAN
• dpkg
• rpm
• pip
• Ruby gems, rvm, rbenv etc

The problem with these tools is they can't guarantee the contents of a zome because they are not deterministic: the order in which things are declared, for instance, can create a difference in the outputs of these package managers.

4.2 Non-functional configuration management

• puppet
• chef

4.3 Functional package management

Each nix expression

5 Deterministic vs non-deterministic

What do we mean by a deterministic package manager? "Determinism is the philosophical theory that all events, including moral choices, are completely determined by previously existing causes."

• yarn Yarn Determinism
• npm (after version 5.0)
• nixpkgs (actually does determinism correctly)

6 Functional Solutions

Nix (and it's FSF offshoot, Guix) is the only purely functional solution in this space

7 Implimentation

7.1 Starting from the DNA

Currently we are defining the functions of a zome using a JSON/YAML file which the points to javascript functions.

• Write the actual functions in nix: ```