Layered Deployment Example

A deployment settings file can list more than one fragment. Fragments are combined in order, and each fragment can use keyPaths to take only part of its file. This allows a hub configuration to be assembled from several sources, for example models that are shared across sites and maintained in Git, combined with connections that are specific to each site.

This example:

  • Pulls the models and instances from a deployment file in Git
  • Combines them with the connections, inputs, and outputs from a deployment file stored on the hub host
  • Starts the hub with the combined configuration

1. Prepare the deployment files

The model deployment file, model-deployment.json, is committed to the root of the models branch of the repository.

The connection deployment file, connection-deployment.json, is exported from a hub configured with the site’s connections, and saved to /usr/local/highbyte/deployments/connection-deployment.json on the hub host:

java -jar intelligencehub-runtime.jar export /usr/local/highbyte/deployments/connection-deployment.json

When no path is given, export writes intelligencehub-deployment.json to the working directory.

2. Define the fragments

The deployment settings file below defines two fragments:

  • Fragment 1: A git fragment that uses the repository named highbyte, reads model-deployment.json from the models branch, and selects only .project.modeling.
  • Fragment 2: A dir fragment that reads /usr/local/highbyte/deployments/connection-deployment.json and selects only .project.connections, .project.inputs, and .project.outputs. Because patterns is not set, rootPath is the full path to a single file.
{
  "version" : 1,
  "repos" : [ {
    "type" : "git",
    "uri" : "<uri>",
    "name" : "highbyte",
    "author" : "",
    "email" : "",
    "auth" : {
      "type" : "pass",
      "password" : "<token>",
      "username" : "<user>"
    }
  } ],
  "fragments" : [ {
    "details" : {
      "type" : "git",
      "patterns" : [ "model-deployment.json" ],
      "repoName" : "highbyte",
      "ref" : "models"
    },
    "keyPaths" : [ ".project.modeling" ]
  }, {
    "details" : {
      "type" : "dir",
      "rootPath" : "/usr/local/highbyte/deployments/connection-deployment.json"
    },
    "keyPaths" : [ ".project.connections", ".project.inputs", ".project.outputs" ]
  } ]
}

3. Understand how the fragments are combined

The hub starts from an empty configuration and applies each fragment in the order listed:

  • A key path is written with dots, such as .project.modeling, and selects that element of the fragment’s file. The selected element is placed at the same location in the combined configuration. If a key path does not exist in the file, the hub does not start.
  • A fragment with no keyPaths contributes its entire file.
  • When a pattern matches more than one file, each matched file is applied in turn.
  • When a later fragment selects an element that an earlier fragment already provided, the later content is merged into the earlier content rather than replacing it.

The combined configuration then replaces the matching parts of the hub configuration:

  • Top-level sections that no fragment provides, such as accessControl or system, keep the values already on the hub.
  • The project and network sections are stored together in the hub’s configuration file and are replaced together. When a fragment provides any part of either, the parts that no fragment provides are cleared. In this example, the hub ends up with only the models, instances, connections, inputs, and outputs from the two fragments. Its pipelines and other project elements, as well as its network settings, are removed. Add key paths such as .project.pipelines or .network to a fragment to keep them.

4. Start the hub

Set HIGHBYTE_DEPLOYMENT_FILE to the path of the deployment settings file and start the hub, as described in Start from a Git Deployment for Windows, Linux, or Docker. For Docker, also mount /usr/local/highbyte/deployments into the container so the dir fragment can read the connection deployment file.

An example of the resulting hub configuration is shown in combined-deployment.json.