Behind the Music: The Delphix node.js CLI

As I indicated in a previous post, the new Delphix CLI is a node.js application that runs locally when the use

As I indicated in a previous post, the new Delphix CLI is a node.js application that runs locally when the user logs into the system. User documentation can be found in the CLI User Guide, but I thought it would be interesting to explore how we ended up with the CLI we have today given the underlying web services upon which it is based. Our web services are all defined by a set of schemas that I hope to one day describe in more detail. For those of you with a delphix server on hand, you can access the full schema at /api/json/delphix.json on any server. These schemas are loaded by the CLI to dynamically generate the content such that we don't need to update the CLI with every change made to the web services layer. Our APIs are generally RESTful with CRUD semantics, though they have a few oddities that are worth noting:

  • We support singleton objects (i.e. NDMP Configuration) that export read/update on a global path.
  • Our objects are polymorphic and leverage inheritance, so that we can have a set of operations (link, provision, delete, etc) that operate on a variety of types (Oracle, MSSQL, etc) without needing an entirely separate namespace (which would prevent consumers from generically iterating over objects).
  • We support non-CRUD operations both per-object (start, stop, etc), and globally (link, provision, etc).
  • We have persistent references to objects that are independent of the object name.
  • Some operations execute asynchronously from the web service call and can take a while to run.

The CLI is very much inspired by Bryan's work on the Fishworks CLI (Fishworkers will undoubtedly take heart in the fact that the CLI docs still pay homage to dory and kiowa). We knew that a modal CLI based on filesystem-like navigation, property manipulation, and rich built-in help and tab completion would be vastly easier to use than a never-ending list of commands with esoteric --options-with-ridiculous-names. But we could also learn from the Fishworks CLI: because the underlying web service layer was not RESTful and well-formed, it meant that maintaining the CLI was quite expensive and semantics varied in subtle ways between contexts. The first decision was to have the CLI namespace mimic that of the web services layer. If an API was rooted at /resources/json/delphix/service/smtp, it meant that the CLI location would be service smtp. This encourages both rational layout of APIs, and eliminates the need for extra translation. We allow for these paths to be typed in directly, but also support the cd command and UNIX-like paths for those familiar with those shells:

    delphix> database
    delphix database> cd /service/ndmp
    delphix service ndmp> cd ..
    delphix service>

For groups of objects, we have a list command (as well as a ls alias that shows everything at the current context) that can display objects, and a select command to select an individual node:

    delphix database> list
    dory  -                -
    delphix database> select dory
    delphix database "dory"> cd /
    delphix> database "dory"
    delphix database "dory">

Now that we could move around the namespace, the next question was how to model operations: create, read, update, delete, and custom global and per-object operations. Each operation, regardless of HTTP implementation (PUT, POST, or DELETE) can take a single JSON object as input. Rather than specifying a complete object on the command line, we place the user into a context where they can interactively change properties and the elect to commit (or discard) the operation.

    delphix database> link
    delphix database link *> get container
        type: OracleDatabaseContainer
        name: (required)
        description: (unset)
        diagnoseNoLoggingFaults: true
        group: (required)
        masked: (unset)
        performanceMode: (unset)
    delphix database link *> set
    delphix database link *> set dbUser=delphix

The APIs support nested objects as input, so the CLI uses dot-delimited properties (and tab completion) for specifying input. For objects that support multiple types (through inheritance), the set of available properties changes whenever the type parameter is changed. For programmatic consumers, data can be output in JSON, and the 'trace' option can be turned on to see the HTTP calls made as part of an operation. Users can then experiment in the CLI, with a more natural interface, tab completion, integrated help, and then use the switch to the raw web services when more advanced manipulation is required. There are many more topics in the world of the CLI and web services, from objects references in names to asynchronous jobs, to integration with the illumos PAM stack to hook into SSH authentication. But for my next post I hope to dive into the structure of the web services schemas, and describe how we use them on the backend, GUI, documentation, and CLI to automate significant parts of the engineering process.