hsk8s (Hyperscience Kubernetes CLI)

  • Updated on Nov 27, 2024
  • Published on Sep 17, 2024
  • 5 minute(s) read

IMPORTANT: These docs are only relevant for v2.x.x+. hsk8s 1.x is no longer supported.

Background

hsk8s is a binary containing a set of utilities created by Hyperscience to help you install the application inside your Kubernetes cluster. Make sure you are using the latest version of hsk8s if possible.

Installation

Hyperscience Token

These commands require a Hyperscience token. Please reach out to your sales representative if you do not have one. Once you have it, export an environment variable with its value:

export HS_TOKEN=<your_token>

Make sure to replace <your_token> with the actual value.

Supported OS and Architectures

The following OS/architecture combinations are supported:

  • darwin-arm64

  • darwin-amd64

  • linux-arm

  • linux-amd64

Download the binary & install

You can download the binary from our asset-hosting partner with the following URL scheme. Replace  with the architecture where you are running it.

export HS_K8S_CLIENT_ARCH=
export HS_K8S_CLIENT_VERSION=latest

# Using your token in the url
curl -OL "https://dl.cloudsmith.io/${HS_TOKEN}/hyperscience/prod/raw/names/hsk8s-${HS_K8S_CLIENT_ARCH}/versions/${HS_K8S_CLIENT_VERSION}/hsk8s-${HS_K8S_CLIENT_ARCH}-${HS_K8S_CLIENT_VERSION}.tgz"

# Or using basic auth
curl -OL -u token:${HS_TOKEN} "https://dl.cloudsmith.io/basic/hyperscience/prod/raw/names/hsk8s-${HS_K8S_CLIENT_ARCH}/versions/${HS_K8S_CLIENT_VERSION}/hsk8s-${HS_K8S_CLIENT_ARCH}-${HS_K8S_CLIENT_VERSION}.tgz"

Once you've successfully downloaded the binary, you need to untar it, change its permissions and move it to a directory in your $PATH:

tar -xvf hsk8s-${HS_K8S_CLIENT_ARCH}-${HS_K8S_CLIENT_VERSION}.tgz
chmod 744 hsk8s-${HS_K8S_CLIENT_ARCH}
mv hsk8s-${HS_K8S_CLIENT_ARCH} /usr/local/bin/hsk8s

Test that you've installed the binary correctly with the following command:

hsk8s version

Usage

Downloading container Images

hsk8s can be used to bounce container images from Hyperscience repositories to your own. The application uses the Docker HTTP V2 Registry API to handle communication, so a local docker installation is NOT required. The application does require network access to our publicly-available repositories, as well as network access to your own internal container registry.

You will need 4 distinct repositories with the following default names:

  • forms

  • sdm_blocks

  • trainer

  • hyperoperator

These names can be overridden with the hsk8s parameters --forms-repo, --blocks-repo, --trainer-repo, and --operator-repo, respectively.

IMPORTANT: If you don't use the hsk8s tool to stream the images to your repositories, you will also need to stream a supporting image for the trainer database with the following tag: postgres...12.10-alpine. If you use hsk8s, the tool will also push this image for you when you stream the trainer image, so you don't have to push it separately.

Examples

IMPORTANT: All internal repositories must already exist before running the commands below.

Streaming Images to AWS ECR

This command will stream images from Hyperscience repositories to an ECR Registry of your choice. The user running this command will need to be logged in to the https://aws.amazon.com/cli/ under a profile that has the following IAM permissions:

  • ecr:BatchGetImage

  • ecr:BatchCheckLayerAvailability

  • ecr:CompleteLayerUpload

  • ecr:GetDownloadUrlForLayer

  • ecr:InitiateLayerUpload

  • ecr:PutImage

  • ecr:UploadLayerPart

To see all the options of the command, type:

hsk8s image stream --help

Now you can run the command:

hsk8s image stream 0123456789.dkr.ecr.us-east-1.amazonaws.com --token $HS_TOKEN --aws

Note: In the command above, replace "0123456789.dkr.ecr.us-east-1.amazonaws.com" with your ECR registry endpoint. Note that it doesn't contain the repository name. You can obtain the name through the AWS Console or CLI.

docker pull docker.cloudsmith.io/hyperscience/prod/postgres:12.10.0
docker tag docker.cloudsmith.io/hyperscience/prod/postgres:12.10.0 0123456789.dkr.ecr.us-east-1.amazonaws.com/trainer:postgres...12.10-alpine
docker push 0123456789.dkr.ecr.us-east-1.amazonaws.com/trainer:postgres...12.10-alpine

Streaming Images to Google Artifact Registry

To push to GCP's Artifact Registry, you must first configure its authentication as explained in Google's Store Docker container images in Artifact Registry.

Then, running the following command will push all images that Hyperscience needs to the my-repo repository. In contrast to other Docker registries, you will need to create only the top-level repository in GAR. The others will be created automatically when the first image is pushed.

hsk8s image stream us-east4-docker.pkg.dev/my-project-id/my-repo --token $HS_TOKEN

Streaming Only Forms to a Private Repository

Sample:

hsk8s image stream 0123456789.dkr.ecr.us-east-1.amazonaws.com --token $HS_TOKEN --forms --forms-repo hyperscience-forms --basic username:password --forms-version 40.0.3

This example demonstrates a few of the many options supported by the hk8s image stream subcommand:

  1. To connect to a private docker repository that supports basic authentication, you can use the --basic flag like so: --basic username:password

  2. To stream only the main Hyperscience image (forms), you can use the --forms flag. Likewise, the command also supports streaming only --blocks, --trainer and --operator. These flags can be combined to stream different combinations of components.

  3. To specify a target repository that's different from the default, you can use the --forms-repo, --blocks-repo, --trainer-repo or --operator-repo flags.

  4. To specify a specific application version to download (that is not latest), you can use --forms-version or --operator-version. Since the block versions must match the forms version, the --forms-version flag is used for all of those images.

  5. To stream trainer of another version, you can issue a command like: hsk8s image stream 0123456789.dkr.ecr.us-east-1.amazonaws.com  --trainer --forms-version  --token $HS_TOKEN --aws

Going back to the above command, this command will stream the forms:40.0.3 image to 0123456789.dkr.ecr.us-east-1.amazonaws.com/hyperscience-forms:40.0.3 using Basic Auth credentials.

Installing kubectl

Linux Instructions

MacOS Instructions

Now create a file named hs_env.bash with the following content:

export HS_K8S_NAMESPACE=hyperscience-dev
export HS_KUBECTL_CONTEXT=hyperscience-dev
export HS_HELM_RELEASE=hyperscience-dev
export HS_HELM_CHART=hyperscience/hyperscience

You can replace hyperscience-dev with whatever names you come up with. Just leave hyperscience/hyperscience as it is.

Import environment from the newly created file:

source hs_env.bash

Then, import the config for your EKS cluster:

aws eks update-kubeconfig --region  --name  --alias $HS_KUBECTL_CONTEXT
kubectl config use-context $HS_KUBECTL_CONTEXT
kubectl config set-context $HS_KUBECTL_CONTEXT --namespace=$HS_K8S_NAMESPACE

Alternatively, follow the instructions in Google's Install kubectl and configure cluster access for guidance on how to authenticate towards a GKE cluster in the Google Cloud Platform.

Set up the Helm repository

IMPORTANT: Setting up the repository requires a local installation of Helm, the package manager for Kubernetes. If you don't have the tool, you should follow Helm's Installing Helm instructions first.

Once you install Helm, run:

hsk8s helm add --token $HS_TOKEN

to add the Hyperscience Helm Repository to your local repository cache.

Now, you can run commands like helm repo update, which will fetch the latest version of the Hyperscience's package for Helm (called chart in Helm's language) from the remote repository.

Block asset management

hsk8s can be used to simplify the process of importing block assets that are necessary for some specific blocks such as the Llama block via the following commands:

  1. hsk8s asset list --token $HS_TOKEN

    • Lists assets available to download

  2. hsk8s asset download   --token $HS_TOKEN

    • Downloads an asset to a local directory

  3. hsk8s asset stream   --token $HS_TOKEN

    • Streams an asset directly from Cloudsmith to an S3 Bucket

    • Requires your default aws profile to be set

    • Your profile needs permission to write to your specified S3 URI

  4. hsk8s asset import

    • Imports assets into Hyperscience from an asset repository given a json manifest file

    • You need to ensure kubectl is installed and your kubeconfig, context, and namespace are set. See Installing kubectl

    • See below for more information

Import Llama Block Assets (beta)

Prerequisites

  • Hyperscience version >= 38.0.0

  • Hyperoperator version >= 5.3.0

  • An S3 Bucket Hyperscience can read from

  1. Set the following environment variables in values.yaml and apply changes:

    operator:
  env:
    S3_ARTIFACTORY_NAME: models
    S3_ARTIFACTORY_PATH: s3://my-s3-bucket/models
    ENABLE_ADD_S3_ARTIFACTORY: "true"

    • Change s3://my-s3-bucket/models to your S3 location

  2. Get list of available assets and choose artifact zip and import payload json:

    $ hsk8s asset list --token $HS_TOKEN
llama_13b_q8_0_artifacts.zip
llama_13b_q8_0_import_payload.json

  3. Stream artifact zip to your S3 Bucket:

    • You will need to agree to the EULA before stream starts

    • This may take a while depending on file size and your network conditions

      $ hsk8s asset stream llama_13b_q8_0_artifacts.zip s3://my-s3-bucket/models/llama_13b_q8_0_artifacts.zip --token $HS_TOKEN
Streaming llama_13b_q8_0_artifacts.zip to s3://my-s3-bucket/models/llama_13b_q8_0_artifacts.zip
13.79 GB / 13.79 GB [----------------------------------------------------------------------] 100.00% 102.77 MB p/s
Finished streaming llama_13b_q8_0_artifacts.zip to: https://my-s3-bucket.s3.us-east-1.amazonaws.com/models/llama_13b_q8_0_artifacts.zip

  4. Download import payload json file:

    • You will need to agree to the EULA before stream starts

      $ hsk8s asset download llama_13b_q8_0_import_payload.json ./ --token $HS_TOKEN
Downloading llama_13b_q8_0_import_payload.json
3.10 kB / 3.10 kB [---------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
Succesfully downloaded llama_13b_q8_0_import_payload.json

  5. Import model artifact asset using import payload json file

    • You need to ensure kubectl is installed and your kubeconfig, context, and namespace are set. See Installing kubectl.

      $ hsk8s asset import llama_13b_q8_0_import_payload.json
Loaded file: llama_13b_q8_0_import_payload.json
Current context: my-cluster
Namespace: hyperscience
Please confirm file, context, and namespace. Enter 'y' to continue with import
y
Writing data
Starting import
...
[2023-10-06 18:50:59.828 | INFO | 940-MainThread | artifacts.management.commands.artifacts_import:29] [trace_id=0 span_id=0 resource.service.name=hyperscience] OK: Imported 27, skipped 0