Deploy Service - Python Client Usage

Install our Client

python3 -m pip install monsterapi==1.0.2b3

Initiate Client

from monsterapi import client

client = client(api_key = "YOUR_API_KEY")

Lets deploy a LLama-7b with on Instance 24GB RTX A5000

launch_payload = {
    "basemodel_path": "meta-llama/Llama-2-7b-chat",
    "prompt_template": "{prompt}{completion}",
    "per_gpu_vram": 24,
    "gpu_count": 1
}

# Launch a deployment
ret = client.deploy("llm", launch_payload) 
deployment_id = ret.get("deployment_id")
print(deployment_id)

Let's deploy a custom docker image on an instance with 8GB Quadro RTX4000

launch_payload = {
  "serving_params": {
    "per_gpu_vram": 16,
    "gpu_count": 1
  },
  "image_registry": {
    "registryName": "rcv1k4s/nvidiadockertest:11.8",
    "username": "rcv1k4s",
    "password": "dckr_pat_VHrqIT07WXF1ILrObKDepaJwmvE"
  },
  "env_params": {
    "API_KEY": "12345",
    "MODE": "PRODUCTION"
  },
  "port_numbers": [
    8000
  ]
}

# Launch a deployment
ret = client.deploy("custom_image", launch_payload) 
deployment_id = ret.get("deployment_id")
print(ret)

Get the Status of Deployment

status_ret = client.get_deployment_status(deployment_id)
print(status_ret)

Get the Logs of Deployment

logs_ret = client.get_deployment_logs(deployment_id)
print(logs_ret)

Terminate the Deployment

terminate_return = client.terminate_deployment(deployment_id)
print(terminate_return)