Deploy Service CURL - API Reference

Here in this guide, we discuss how to use curl commands to send a request to launch, manage, and use Monster Deploy service through /deploy API path.

Example to Deploy a llama-2-13b API endpoint on 24GB GPU

URL="https://api.monsterapi.ai/v1"
curl -X 'POST' \
  "$URL/deploy/llm" \
  --header 'accept: application/json' \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer ${MONSTER_API_KEY}" \
  --data '{
    "basemodel_path": "meta-llama/Llama-2-13b-hf",
    "prompt_template": "{prompt}{completion}",
    "api_auth_token": "57b7b903-a4b6-4720-8154-af71aa8e8313",
    "per_gpu_vram": 24,
    "gpu_count": 2
}'

Launch a custom image deployment. This is suited to deploy any of your custom docker images from your docker registry.

curl -X 'POST' \
  'https://api.monsterapi.ai/v1/deploy/custom_image' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "serving_params": {
    "per_gpu_vram": "24",
    "gpu_count": "1"
  },
  "image_registry": {
    "registryName": "rcv1k4s/nvidiadockertest:11.8",
    "username": "rcv1k4s",
    "password": "dckr_pat_VHrqIT07WXF1ILrObKDepaJwmvE"
  },
  "env_params": {
    "API_KEY": "12345",
    "MODE": "PRODUCTION"
  },
  "port_numbers": [
    8000
  ]
}'

Get the status of your deployment

Once deployed /status URL can be used to get status and receive the URL to the LLM service endpoint.

URL="https://api.monsterapi.ai/v1"
curl --request GET \
     --url $URL/deploy/status/<deployment_id>\
     --header 'accept: application/json' \
     --header "Authorization: Bearer ${MONSTER_API_KEY}"

Progress of status responses:

In-Progress

{
    "status":"pending",
    "message":"Instance is still being provisioned, please wait and try again"
}

Fail

{
    "status": "failed",
    "message": "Instance has failed, please launch a new instance"
}

Building

{
    "status": "building",
    "message": "Server has started but trying to connect to deployment container, just downloading your model and setting things up, please try again in few mins, if state persists, please use /restart or /terminate!"
}

Live

{
    "status":"live",
    "message":"Server has started !!!",
    "URL":"https://jus.qblocks.cloud:58744",
    "api_auth_token":"57b7b903-a4b6-4720-8154-af71aa8e8313"
}
visit the url to get the llm service endpoint details or above url /docs to get swagger docs

Terminated by User

{
    "status":"terminatedByUser",
    "message":"Instance is terminatedByUser"
}

Terminated by System (Out of Credits)

{
    "status":"terminatedBySystem",
    "message":"Instance is terminatedBySystem"
}

Fetch Logs of deployment

Example curl request to fetch logs of 100 lines for a live deployment

URL="https://api.monsterapi.ai/v1"
curl --request GET \
     --url $URL/deploy/logs/<deployment_id>?n_lines=100\
     --header 'accept: application/json' \
     --header "Authorization: Bearer ${MONSTER_API_KEY}"

Terminate a deployment using the deployment-id

#!/bin/bash
URL="https://api.monsterapi.ai/v1"
deployment_id="enter_your_deployment_id"

# Execute the curl command
curl -X 'POST' \
  --url $URL/deploy/terminate \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  --header "Authorization: Bearer ${MONSTER_API_KEY}" \
  -d '{
  "deployment_id": "'"${deployment_id}"'"
}'

Get the list of deployments for a user

URL="https://api.monsterapi.ai/v1"
# Set service to a valid value or leave it empty
service="/deploy/llm" 
# Possible values for service /deploy/llm - Deploy service -LLM /deploy/custom_image - Deploy service-Custom_Image
# Set status to a valid value or leave it empty
status="terminatedByUser"
# Possible values for status: pending, building, live, completed, terminatedByUser, teerminatedBySystem

# Initialize query parameters
query_params=""

# Check if status is set and add to query parameters
if [ -n "$status" ]; then
  query_params="status=$status"
fi

# Check if service is set to a valid value and add to query parameters
if [ "$service" = "/deploy/llm" ] || [ "$service" = "/deploy/custom_image" ]; then
  # Add an '&' if query_params already has status
  if [ -n "$query_params" ]; then
    query_params="${query_params}&"
  fi
  query_params="${query_params}deployment_service=$service"
fi

# Construct the final URL
final_url="${URL}/deploy/list"
if [ -n "$query_params" ]; then
  final_url="${final_url}?${query_params}"
fi

curl --request GET \
     --url "$final_url" \
     --header 'accept: application/json' \
     --header "Authorization: Bearer ${MONSTER_API_KEY}"

Get a List of available compute instances

URL="https://api.monsterapi.ai/v1"
curl --request GET \
    --url $URL/deploy/instances\
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    --header "Authorization: Bearer ${MONSTER_API_KEY}" \

Here is a table showing the tentative list of actual compute instance availability that might change.