Deploy Service CURL - API Reference

Here in this guide, we discuss how to use curl commands to send a request to launch, manage, and use Monster Deploy service through /deploy API path.

Example to Deploy a Llama-3.1-8B API endpoint on 24GB GPU

URL="https://api.monsterapi.ai/v1"
curl -X 'POST' \
  "$URL/deploy/llm" \
  --header 'accept: application/json' \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer ${MONSTER_API_KEY}" \
  --data '{
    "basemodel_path": "meta-llama/Meta-Llama-3.1-8B-Instruct",
    "prompt_template": "{prompt}{completion}",
    "per_gpu_vram": 24,
    "gpu_count": 2
}'

Launch a custom image deployment. This is suited to deploy any of your custom docker images from your docker registry.

curl -X 'POST' \
  'https://api.monsterapi.ai/v1/deploy/custom_image' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "serving_params": {
    "per_gpu_vram": "24",
    "gpu_count": "1"
  },
# For private Docker images, use image_registry
  "image_registry": {
    "registryName": "IMAGE_NAME",
    "username": "YOUR_USER_NAME",
    "password": "YOUR_DOCKER_PASSWORD"
  },
  "env_params": {
    "API_KEY": "12345",
    "MODE": "PRODUCTION"
  },
  "port_numbers": [
    8000 # Use the port specified in your Docker image
    ]
}'

Get the status of your deployment

Once deployed /status URL can be used to get status and receive the URL to the LLM service endpoint.

URL="https://api.monsterapi.ai/v1"
curl --request GET \
     --url $URL/deploy/status/<deployment_id>\
     --header 'accept: application/json' \
     --header "Authorization: Bearer ${MONSTER_API_KEY}"

Progress of status responses:

In-Progress

{
    "status":"pending",
    "message":"Instance is still being provisioned, please wait and try again."
}

Fail

{
    "status": "failed",
    "message": "Instance has failed, please launch a new instance."
}

Building

{
    "status": "building",
    "message": "Server has started but trying to connect to deployment container, just downloading your model and setting things up, please try again in few minutes; if state persists, please use /restart or /terminate!"
}

Live

{
    "status":"live",
    "message":"Server has started !!!",
    "URL":"https://jus.qblocks.cloud:58744",
    "api_auth_token":"57b7b903-a4b6-4720-8154-af71aa8e8313"
}

Visit the URL to get the LLM service endpoint details or above url/docs to get swagger documents.

Terminated by User

{
    "status":"terminatedByUser",
    "message":"Instance is terminatedByUser"
}

Terminated by System (Out of Credits)

{
    "status":"terminatedBySystem",
    "message":"Instance is terminatedBySystem"
}

Fetch Logs of deployment

Example curl request to fetch logs of 100 lines for a live deployment

URL="https://api.monsterapi.ai/v1"
curl --request GET \
     --url $URL/deploy/logs/<deployment_id>?n_lines=100\
     --header 'accept: application/json' \
     --header "Authorization: Bearer ${MONSTER_API_KEY}"

Terminate a deployment using the deployment-id

#!/bin/bash
URL="https://api.monsterapi.ai/v1"
deployment_id="enter_your_deployment_id"

# Execute the curl command
curl -X 'POST' \
  --url $URL/deploy/terminate \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  --header "Authorization: Bearer ${MONSTER_API_KEY}" \
  -d '{
  "deployment_id": "'"${deployment_id}"'"
}'

Get the list of deployments for a user

URL="https://api.monsterapi.ai/v1"

# Set service to a valid value or leave it empty
service="/deploy/llm" 

# Possible values for service /deploy/llm - Deploy service -LLM /deploy/custom_image - Deploy service-Custom_Image
# Set status to a valid value or leave it empty
status="terminatedByUser"

# Possible values for status: pending, building, live, completed, terminatedByUser, teerminatedBySystem

# Initialize query parameters
query_params=""

# Check if status is set and add to query parameters
if [ -n "$status" ]; then
  query_params="status=$status"
fi

# Check if service is set to a valid value and add to query parameters
if [ "$service" = "/deploy/llm" ] || [ "$service" = "/deploy/custom_image" ]; then
  # Add an '&' if query_params already has status
  if [ -n "$query_params" ]; then
    query_params="${query_params}&"
  fi
  query_params="${query_params}deployment_service=$service"
fi

# Construct the final URL
final_url="${URL}/deploy/list"
if [ -n "$query_params" ]; then
  final_url="${final_url}?${query_params}"
fi

curl --request GET \
     --url "$final_url" \
     --header 'accept: application/json' \
     --header "Authorization: Bearer ${MONSTER_API_KEY}"

Get a List of available compute instances

URL="https://api.monsterapi.ai/v1"
curl --request GET \
    --url $URL/deploy/instances\
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    --header "Authorization: Bearer ${MONSTER_API_KEY}" \

Here is a table showing the tentative list of actual compute instance availability that might change.