Deploy Service CURL - API Reference
MonsterAPI Deploy service CURL Usage Reference
Here in this guide, we discuss how to use curl commands to send a request to launch, manage, and use Monster Deploy service through /deploy
API path.
Example to Deploy a Llama-3.1-8B API endpoint on 24GB GPU
URL="https://api.monsterapi.ai/v1"
curl -X 'POST' \
"$URL/deploy/llm" \
--header 'accept: application/json' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer ${MONSTER_API_KEY}" \
--data '{
"basemodel_path": "meta-llama/Meta-Llama-3.1-8B-Instruct",
"prompt_template": "{prompt}{completion}",
"per_gpu_vram": 24,
"gpu_count": 2
}'
Launch a custom image deployment. This is suited to deploy any of your custom docker images from your docker registry.
curl -X 'POST' \
'https://api.monsterapi.ai/v1/deploy/custom_image' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"serving_params": {
"per_gpu_vram": "24",
"gpu_count": "1"
},
# For private Docker images, use image_registry
"image_registry": {
"registryName": "IMAGE_NAME",
"username": "YOUR_USER_NAME",
"password": "YOUR_DOCKER_PASSWORD"
},
"env_params": {
"API_KEY": "12345",
"MODE": "PRODUCTION"
},
"port_numbers": [
8000 # Use the port specified in your Docker image
]
}'
Get the status of your deployment
Once deployed /status
URL can be used to get status and receive the URL to the LLM service endpoint.
URL="https://api.monsterapi.ai/v1"
curl --request GET \
--url $URL/deploy/status/<deployment_id>\
--header 'accept: application/json' \
--header "Authorization: Bearer ${MONSTER_API_KEY}"
Progress of status responses:
-
In-Progress
{ "status":"pending", "message":"Instance is still being provisioned, please wait and try again." }
-
Fail
{ "status": "failed", "message": "Instance has failed, please launch a new instance." }
-
Building
{ "status": "building", "message": "Server has started but trying to connect to deployment container, just downloading your model and setting things up, please try again in few minutes; if state persists, please use /restart or /terminate!" }
-
Live
{ "status":"live", "message":"Server has started !!!", "URL":"https://jus.qblocks.cloud:58744", "api_auth_token":"57b7b903-a4b6-4720-8154-af71aa8e8313" }
Visit the URL to get the LLM service endpoint details or above url/docs to get swagger documents.
-
Terminated by User
{
"status":"terminatedByUser",
"message":"Instance is terminatedByUser"
}
- Terminated by System (Out of Credits)
{
"status":"terminatedBySystem",
"message":"Instance is terminatedBySystem"
}
Fetch Logs of deployment
Example curl request to fetch logs of 100 lines for a live deployment
URL="https://api.monsterapi.ai/v1"
curl --request GET \
--url $URL/deploy/logs/<deployment_id>?n_lines=100\
--header 'accept: application/json' \
--header "Authorization: Bearer ${MONSTER_API_KEY}"
Terminate a deployment using the deployment-id
#!/bin/bash
URL="https://api.monsterapi.ai/v1"
deployment_id="enter_your_deployment_id"
# Execute the curl command
curl -X 'POST' \
--url $URL/deploy/terminate \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
--header "Authorization: Bearer ${MONSTER_API_KEY}" \
-d '{
"deployment_id": "'"${deployment_id}"'"
}'
Get the list of deployments for a user
URL="https://api.monsterapi.ai/v1"
# Set service to a valid value or leave it empty
service="/deploy/llm"
# Possible values for service /deploy/llm - Deploy service -LLM /deploy/custom_image - Deploy service-Custom_Image
# Set status to a valid value or leave it empty
status="terminatedByUser"
# Possible values for status: pending, building, live, completed, terminatedByUser, teerminatedBySystem
# Initialize query parameters
query_params=""
# Check if status is set and add to query parameters
if [ -n "$status" ]; then
query_params="status=$status"
fi
# Check if service is set to a valid value and add to query parameters
if [ "$service" = "/deploy/llm" ] || [ "$service" = "/deploy/custom_image" ]; then
# Add an '&' if query_params already has status
if [ -n "$query_params" ]; then
query_params="${query_params}&"
fi
query_params="${query_params}deployment_service=$service"
fi
# Construct the final URL
final_url="${URL}/deploy/list"
if [ -n "$query_params" ]; then
final_url="${final_url}?${query_params}"
fi
curl --request GET \
--url "$final_url" \
--header 'accept: application/json' \
--header "Authorization: Bearer ${MONSTER_API_KEY}"
Get a List of available compute instances
URL="https://api.monsterapi.ai/v1"
curl --request GET \
--url $URL/deploy/instances\
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
--header "Authorization: Bearer ${MONSTER_API_KEY}" \
Here is a table showing the tentative list of actual compute instance availability that might change.
Updated about 2 months ago