Status API: metrics endpoint
Puppet Server is capable of tracking advanced metrics to give you additional insight into its performance and health.
The HTTPS metrics endpoints are available on port 8140 of the primary server:
uri="https://$(puppet config print server):8140/status/v1/services"
curl --insecure "$uri"
See Usage notes for curl examples for information about forming curl commands.
level=debug
URL parameter, and the
structure of the returned data might change in future versions.These metrics fall into three categories:
- JRuby metrics (
/status/v1/services/pe-jruby-metrics
) - HTTP route metrics (
/status/v1/services/pe-master
) - Catalog compilation profiler metrics (
/status/v1/services/pe-puppet-profiler
)
All of these metrics reflect data for the lifetime of the current Puppet Server process and reset whenever the service is restarted. Any time-related metrics report milliseconds unless otherwise noted.
Like the standard status endpoints, the metrics endpoints return
machine-consumable information about running services. This JSON response includes the
same keys returned by a standard status endpoint request (see JSON endpoints). Each
endpoint also returns additional keys in an experimental
section.
GET /status/v1/services/pe-jruby-metrics
The /status/v1/services/pe-jruby-metrics
endpoint returns JSON containing
information about the JRuby pools from which Puppet Server fulfills agent requests.
You must query it at port 8140 and append the level=debug
URL
parameter.
Query parameters
No parameters are supported. Defaults to using the critical
status level.
Response codes
- 200 if and only if all services
report a status of
running
- 503 if any service’s status
i
unknown
orerror
Response keys
The metrics
are returned in two subsections of the experimental
section: jruby-pool-lock-status
and metrics
.
The response's experimental/jruby-pool-lock-status
section
contains the following keys:
Key | Definition |
---|---|
current-state
|
The state of the JRuby pool
lock, which should be either :not-in-use (unlocked), :requested (waiting for lock), or :acquired
(locked). |
last-change-time
|
The date and time of the
last current-state
update, formatted as an ISO 8601 combined date and time in
UTC. |
The response's experimental/metrics
section contains the following keys:
Key | Definition |
---|---|
average-borrow-time
|
The average amount of time a JRuby
instance spends handling requests, calculated by dividing the total
duration in milliseconds of the borrowed-instances value by the borrow-count value.
|
average-free-jrubies
|
The average number of JRuby instances that are not in use over the Puppet Server process’s lifetime. |
average-lock-held-time
|
The average time the JRuby pool held a
lock, starting when the value of jruby-pool-lock-status/current-state
changed to :acquired . This time mostly represents file sync
syncing code into the live codedir, and is calculated by dividing
the total length of time that Puppet Server held the lock by the
value of num-pool-locks . |
average-lock-wait-time
|
The average time Puppet Server spent
waiting to lock the JRuby pool, starting when the value of jruby-pool-lock-status/current-state changed to:requested ). This time
mostly represents how long Puppet Server takes to fulfill agent
requests, and is calculated by dividing the total length of time
that Puppet Server waits for locks by the value of num-pool-locks . |
average-requested-jrubies
|
The average number of requests waiting on an available JRuby instance over the Puppet Server process’s lifetime. |
average-wait-time
|
The average time Puppet Server spends waiting to reserve an instance from the JRuby pool, calculated by dividing the total duration in milliseconds of requested-instances by the requested-count value. |
borrow-count
|
The total number of JRuby instances that have been used. |
borrow-retry-count
|
The total number of times that a borrow attempt failed and was retried, such as when the JRuby pool is flushed while a borrow attempt is pending. |
borrow-timeout-count
|
The number of requests that were not served because they timed out while waiting for a JRuby instance. |
borrowed-instances
|
A list of the JRuby instances currently in
use, each reporting:
|
num-free-jrubies
|
The number of JRuby instances in the pool that are ready to be used. |
num-jrubies
|
The total number of JRuby instances. |
num-pool-locks
|
The total number of times the JRuby pools have been locked. |
requested-count
|
The number of JRuby instances borrowed, waiting, or that have timed out. |
requested-instances
|
A list of the requests waiting to be
served, each reporting:
|
return-count
|
The total number of JRuby instances that have been used. |
For example:
"pe-jruby-metrics": {
"detail_level": "debug",
"service_status_version": 1,
"service_version": "2.2.22",
"state": "running",
"status": {
"experimental": {
"jruby-pool-lock-status": {
"current-state": ":not-in-use",
"last-change-time": "2015-12-03T18:59:12.157Z"
},
"metrics": {
"average-borrow-time": 292,
"average-free-jrubies": 0.4716243097301104,
"average-lock-held-time": 1451,
"average-lock-wait-time": 0,
"average-requested-jrubies": 0.21324752542875958,
"average-wait-time": 156,
"borrow-count": 639,
"borrow-retry-count": 0,
"borrow-timeout-count": 0,
"borrowed-instances": [
{
"duration-millis": 3972,
"reason": {
"request": {
"request-method": "post",
"route-id": "puppet-v3-catalog-/*/",
"uri": "/puppet/v3/catalog/hostname.example.com"
}
},
"time": 1448478371406
}
],
"num-free-jrubies": 0,
"num-jrubies": 1,
"num-pool-locks": 2849,
"requested-count": 640,
"requested-instances": [
{
"duration-millis": 3663,
"reason": {
"request": {
"request-method": "put",
"route-id": "puppet-v3-report-/*/",
"uri": "/puppet/v3/report/hostname.example.com"
}
},
"time": 1448478371715
}
],
"return-count": 638
}
}
}
}
GET /status/v1/services/pe-master
The /status/v1/services/pe-master
endpoint returns JSON containing information
about the routes that agents use to connect to this server.
You must query it at port 8140 and append the level=debug
URL
parameter.
Query parameters
No parameters are supported. Defaults to using the critical
status level.
Response codes
- 200 if and only if all services
report a status of
running
- 503 if any service’s status
i
unknown
orerror
Response keys
The response's experimental/http-metrics
section contains a list of
routes, each containing the following keys:
Key | Definition |
---|---|
aggregate
|
The total time Puppet Server spent processing requests for this route. |
count
|
The total number of requests Puppet Server processed for this route. |
mean
|
The average time Puppet
Server spent on each request for this route, calculated by dividing
the aggregate value
by the count . |
route-id
|
The route being served. The
request returns a route with the special route-id of "total", which represents the
aggregate data for all requests along all routes. |
Routes for newer versions of Puppet Enterprise and newer
agents are prefixed with puppet-v3
,
while Puppet Enterprise 3 agents' routes are not. For example, a PE 2017.3 route-id
might be puppet-v3-report-/*/
, while the equivalent PE 3
agent's route-id
is :environment-report-/*/
.
For example:
"pe-master": {
{...},
"status": {
"experimental": {
"http-metrics": [
{
"aggregate": 70668,
"count": 234,
"mean": 302,
"route-id": "total"
},
{
"aggregate": 28613,
"count": 13,
"mean": 2201,
"route-id": "puppet-v3-catalog-/*/"
},
{...}
]
}
}
}
GET /status/v1/services/pe-puppet-profiler
The /status/v1/services/pe-puppet-profiler
endpoint returns JSON containing
statistics about catalog compilation. You can use this data to discover which functions or
resources are consuming the most resources or are most frequently used.
You must query it at port 8140 and append the level=debug
URL
parameter.
The Puppet Server profiler is enabled by default, but if it has been disabled, this
endpoint's metrics are not available. Instead, the endpoint returns the same keys
returned by a standard status endpoint request and an empty status
key.
Query parameters
No parameters are supported. Defaults to using the critical
status level.
Response codes
- 200 if and only if all services
report a status of
running
- 503 if any service’s status
i
unknown
orerror
Response keys
If the profiler is enabled, the response returns
two subsections in the experimental
section:
-
experimental/function-metrics
, containing statistics about functions evaluated by Puppet Server when compiling catalogs. -
experimental/resource-metrics
, containing statistics about resources declared in manifests compiled by Puppet Server.
Each function measured in the function-metrics
section also has a function key containing the function's name, and each resource
measured in the resource-metrics
section has a resource key containing the
resource's name.
The two sections otherwise share these keys:
Key | Definition |
---|---|
aggregate
|
The total time spent handling this function call or resource during catalog compilation. |
count
|
The number of times Puppet Server has called the function or instantiated the resource during catalog compilation. |
mean
|
The average time spent
handling this function call or resource during catalog compilation,
calculated by dividing the aggregate value by the count . |
For example:
"pe-puppet-profiler": {
{...},
"status": {
"experimental": {
"function-metrics": [
{
"aggregate": 1628,
"count": 407,
"function": "include",
"mean": 4
},
{...},
"resource-metrics": [
{
"aggregate": 3535,
"count": 5,
"mean": 707,
"resource": "Class[Puppet_enterprise::Profile::Console]"
},
{...},
]
}
}
}