Skip to content

UPS Monitor

EEPer cluster

Reads UPS battery levels, power status, load conditions and runtime estimates continuously, every few seconds. Sends alerts when the UPS runs on battery, or when its charge or runtime drops below the configured thresholds. On a power failure it can shut the nodes of the cluster down in order, or simulate it: say which nodes it would shut down, in which order and why, without shutting anything down.

Status Monitoring

Each reading takes battery charge and voltage, input/output voltage, load percentage, temperature and power source.

Notifications when the UPS switches to battery, the battery charge falls below the low or critical threshold, or the estimated runtime drops below the configured minutes.

Runtime Estimation

Reads the UPS's own runtime estimate and alerts when it falls below the configured threshold.

SNMP and NUT

Reads UPS devices via SNMP v1, v2c or v3, or from a NUT server for a UPS on a USB or serial cable, at a configurable interval, shorter while a UPS runs on battery. Profiles for the standard UPS MIB and for the major brands.

On a power failure, shuts the nodes down in order when a condition is met, the node that hosts the application last. With Dry-run it only records and notifies what it would do.

Notifier Integration

Send alerts via any configured notification channel (Telegram, Slack, email, and more) using the Notifier configuration.

Why integrate UPS into cv4pve-admin instead of NUT alone?

Status, battery, voltages, load, temperature and runtime are stored once a minute, and at every reading during a power failure. Expand a device to see its reading history, kept for the number of days set in Max days logs.

The standard UPS MIB covers most network cards, with dedicated profiles for the major brands: one module for the whole fleet, no per-vendor agent on every PVE node.

On battery, battery low, runtime dropping: Notifier sends the alert on any configured channel before the power goes out.

Battery health degrades slowly. Charts of battery charge, load, input voltage and temperature over time show when a UPS will not last another outage.

  • Overview: short description of the module
  • Dashboard: one status card for each UPS device
  • Devices: UPS device configuration, on-demand scan, connection test and reading history
  • Trends: charts of the readings over time

Devices, readings and settings are separate for each cluster.

  • SNMP v2c connection: the UPS (or its network card) must answer SNMP v2c queries from the cv4pve-admin server on the configured port (default UDP 161) with the configured community.
  • SNMP v1 connection: the same as v2c, for old network cards that answer only SNMP v1.
  • SNMP v3 connection: the same, with a user of the device instead of the community. See SNMP v3.
  • NUT connection: a NUT server (Network UPS Tools, upsd) must accept connections from the cv4pve-admin server on the configured port (default TCP 3493). See NUT connection.
  • Host is an IP address or a host name that the cv4pve-admin server can resolve.
  • A request that gets no answer within 3 seconds is treated as a failed reading.

At the top, four counters: devices Online, On Battery (with those on low battery), Critical and Offline (with those never read or in an unknown state). A fifth counter, Bypass, appears when at least one device is in bypass, and Disabled when at least one device is disabled. The disabled devices are not in the other counters.

Below, each device has a card. The devices that need attention come first: critical, on battery, bypass, low battery, offline, unknown or never read, then online. Devices with the same status are in order of name; the disabled devices come last. A disabled device is not read: its card is grey, says Disabled instead of the status and shows the values of the last reading, with its time. A card shows:

  • name and status badge, manufacturer and model, detected brand, location. The left edge of the card has the colour of the status
  • a badge with what the device does on a power failure: Alerts only, Dry-run or Shutdown
  • a notice while the device runs on battery (On battery for and how long) or does not answer (No answer for and how long)
  • for a device on Dry-run or Shutdown that runs on battery, the time at which its shutdown condition is expected (Shutdown condition at) or the condition met (Shutdown condition met)
  • Battery with its value, a bar and a small chart of the last three hours: red up to 20%, yellow up to 50%, green above
  • Load with its value, a bar and a small chart of the last three hours: green below 50%, yellow from 50%, red from 80%
  • Input and Output voltage, Runtime and Temperature
  • the time of the last reading

A value the device does not give is shown as -. An offline device keeps its last values, dimmed.

The cards show the last reading. An open page is updated every few seconds, at once when the status of a device changes.

On large screens the cards are three, four or six in a row.

The page also has the buttons Test shutdown and Simulate power failure. When at least one device is on Dry-run or Shutdown it shows a warning while Where this application runs is not set, and a strip for the power failure in progress. With a device on Shutdown it also warns when the time needed to shut everything down is not set, when Estimated runtime below (minutes) leaves less time than the shutdown needs, and when the credentials of the cluster may not shut the nodes down. See Power failure and automatic shutdown.

Status Meaning
Online Output supplied by mains power
On Battery Output supplied by the battery, charge above Battery critical alert (%)
Low Battery The UPS reports a low battery while not running on battery
Critical The UPS reports a depleted battery. Or it runs on battery and either reports a low battery or has a charge at or below Battery critical alert (%)
Offline The device gave no reading for the number of readings in a row set in Unreachable after: it did not answer, its name could not be resolved, or it has none of the values of the selected profile. The reading history shows the reason
Bypass Output supplied by the bypass line: the load is fed by mains and is not protected by the battery
Unknown The device answered but its state is none of the above, for example voltage regulation active

The grid lists the devices of the cluster with Enabled, Name, Host, Port, Connection, Detected Brand, On power failure, Last Status, Last Scan, Location and Managed Nodes (All or the number of nodes). Click the name of a device to edit it.

Each UPS device is configured with:

Field Description
Enabled Read the device (default: on). A disabled device stays in the Dashboard with its last values, marked as disabled
Name Friendly name for the UPS device (required)
Connection SNMP v1, SNMP v2c (default), SNMP v3 or NUT (Network UPS Tools). Changing it also changes the port, while the port is still the default of the connection left
Host IP address or host name of the UPS SNMP agent, or of the NUT server (required)
Port Default: 161 for SNMP, 3493 for NUT
Community SNMP v1 and v2c only: community string (default: public)
User, Security level, Authentication, Privacy and their passwords SNMP v3 only: see SNMP v3
Profile SNMP only: device profile (auto = detect the brand from the device, or pick one explicitly)
UPS name NUT only: the name of the UPS on the NUT server (default: ups)
Location Physical location label
Description Free-text notes
Managed Nodes Proxmox nodes powered by this UPS (empty = all nodes). They decide which nodes the shutdown, real or simulated, covers
Thresholds Battery low alert (%), Battery critical alert (%) and Runtime critical alert (minutes) for this device only. Empty uses the value of the cluster settings, shown in the field
On power failure Nothing (alerts only) (default), Dry-run (simulate the shutdown) or Shutdown (shut the nodes down). Changing it needs the permission Change shutdown action and is written in the audit log. Choosing Shutdown asks for a confirmation that names the nodes. See Power failure and automatic shutdown
Identification Manufacturer, Model, Firmware Version, Serial Number: filled from the device with Retrieve Info, or typed by hand

Retrieve Info reads the identification values from the device (with the selected profile for SNMP, from the NUT server for NUT) and writes them in the four fields. It needs the Host to be filled in. The values are saved with the device.

Per-device actions (select a row first):

  • Scan: reads the selected device now, also when the monitoring is off. The result appears in the grid and is always stored in the reading history, also a single missed answer
  • Test Connection: queries the device. When it answers, a dialog shows what the device declares (the SNMP system description, or the version of the NUT server), the profile or NUT driver detected and one row for each value: where it is read from (an OID or a NUT variable) and the value received, or Missing when the device does not have it. The power source and the battery state are shown with their meaning next to the number. A warning says which alert thresholds cannot fire because the device has no charge or no runtime value. When the device does not answer, a message gives the reason: no answer within 3 seconds, or a host name that cannot be resolved. A missing answer usually means a wrong address, port or community, or the SNMP version of the Connection turned off on the device. With SNMP v3 and privacy, a wrong privacy password or algorithm also gives no answer

With auto the profile is chosen by asking the device which values it has, and shown in Detected Brand. The profile found is used for the following readings and searched again only when the device stops giving its values:

  1. the profile of the manufacturer the device declares (its SNMP object identifier), when the device has the values of that brand;
  2. otherwise Generic RFC 1628, the standard UPS MIB, when the device has it. It is the case of most third-party SNMP cards;
  3. otherwise any other verified brand whose values the device has.

The text of the system description is not used: it is written by the SNMP card and often names no brand. When the device answers but has none of these values, the reading and Test Connection fail with a message saying so: the device is not a UPS, or its SNMP card does not publish the values.

With a profile picked explicitly, a device that has none of its main values (battery status, charge, output source) gives no reading, with a message that says the profile is probably the wrong one.

A verified profile is written from the published definitions (MIB) of the standard or of the manufacturer: every value, its unit and the meaning of each state. Verified profiles have been tried against a simulator, not on a real device of each brand: if the readings look wrong, run Test Connection and compare the values with the display of the device, or select Generic RFC 1628.

Profile Source Notes
Generic RFC 1628 Standard UPS MIB The standard has no serial number
APC / Schneider Electric PowerNet MIB Self-test, boost, trim, ECO and standby states count as on mains. No manufacturer value
CyberPower CPS MIB No manufacturer value
Delta DeltaUPS MIB No serial number
Eaton / Powerware XUPS MIB No battery-low state and no serial number
HPE / HP CPQPOWER MIB No battery-low state and no serial number. The MIB does not give the unit of the battery voltage: tenths of volt are assumed
Mach Power / NetAgent / Ablerex / Tecnoware XPPC MIB (NetAgent cards) No manufacturer value, no output power
MGE / Pulsar MG-SNMP-UPS MIB The MIB says “on battery: yes or no”: bypass is not told apart from mains. The MIB gives no units: tenths of volt and of hertz, runtime in seconds are assumed
Riello RIELLOUPS MIB Same structure as the standard. The MIB gives no units: those of the standard are assumed. No serial number
Tripp Lite TRIPPLITE-PRODUCTS MIB Reads the first device of the card

A value the brand does not define is read from the standard MIB, when the device has it.

These profiles are not verified: auto never chooses them, they work only when picked by hand, and the editor shows a note when one is selected.

Profile State
Socomec For Net Vision cards. Not checked against a Socomec MIB. Only the mains state is known: on battery the status is Unknown
Huawei Not checked against a Huawei MIB
Chloride, GE / General Electric, Phoenix Contact No definition found to check them against

A brand that is not in the lists is read with Generic RFC 1628, which most network cards publish. A device saved with a profile that is no longer in the lists is read as auto.

A device in bypass has the status Bypass and raises a warning: it does not start a shutdown, because mains power is there. Another state that is neither mains nor battery (output off, a state the profile does not know) gives the status Unknown and no alert.

SNMP v2c sends the community in clear text and does not authenticate who asks. SNMP v3 uses a user of the device, with authentication and, if wanted, encryption. The profiles and the values read are the same as with v2c.

Field Description
User The SNMP v3 user configured on the device
Security level Authentication and privacy (default): requests are authenticated and encrypted. Authentication only: authenticated, the values travel in clear text
Authentication and its password SHA-256 (default), SHA-384, SHA-512, SHA-1 or MD5. The password has at least 8 characters
Privacy and its password AES-128 (default), AES-192, AES-256, 3DES or DES. Shown with Authentication and privacy. The password has at least 8 characters
  • The algorithms and the passwords must be the ones set for that user on the device.
  • SHA-1, MD5, 3DES and DES are weak: the editor says so. Use them only when the device offers nothing else.
  • The passwords are stored encrypted.
  • The community of v2c and the settings of v3 are both kept: changing the connection and coming back loses nothing.

When the reading fails the message says why: user unknown to the device, authentication failed (password or algorithm), security level not accepted by the device or not allowed to that user. A wrong privacy password or algorithm usually gives no answer at all, because the device drops what it cannot decrypt: the message for a missing answer mentions it.

Tried against a Net-SNMP agent: every authentication algorithm with AES-128, SHA-256 with AES-192 and AES-256, and authentication only. Not tried on the network card of a real UPS. AES-192 and AES-256 have more than one variant among manufacturers: if a device refuses them, use AES-128. DES is tried against the same agent; 3DES is not tried, because that agent does not offer it.

cv4pve-admin asks a device all the values of a reading in one request. Some network cards refuse a request with more values, and an SNMP v1 agent fails the whole request when one of the values is missing. When the request is refused, or gets no answer while a single value does, the device is asked one value at a time, and it stays so until the application restarts. Nothing to set: the reading takes a little longer. A device that does not answer at all is waited for twice before the error. Tried with SNMP v1 against a Net-SNMP agent with a missing value, and against a simulated card that refuses requests with more values with an error or with no answer; not tried on a real card.

For a UPS without a network card, connected by USB or serial cable to another machine (a server, a NAS, a small board) where a NUT server runs. cv4pve-admin connects to that server over the network and reads the values of the UPS. It only reads: it never registers as a monitor of the UPS and never sends it a command.

What the NUT server needs:

  • to listen on the network, not only on the local address: a LISTEN line with an address the cv4pve-admin server can reach in upsd.conf, and the port (3493 by default) open on its firewall;
  • the name of the UPS, which is the name of its section in ups.conf, written in UPS name.

No user is needed: a NUT server gives the values of a UPS to any client it accepts connections from.

There are no profiles: the variable names are the same for every brand and driver. Detected Brand shows NUT followed by the driver in use on the server.

Value NUT variable
Power source and battery low ups.status: OL on mains, OB on battery, BYPASS bypass, LB battery low. The other flags are ignored
Battery charge battery.charge
Runtime battery.runtime, which is in seconds, shown in minutes
Temperature ups.temperature, or battery.temperature
Battery, input and output voltage battery.voltage, input.voltage, output.voltage
Input frequency input.frequency
Load ups.load
Output power ups.realpower
Manufacturer, model, firmware, serial number device.mfr or ups.mfr, device.model or ups.model, ups.firmware, device.serial or ups.serial

A value the driver does not give is missing, not zero: Test Connection shows which ones.

The reading fails, with a message that says what to check, when the server does not answer within 3 seconds or refuses the connection, when it has no UPS with that name, when its driver is not connected, and when its data are stale. Stale data mean that the driver has stopped updating: the server would go on giving the last good values, so they are treated as no reading, never as “on mains”.

Tried against a real NUT server with a simulated UPS (the dummy-ups driver), not with a UPS on a cable.

Expand a device row to see its readings, newest first: Timestamp, Status, Battery (%), Input (V), Load (%), Temp (C), Runtime (min) and Error. Export saves the list as an Excel file.

The devices are read much more often than the history is written. A reading is stored:

  • at every reading while the UPS runs on battery (status On Battery or Critical);
  • whenever the status changes;
  • otherwise once a minute.

Readings older than Max days logs are deleted once a day.

Pick the Period: a single date-range field, limited to the dates that have readings. Then pick one or more Devices among those with readings in that period. When the period changes, the devices already picked stay picked if they have readings in the new period. The page draws four line charts with one line for each device: Battery (%), Input Voltage (V), Load (%) and Temperature (C). Charts can be zoomed and panned. Offline readings are left out.

After each reading the module checks the values against the thresholds of the cluster settings, or against those set in the device editor when a device has its own. If at least one notifier is selected in the settings, it sends one notification that lists the alerts found:

Condition Severity
Battery charge at or below Battery critical alert (%) Error
Battery charge at or below Battery low alert (%) (and above the critical one) Warning
Runtime reported by the UPS above 0 and at or below Runtime critical alert (minutes) Error
Status Critical Error
Status On Battery Warning
Status Bypass Warning
No reading from the device for the number of readings in a row set in Unreachable after Error

The notification reports device, status, battery charge and runtime. Its severity is Error when at least one alert is an error, otherwise Warning.

  • A notification is sent when the alerts change, not at every reading: when the UPS goes on battery, when the charge then drops below a threshold, and so on. While nothing changes no further notification is sent.
  • When no alert is active any more, one back to normal notification closes the episode.
  • A single missed reading changes nothing: SNMP runs over UDP and one lost answer is common. The device keeps its status until the readings missed in a row reach Unreachable after, then it becomes Offline and the alert is sent: with the default values after about a minute and a half.
  • A UPS on battery is read at the shorter interval, also when it stops answering after going on battery.
  • No notification is sent while the state is Unknown.

These thresholds only send alerts. What a device does on a power failure is decided by other settings, the shutdown conditions.

Nothing is shut down unless a device is set to Shutdown. Start with Dry-run: it shows what the shutdown would do with your devices and your settings, over real power failures, before anything is switched off.

Each device has its own On power failure setting:

Value Meaning
Nothing (alerts only) Readings and alerts only. The default
Dry-run (simulate the shutdown) When a shutdown condition is met, records and notifies which nodes would be shut down, without shutting anything down
Shutdown (shut the nodes down) When a shutdown condition is met, sends the shutdown command to the nodes the device covers

A node fed by more than one device takes the most cautious of their values: with one on Dry-run and one on Shutdown it is simulated.

  • The command is the shutdown of the node of Proxmox VE, sent through the API of the cluster with the credentials of the cluster settings. Nothing is installed on the nodes and SSH is not used. The node stops its own guests, as with the Shutdown button of the node.
  • The credentials need the privilege Sys.PowerMgmt on the nodes. It is checked once a day for the clusters with a device on Shutdown, except during a power failure in progress (a simulated one does not count): when it is missing a notification is sent and the Dashboard says so.
  • The nodes are commanded in the order of the Nodes tab, those with the same order together. A step starts when the commands of the one before were sent, without waiting for those nodes to be off.
  • A command that fails is tried again at the following readings, three times in all. When the node the application was connected to is already off, the application connects to another node of the cluster and sends the command again. After the third failure a notification says so, and the other nodes are commanded all the same.
  • A node that is already off is skipped.
  • The node that hosts the application is commanded last, when nothing else is left to try. What happened until then is written and notified before its command, then one more notification says it is the last.
  • A node that was sent the command is not commanded again until the power failure is closed.
  • Guests are not stopped one by one, and nothing is switched on again when the mains are back.
  • High availability and Ceph need steps of their own before the nodes stop: they are not done. Test shutdown warns when the cluster has them.

Tried against Proxmox VE 9.2 nodes that run in containers, with a simulated UPS: a single node, a cluster of three, a node that is already off, credentials without the privilege, the node that hosts the application. Not tried with guests running on the nodes, nor with a real UPS during a real power failure.

They are in the Shutdown tab of the settings and are separate from the alert thresholds. They count only while a UPS runs on battery, and one is enough:

Condition Default
On battery for more than (seconds) on, 600
Estimated runtime below (minutes) on, 10
Battery charge below (%) on, 30
The UPS reports battery low or depleted on
  • A condition that is met stays met until the device is back on mains and answering: a charge that goes back above the limit does not cancel it.
  • A UPS on battery that stops answering is judged on the time on battery only, counted from when it went on battery.
  • A UPS that stops answering while on mains raises the alert and is not taken for a power failure, unless Treat a UPS that stops answering as a power failure after (minutes) is filled in. Leaving it empty is recommended.
  • Time needed to shut everything down (minutes) is how long your nodes need to stop their guests and switch off. It has no default: measure it on your installation. It gives the estimated time of the plan, and a warning when Estimated runtime below (minutes) leaves less time than that.

The Nodes tab of the settings lists the nodes of the cluster, each with the UPS devices that feed it, taken from Managed Nodes of each device. A device with no managed nodes feeds all of them.

  • Order: the commands go from the first row to the last. Move a node with the arrows.
  • Rule, for a node fed by more than one UPS: All its UPS in condition (redundant power supplies, the default) or Any one of its UPS.
  • A node whose devices include one set to Nothing (alerts only) is left on, and the plan says so.
  • Shut down the whole cluster together (default: on): when one node has to stop, all the nodes do, also those with no UPS device.
  • Where this application runs: Outside this cluster, In a guest of this cluster or Not set. Inside the cluster, the node that runs cv4pve-admin has to be the last one: after that command nothing else can be sent. With In a guest of this cluster select the Guest that hosts this application: the guest is chosen, not the node, because it can be migrated. While the value is Not set the Dashboard shows a warning.

Test shutdown, in the Dashboard, shows what would happen now if every device with an action met a shutdown condition: the nodes, the step of each one, the action and the reason. It also shows the estimated time, the warnings of the plan, and a notice when the cluster has high availability resources or uses Ceph, which need steps of their own that are not done. It changes nothing and records nothing.

Simulate power failure, in the Dashboard, asks for a confirmation and then runs the whole sequence as a test: a power failure marked as a test, one entry in Tasks with the nodes in order, the entries of the audit log and one notification marked TEST. It is refused while a real power failure is in progress.

When a device on Dry-run or Shutdown goes on battery, a power failure starts for the cluster and the Dashboard shows a strip with its start time and the nodes already recorded.

  1. A device meets a shutdown condition.
  2. The nodes of the plan are commanded, or only recorded for Dry-run, each with its order, the outcome and the reason, in Tasks and in the audit log. A node recorded is not recorded again in the same power failure, except to try a failed command again.
  3. When every device is back on mains and answering, the strip shows the time the mains came back. After five minutes of stable mains the power failure is closed.
  4. If the power fails again before that, it is the same power failure: the nodes already recorded stay so.

Close episode closes the power failure by hand and clears the conditions met. If a UPS is still on battery a new one starts at the next reading.

A restart of the application during a power failure keeps it: a condition met before the restart counts again only after a reading confirms that the device is still on battery.

With at least one notifier selected in the settings, one notification is sent for each of these events, once:

Event Severity
Power failure: the devices on battery Warning
Shutdown condition met: the devices and the condition of each one Error
Shutdown simulated: the nodes in order, with the reason. The text says that no command was sent and nothing was shut down Error
Shutdown started: the nodes that were sent the command, with the outcome and the reason Error
Shutdown, last node: the command is being sent to the node that hosts the application Error
Shutdown failed: the nodes whose command failed three times, with the reason Error
Mains back Success

Once a day, for a cluster with a device on Shutdown and no power failure in progress: a notification when the credentials may not shut the nodes down.

They are sent in addition to the alerts of each device, which do not change.

Settings are separate for each cluster.

A change of the settings, a new device or an edited one is applied within a few seconds, without restarting the application.

The settings are in four tabs: Monitoring, Alerts, Shutdown and Nodes.

Show all settings
Setting Default Purpose
Monitoring enabled off Master on/off switch: with it off the devices are read only with Scan
Reading interval (seconds) 30 How often each device is read (10 to 3600)
Reading interval on battery (seconds) 8 How often a device is read while it runs on battery (5 to 300)
Unreachable after (missed readings) 3 Readings missed in a row before the device becomes Offline and the alert is sent (1 to 20)
Max days logs 30 How many days of historical readings to retain
Battery low alert (%) 20 Warning alert when the battery charge is at or below this level (0 to 100)
Battery critical alert (%) 10 Error alert when the battery charge is at or below this level (0 to 100). When the UPS also runs on battery, the device status becomes Critical
Runtime critical alert (minutes) 5 Error alert when the estimated runtime is at or below this value (1 to 60)
Notifier - Notifier configurations that receive the alerts and the notifications of a power failure. With none selected nothing is sent
On battery for more than (seconds) on, 600 Shutdown condition: time on battery (10 to 86400)
Estimated runtime below (minutes) on, 10 Shutdown condition: runtime reported by the UPS (1 to 1440)
Battery charge below (%) on, 30 Shutdown condition: battery charge (1 to 100)
The UPS reports battery low or depleted on Shutdown condition: the state reported by the UPS
Time needed to shut everything down (minutes) empty How long the nodes need to stop their guests and switch off (1 to 1440)
Treat a UPS that stops answering as a power failure after (minutes) empty Empty: a UPS that does not answer while on mains is never taken for a power failure (1 to 1440)
Shut down the whole cluster together on When one node has to stop, all of them do
Table of the nodes - Order and Rule of each node
Where this application runs Not set Outside this cluster or In a guest of this cluster, with the Guest that hosts this application
Permission Description
UPS Monitor Open the module and its Overview page
Dashboard, Devices, Trends Open the page with that name (one permission per page)
Edit Settings Change the module settings
Device: Create, Edit, Delete, Export UPS devices. Create, Edit and Delete enable the matching actions of the Devices page
Device: Scan Now Shows the Scan button
Device: Test Connection Shows the Test Connection button
Device: Change shutdown action Change On power failure of a device, Shutdown included. Without it the field is read-only
Shutdown: Test shutdown Shows the Test shutdown button
Shutdown: Simulate power failure Shows the Simulate power failure button
Shutdown: Close episode Shows the Close episode button

The administrator role of the module includes the new permissions.

Proxmox VE privileges: the base (PVEAuditor) is enough. It reads the UPS over SNMP.

The base and the rules common to all modules are in Proxmox VE Permissions.