Skip to content
Rackone

TOOLS · CEPH

Ceph PG calculator

Calculate pg_num for every pool with the PGCalc rule or prepare the autoscaler settings, estimate how many PG instances each OSD holds before and after losing a node, and copy the ceph and pveceph commands generated from the same table.

Calculate the PGs of your pools

It starts from the base installation, as in the Ceph calculator: 3 nodes with four NVMe drives and one RBD pool in 3-way replication. The rule gives 512 PGs, because 100 × 12 / 3 is 400 and the nearest power of 2 is 512. Change the inputs or start from an example.

Calculate the PGs of your pools

Start from an editable example:
Pools and calculated PGs
Pool Protection Class Pool's share of the class data pg_num PG instances per OSD, this pool Actions
% 512 128.0 instances per OSD

PGs calculated: pool vm-disks: 512. Estimated maximum PG instances per OSD: 128, below mon_max_pg_per_osd 250.

PG instances per OSD, NVMe average 128, maximum 128, limit 250

After losing a node, the remaining nodes are not enough for every instance: 512 cannot be placed. The estimated maximum is below mon_max_pg_per_osd (250); the hard limit of a single OSD is 750. Comparing the estimated maximum with mon_max_pg_per_osd is a conservative check: the Ceph TOO_MANY_PGS warning uses the cluster average. The scale runs up to the hard limit of a single OSD.

ceph osd crush rule create-replicated rep-nvme default host nvme
ceph osd pool create vm-disks 512 512 replicated rep-nvme
ceph osd pool set vm-disks size 3
ceph osd pool set vm-disks min_size 2
ceph osd pool application enable vm-disks rbd
ceph osd pool set vm-disks pg_autoscale_mode warn
On this page
  1. How pg_num is calculated
  2. PGCalc or the autoscaler
  3. PG instances per OSD: average and maximum
  4. Ready-made commands
  5. Formulas and sources

Need a second opinion on your pools?

We review the pools, CRUSH rules and PGs of an existing Ceph cluster, or define them as part of a new cluster design.

Ceph PG calculator FAQ

What are Ceph placement groups?

Placement groups, or PGs, are the groups into which Ceph splits the objects of a pool: each PG lives on a set of OSDs, one per copy or chunk, and Ceph moves, recovers and balances data PG by PG. With too few PGs data is spread unevenly, with too many the OSDs use more memory and CPU.

Why is pg_num a power of 2?

Ceph assigns every object of a pool to a PG with a hash: with pg_num a power of 2 each PG gets the same share of objects on average, with another value some PGs get twice as many as others. That is why the PGCalc rule always rounds to a power of 2.

100 or 200 PGs per OSD?

In Ceph, 100 PGs per OSD is the default of mon_target_pg_per_osd and suits a cluster that will not grow. The PGCalc rule uses 200 when the OSD count may double, so that you do not have to raise pg_num soon after creating the pools. Staying below mon_max_pg_per_osd, 250 by default, matters more than the exact value.

What does the PG autoscaler do, and when should I turn it off?

The Ceph PG autoscaler adjusts the pg_num of every pool based on the data stored, target_size_ratio and target_size_bytes, and with bulk it makes large pools start with many PGs. Set it to warn (warnings only) or off when you want to decide pg_num yourself, for example to avoid unplanned data movement.

What is mon_max_pg_per_osd?

In Ceph, mon_max_pg_per_osd defaults to 250 in Squid and Tentacle and applies to the cluster average of PG instances per OSD: above it, Ceph raises TOO_MANY_PGS and refuses pools that would exceed it. The limit of a single OSD is three times as much, with osd_max_pg_per_osd_hard_ratio set to 3: above it the OSD creates no new PGs. The calculator also compares the estimated maximum on a single OSD with mon_max_pg_per_osd, before and after losing a node, as a conservative check.

How many PGs should an erasure-coded Ceph pool get?

In an erasure-coded Ceph pool every PG has k+m instances, one per chunk: the PGCalc rule therefore divides by k+m instead of by the number of copies. An EC 8+3 pool on 288 OSDs, with 80% of the class data and a target of 100, gets 2,048 PGs.

How does it differ from the PG Calc in the Ceph documentation?

The PG Calc in the Ceph documentation applies the same rule, but only to replicated pools. This calculator adds erasure coding, device classes, an estimate of the instances on every OSD before and after losing a node, and commands for pveceph; the old ceph.com/pgcalc address now returns a 404 error.

  • 700+ active customers
  • 24h support, every day
  • GDPR data kept in Italy

We design your Ceph cluster

We size drives, network, block.db and protection for your workload and, before production, test how the cluster reacts to failures.

Talk to a systems engineer

+39 0421 1885889 · sales@rackone.it

Let's talk about your project

Leave us your details: a systems engineer gets back to you, not a salesperson.

Privacy policy

No spam. If you are already a customer and need support, open a ticket: the link is in the contact details.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Office hours
Monday to Friday, 9:00-18:00 (Italian time) · 24h support for customers

Emergency support, 24h

Cluster down or attack in progress: call now

+39 0421 1880420
Call Talk to an expert