Frequently Asked Question
When raising a Ceph support ticket, attach a standard diagnostic capture taken as close as possible to the time of the issue. This provides an immediate snapshot of cluster health, quorum state, MON/MGR status, OSD state, pools, RADOS, CephFS and local service status.
General guidance
- Run the commands from a host that has working Ceph CLI access, typically an admin node or a monitor.
- Use a user with sufficient privileges to query the cluster.
- Capture the output exactly as returned, including warnings and errors.
- If the issue is transient, collect the information while the problem is occurring if possible.
- If the cluster uses a non-default configuration or keyring, include the appropriate options in each command.
A useful approach is to save all command output into a single text file and attach that file to the ticket. Most good terminal tools can do this by default.
Minimum information to include in the ticket
Also include the following non-command information in the ticket body:
- Time the issue started
- Whether the issue is ongoing or intermittent
- Symptoms observed by users or applications
- Recent changes, for example:
- node reboot
- package updates
- network changes
- disk replacement
- OSD maintenance
- monitor or manager failover
- Impacted services:
- RBD
- CephFS
- RGW
- backups
- virtual machines
Core Ceph commands to attach
Run all of the following and attach the output.
1. Cluster health and overall status
ceph -s
ceph health
ceph health detail
ceph df
ceph osd df
ceph osd perf
These show:
- overall health summary
- detailed health warnings and errors
- cluster usage
- OSD utilisation balance
- OSD latency figures
2. Quorum and monitor status
ceph quorum_status -f json-pretty
ceph mon stat
ceph mon dump
ceph mon metadata
These show:
- which monitors are in quorum
- monitor election and membership details
- monitor addresses and ranks
- monitor daemon metadata
3. Manager status and metadata
ceph mgr stat
ceph mgr dump
ceph mgr metadata
These show:
- active and standby managers
- manager module and service state
- manager daemon metadata
4. OSD state and topology
ceph osd stat
ceph osd tree
ceph osd dump
ceph osd crush tree
ceph osd crush rule dump
ceph osd pool ls detail
These show:
- OSD up/in counts
- OSD hierarchy by host and crush location
- OSD map and flags
- CRUSH topology and rules
- pool configuration details
5. Placement groups and recovery state
ceph pg stat
ceph pg dump pgs_brief
ceph osd blocked-by
These show:
- PG state summary
- individual PG states
- any blocked OSD relationships
If the cluster is large and pg dump output is very large, it is still useful, but compress the output before attaching if needed.
6. RADOS and pool checks
rados df
rados lspools
These show:
- per-pool object and space usage
- list of pools visible to the client
7. CephFS status (if applicable)
If CephFS is in use, include:
ceph fs status
ceph fs ls
ceph fs dump
ceph mds stat
ceph mds metadata
These show:
- filesystem health and active ranks
- filesystem definitions
- MDS map and failover information
- MDS daemon metadata
8. RBD status (if applicable)
If RBD is in use, include:
rbd ls -p <pool-name>
rbd du -p <pool-name>
Use the affected RBD pool or pools. This helps confirm image visibility and space usage.
Service status from each Ceph node
For the node where the issue is observed, and ideally for all Ceph nodes, attach service state output.
Systems using systemd
systemctl status ceph.target
systemctl status 'ceph*.service'
systemctl --type=service --state=failed | grep ceph
systemctl list-units --type=service | grep ceph
These show:
- whether the Ceph target is active
- individual Ceph service states
- failed Ceph services
- loaded Ceph-related units
If using cephadm or containerised Ceph
If the cluster is managed with cephadm, also include:
ceph orch status
ceph orch ps
ceph orch host ls
These show:
- orchestrator health
- daemon placement and runtime state
- known hosts in the cephadm inventory
Optional but strongly recommended commands
These are often very useful for diagnosis and should be included where practical.
Cluster versions and report
ceph versions
ceph report
Notes:
ceph reportis extremely useful because it contains a broad cluster snapshot.- The output can be large, so save it to a file and compress it if necessary.
Configuration snapshot
ceph config dump
ceph config get mon
ceph config get mgr
If daemon-specific settings are relevant, collect them for affected daemons as well.
Recent crash information
ceph crash ls
ceph crash info <crash-id>
Include crash details for any recent entries around the time of the issue.
Suggested collection method
The following example creates a timestamped text file containing the most useful outputs.
TS=$(date +%F-%H%M%S)
OUT="ceph-helpdesk-${TS}.txt"
{
echo "===== DATE ====="
date
echo "===== HOSTNAME ====="
hostname -f
echo "===== CEPH STATUS ====="
ceph -s
ceph health
ceph health detail
ceph df
ceph osd df
ceph osd perf
echo "===== QUORUM / MON ====="
ceph quorum_status -f json-pretty
ceph mon stat
ceph mon dump
ceph mon metadata
echo "===== MGR ====="
ceph mgr stat
ceph mgr dump
ceph mgr metadata
echo "===== OSD ====="
ceph osd stat
ceph osd tree
ceph osd dump
ceph osd crush tree
ceph osd crush rule dump
ceph osd pool ls detail
echo "===== PG ====="
ceph pg stat
ceph pg dump pgs_brief
ceph osd blocked-by
echo "===== RADOS ====="
rados df
rados lspools
echo "===== CEPHFS ====="
ceph fs status
ceph fs ls
ceph fs dump
ceph mds stat
ceph mds metadata
echo "===== ORCHESTRATOR ====="
ceph orch status
ceph orch ps
ceph orch host ls
echo "===== VERSIONS / REPORT ====="
ceph versions
ceph report
echo "===== CONFIG ====="
ceph config dump
echo "===== CRASH ====="
ceph crash ls
echo "===== SYSTEMD ====="
systemctl status ceph.target
systemctl status 'ceph*.service'
systemctl --type=service --state=failed | grep ceph
systemctl list-units --type=service | grep ceph
} > "${OUT}" 2>&1
Compress the file before attaching:
gzip "${OUT}"
If the issue relates to one specific area
For monitor quorum issues
Ensure these are present:
ceph quorum_status -f json-pretty
ceph mon stat
ceph mon dump
systemctl status 'ceph-mon*.service'
For manager failover or dashboard issues
Ensure these are present:
ceph mgr stat
ceph mgr dump
ceph mgr metadata
systemctl status 'ceph-mgr*.service'
For OSD down, full or recovery issues
Ensure these are present:
ceph osd stat
ceph osd tree
ceph osd dump
ceph osd df
ceph osd perf
ceph pg stat
ceph pg dump pgs_brief
systemctl status 'ceph-osd*.service'
For CephFS issues
Ensure these are present:
ceph fs status
ceph fs dump
ceph mds stat
ceph mds metadata
systemctl status 'ceph-mds*.service'
For client, pool or object access issues
Ensure these are present:
ceph osd pool ls detail
rados lspools
rados df
If one pool is affected, also include relevant pool-specific client commands where available.
Important notes
- Some commands may return errors if a subsystem is not in use, for example CephFS or cephadm. Include that output anyway.
- Some outputs contain hostnames, IP addresses, pool names and crash metadata. Review internal data handling requirements before sharing externally.
- Large outputs such as
ceph report,ceph pg dump pgs_briefandceph config dumpshould be attached as files rather than pasted into the ticket body. - If the cluster is unhealthy enough that some commands hang, attach the output from the commands that do complete and note which commands did not return.
