222 lines
7.4 KiB
ReStructuredText
222 lines
7.4 KiB
ReStructuredText
|
.. SPDX-License-Identifier: GPL-2.0
|
||
|
|
|
||
|
|
====================
|
||
|
|
Interface statistics
|
||
|
|
====================
|
||
|
|
|
||
|
|
Overview
|
||
|
|
========
|
||
|
|
|
||
|
|
This document is a guide to Linux network interface statistics.
|
||
|
|
|
||
|
|
There are three main sources of interface statistics in Linux:
|
||
|
|
|
||
|
|
- standard interface statistics based on
|
||
|
|
:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`;
|
||
|
|
- protocol-specific statistics; and
|
||
|
|
- driver-defined statistics available via ethtool.
|
||
|
|
|
||
|
|
Standard interface statistics
|
||
|
|
-----------------------------
|
||
|
|
|
||
|
|
There are multiple interfaces to reach the standard statistics.
|
||
|
|
Most commonly used is the `ip` command from `iproute2`::
|
||
|
|
|
||
|
|
$ ip -s -s link show dev ens4u1u1
|
||
|
|
6: ens4u1u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
|
||
|
|
link/ether 48:2a:e3:4c:b1:d1 brd ff:ff:ff:ff:ff:ff
|
||
|
|
RX: bytes packets errors dropped overrun mcast
|
||
|
|
74327665117 69016965 0 0 0 0
|
||
|
|
RX errors: length crc frame fifo missed
|
||
|
|
0 0 0 0 0
|
||
|
|
TX: bytes packets errors dropped carrier collsns
|
||
|
|
21405556176 44608960 0 0 0 0
|
||
|
|
TX errors: aborted fifo window heartbeat transns
|
||
|
|
0 0 0 0 128
|
||
|
|
altname enp58s0u1u1
|
||
|
|
|
||
|
|
Note that `-s` has been specified twice to see all members of
|
||
|
|
:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
|
||
|
|
If `-s` is specified once the detailed errors won't be shown.
|
||
|
|
|
||
|
|
`ip` supports JSON formatting via the `-j` option.
|
||
|
|
|
||
|
|
Protocol-specific statistics
|
||
|
|
----------------------------
|
||
|
|
|
||
|
|
Protocol-specific statistics are exposed via relevant interfaces,
|
||
|
|
the same interfaces as are used to configure them.
|
||
|
|
|
||
|
|
ethtool
|
||
|
|
~~~~~~~
|
||
|
|
|
||
|
|
Ethtool exposes common low-level statistics.
|
||
|
|
All the standard statistics are expected to be maintained
|
||
|
|
by the device, not the driver (as opposed to driver-defined stats
|
||
|
|
described in the next section which mix software and hardware stats).
|
||
|
|
For devices which contain unmanaged
|
||
|
|
switches (e.g. legacy SR-IOV or multi-host NICs) the events counted
|
||
|
|
may not pertain exclusively to the packets destined to
|
||
|
|
the local host interface. In other words the events may
|
||
|
|
be counted at the network port (MAC/PHY blocks) without separation
|
||
|
|
for different host side (PCIe) devices. Such ambiguity must not
|
||
|
|
be present when internal switch is managed by Linux (so called
|
||
|
|
switchdev mode for NICs).
|
||
|
|
|
||
|
|
Standard ethtool statistics can be accessed via the interfaces used
|
||
|
|
for configuration. For example ethtool interface used
|
||
|
|
to configure pause frames can report corresponding hardware counters::
|
||
|
|
|
||
|
|
$ ethtool --include-statistics -a eth0
|
||
|
|
Pause parameters for eth0:
|
||
|
|
Autonegotiate: on
|
||
|
|
RX: on
|
||
|
|
TX: on
|
||
|
|
Statistics:
|
||
|
|
tx_pause_frames: 1
|
||
|
|
rx_pause_frames: 1
|
||
|
|
|
||
|
|
General Ethernet statistics not associated with any particular
|
||
|
|
functionality are exposed via ``ethtool -S $ifc`` by specifying
|
||
|
|
the ``--groups`` parameter::
|
||
|
|
|
||
|
|
$ ethtool -S eth0 --groups eth-phy eth-mac eth-ctrl rmon
|
||
|
|
Stats for eth0:
|
||
|
|
eth-phy-SymbolErrorDuringCarrier: 0
|
||
|
|
eth-mac-FramesTransmittedOK: 1
|
||
|
|
eth-mac-FrameTooLongErrors: 1
|
||
|
|
eth-ctrl-MACControlFramesTransmitted: 1
|
||
|
|
eth-ctrl-MACControlFramesReceived: 0
|
||
|
|
eth-ctrl-UnsupportedOpcodesReceived: 1
|
||
|
|
rmon-etherStatsUndersizePkts: 1
|
||
|
|
rmon-etherStatsJabbers: 0
|
||
|
|
rmon-rx-etherStatsPkts64Octets: 1
|
||
|
|
rmon-rx-etherStatsPkts65to127Octets: 0
|
||
|
|
rmon-rx-etherStatsPkts128to255Octets: 0
|
||
|
|
rmon-tx-etherStatsPkts64Octets: 2
|
||
|
|
rmon-tx-etherStatsPkts65to127Octets: 3
|
||
|
|
rmon-tx-etherStatsPkts128to255Octets: 0
|
||
|
|
|
||
|
|
Driver-defined statistics
|
||
|
|
-------------------------
|
||
|
|
|
||
|
|
Driver-defined ethtool statistics can be dumped using `ethtool -S $ifc`, e.g.::
|
||
|
|
|
||
|
|
$ ethtool -S ens4u1u1
|
||
|
|
NIC statistics:
|
||
|
|
tx_single_collisions: 0
|
||
|
|
tx_multi_collisions: 0
|
||
|
|
|
||
|
|
uAPIs
|
||
|
|
=====
|
||
|
|
|
||
|
|
procfs
|
||
|
|
------
|
||
|
|
|
||
|
|
The historical `/proc/net/dev` text interface gives access to the list
|
||
|
|
of interfaces as well as their statistics.
|
||
|
|
|
||
|
|
Note that even though this interface is using
|
||
|
|
:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
|
||
|
|
internally it combines some of the fields.
|
||
|
|
|
||
|
|
sysfs
|
||
|
|
-----
|
||
|
|
|
||
|
|
Each device directory in sysfs contains a `statistics` directory (e.g.
|
||
|
|
`/sys/class/net/lo/statistics/`) with files corresponding to
|
||
|
|
members of :c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
|
||
|
|
|
||
|
|
This simple interface is convenient especially in constrained/embedded
|
||
|
|
environments without access to tools. However, it's inefficient when
|
||
|
|
reading multiple stats as it internally performs a full dump of
|
||
|
|
:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`
|
||
|
|
and reports only the stat corresponding to the accessed file.
|
||
|
|
|
||
|
|
Sysfs files are documented in
|
||
|
|
`Documentation/ABI/testing/sysfs-class-net-statistics`.
|
||
|
|
|
||
|
|
|
||
|
|
netlink
|
||
|
|
-------
|
||
|
|
|
||
|
|
`rtnetlink` (`NETLINK_ROUTE`) is the preferred method of accessing
|
||
|
|
:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` stats.
|
||
|
|
|
||
|
|
Statistics are reported both in the responses to link information
|
||
|
|
requests (`RTM_GETLINK`) and statistic requests (`RTM_GETSTATS`,
|
||
|
|
when `IFLA_STATS_LINK_64` bit is set in the `.filter_mask` of the request).
|
||
|
|
|
||
|
|
ethtool
|
||
|
|
-------
|
||
|
|
|
||
|
|
Ethtool IOCTL interface allows drivers to report implementation
|
||
|
|
specific statistics. Historically it has also been used to report
|
||
|
|
statistics for which other APIs did not exist, like per-device-queue
|
||
|
|
statistics, or standard-based statistics (e.g. RFC 2863).
|
||
|
|
|
||
|
|
Statistics and their string identifiers are retrieved separately.
|
||
|
|
Identifiers via `ETHTOOL_GSTRINGS` with `string_set` set to `ETH_SS_STATS`,
|
||
|
|
and values via `ETHTOOL_GSTATS`. User space should use `ETHTOOL_GDRVINFO`
|
||
|
|
to retrieve the number of statistics (`.n_stats`).
|
||
|
|
|
||
|
|
ethtool-netlink
|
||
|
|
---------------
|
||
|
|
|
||
|
|
Ethtool netlink is a replacement for the older IOCTL interface.
|
||
|
|
|
||
|
|
Protocol-related statistics can be requested in get commands by setting
|
||
|
|
the `ETHTOOL_FLAG_STATS` flag in `ETHTOOL_A_HEADER_FLAGS`. Currently
|
||
|
|
statistics are supported in the following commands:
|
||
|
|
|
||
|
|
- `ETHTOOL_MSG_PAUSE_GET`
|
||
|
|
- `ETHTOOL_MSG_FEC_GET`
|
||
|
|
- `ETHTOOL_MSG_MM_GET`
|
||
|
|
|
||
|
|
debugfs
|
||
|
|
-------
|
||
|
|
|
||
|
|
Some drivers expose extra statistics via `debugfs`.
|
||
|
|
|
||
|
|
struct rtnl_link_stats64
|
||
|
|
========================
|
||
|
|
|
||
|
|
.. kernel-doc:: include/uapi/linux/if_link.h
|
||
|
|
:identifiers: rtnl_link_stats64
|
||
|
|
|
||
|
|
Notes for driver authors
|
||
|
|
========================
|
||
|
|
|
||
|
|
Drivers should report all statistics which have a matching member in
|
||
|
|
:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>` exclusively
|
||
|
|
via `.ndo_get_stats64`. Reporting such standard stats via ethtool
|
||
|
|
or debugfs will not be accepted.
|
||
|
|
|
||
|
|
Drivers must ensure best possible compliance with
|
||
|
|
:c:type:`struct rtnl_link_stats64 <rtnl_link_stats64>`.
|
||
|
|
Please note for example that detailed error statistics must be
|
||
|
|
added into the general `rx_error` / `tx_error` counters.
|
||
|
|
|
||
|
|
The `.ndo_get_stats64` callback can not sleep because of accesses
|
||
|
|
via `/proc/net/dev`. If driver may sleep when retrieving the statistics
|
||
|
|
from the device it should do so periodically asynchronously and only return
|
||
|
|
a recent copy from `.ndo_get_stats64`. Ethtool interrupt coalescing interface
|
||
|
|
allows setting the frequency of refreshing statistics, if needed.
|
||
|
|
|
||
|
|
Retrieving ethtool statistics is a multi-syscall process, drivers are advised
|
||
|
|
to keep the number of statistics constant to avoid race conditions with
|
||
|
|
user space trying to read them.
|
||
|
|
|
||
|
|
Statistics must persist across routine operations like bringing the interface
|
||
|
|
down and up.
|
||
|
|
|
||
|
|
Kernel-internal data structures
|
||
|
|
-------------------------------
|
||
|
|
|
||
|
|
The following structures are internal to the kernel, their members are
|
||
|
|
translated to netlink attributes when dumped. Drivers must not overwrite
|
||
|
|
the statistics they don't report with 0.
|
||
|
|
|
||
|
|
- ethtool_pause_stats()
|
||
|
|
- ethtool_fec_stats()
|