Get cluster configuration

Capacity (in number of txns) of an abort index segment

How often look for the inactive transactions and abort them

Whether admin API clients must provide HTTP Basic authentication headers

Enable aggregations of metrics returned by the prometheus '/metrics' endpoint. Metric aggregation is performed by summing the values of samples by labels. Aggregations are performed where it makes sense by the shard and/or partition labels.

Time to wait for entries replication in controller log when executing alter configuration requst

Size of direct write operations to disk in bytes

Allow topic auto creation

AWS access key

Optional API endpoint

TLS port override

The name of the Azure container to use with Tiered Storage. Note that the container must belong to cloud_storage_azure_storage_account

The shared key to be used for Azure Shared Key authentication with the configured Azure storage account (see cloud_storage_azure_storage_account). Note that Redpanda expects this string to be Base64 encoded.

The name of the Azure storage account to use with Tiered Storage

Optional cloud storage backend variant used to select API capabilities. If not supplied, will be inferred from other configuration parameters.

AWS bucket that should be used to store data

Minimum time between trims of tiered storage cache. If a fetch operation requires trimming the cache, and the most recent trim was within this period, then trimming will be delayed until this period has elapsed

Size of chunks of segments downloaded into cloud storage cache. Reduces space usage by only downloading the necessary chunk from a segment.

Max size of archival cache

Selects a strategy for evicting unused cache chunks.

Time interval to wait between cluster metadata uploads.

The hostname to connect to for retrieving role based credentials. Derived from cloud_storage_credentials_source if not set. Only required when using IAM role based access.

The source of credentials to connect to cloud services

Disable chunk reads and switch back to legacy mode where full segments are downloaded.

Disable TLS for all S3 connections

Enable re-uploading data for compacted topics

Default remote read config value for new topics

Default remote write value for new topics

Enables adjacent segment merging. The segments are reuploaded if there is an opportunity for that and if it will improve the tiered-storage performance

Enable archival storage

Time limit on waiting for uploads to complete before a leadership transfer. If this is null, leadership transfers will proceed without waiting.

Interval for cloud storage housekeeping tasks

The maximum number of chunks per segment that can be hydrated at a time. Above this number, unused chunks will be trimmed.

The cloud storage request rate threshold for idle state detection. If the average request rate for the configured period is lower than this threshold the cloud storage is considered being idle.

Timeout used to detect idle state of the cloud storage API. If the average cloud storage request rate is below this threshold for a configured amount of time the cloud storage is considered idle and the housekeeping jobs are started.

Initial backoff time for exponential backoff algorithm (ms)

Amount of memory that can be used to handle tiered-storage metadata

Wait at least this long between partition manifest uploads. Actual time between uploads may be greater than this interval. If this is null, metadata will be updated after each segment upload.

Manifest upload timeout (ms)

The time interval that determins how long the materialized manifest can stay in cache under contention. This parameter is used for performance tuning. When the spillover manifest is materialized and stored in cache and the cache needs to evict it it will use 'cloud_storage_materialized_manifest_ttl_ms' value as a timeout. The cursor that uses the spillover manifest uses this value as a TTL interval after which it stops referencing the manifest making it available for eviction. This only affects spillover manifests under contention.

Max https connection idle time (ms)

Max number of simultaneous connections to S3 per shard (includes connections used for both uploads and downloads)

Maximum concurrent readers of remote data per CPU core. If unset, value of topic_partitions_per_shard multiplied by 2 is used.

Maximum concurrent readers of remote data per CPU core. If unset, value of topic_partitions_per_shard is used, i.e. one reader per partition if the shard is at its maximum partition capacity.

The per-partition limit for the number of segments pending deletion from the cloud. Segments can be deleted due to retention or compaction. If this limit is breached and deletion fails, then segments will be orphaned in the cloud and will have to be removed manually

Timeout for SI metadata synchronization

The minimum number of chunks per segment for trimming to be enabled. If the number of chunks in a segment is below this threshold, the segment is small enough that all chunks in it can be hydrated at any given time

Timeout to check if new data is available for partition in S3 for read replica

Retention in bytes for topics created during automated recovery

AWS region that houses the bucket used for storage

Timeout for IAM role related operations (ms)

AWS secret key

Time that segment can be kept locally without uploading it to the remote storage (sec)

Smallest acceptable segment size in the cloud storage. Default: cloud_storage_segment_size_target/2

Desired segment size in the cloud storage. Default: segment.bytes

Log segment upload timeout (ms)

The size of the manifest which can be offloaded to the cloud. If the size of the local manifest stored in redpanda exceeds cloud_storage_spillover_manifest_size x2 the spillover mechanism will split the manifest into two parts and one of them will be uploaded to S3.

Path to certificate that should be used to validate server certificate during TLS handshake

derivative coefficient for upload PID controller.

maximum number of IO and CPU shares that archival upload can use

minimum number of IO and CPU shares that archival upload can use

proportional coefficient for upload PID controller

Initial backoff interval when there is nothing to upload for a partition (ms)

Max backoff interval when there is nothing to upload for a partition (ms)

Cluster identifier

How large in bytes should each compacted log segment be (default 256MiB)

target backlog size for compaction controller. if not set compaction target compaction backlog would be equal to

derivative coefficient for compaction PID controller.

integral coefficient for compaction PID controller.

maximum number of IO and CPU shares that compaction process can use

minimum number of IO and CPU shares that compaction process can use

proportional coefficient for compaction PID controller. This has to be negative since compaction backlog should decrease when number of compaction shares increases

Interval between iterations of controller backend housekeeping loop

Maximum capacity of rate limit accumulationin controller acls and users operations limit

Maximum capacity of rate limit accumulationin controller configuration operations limit

Maximum capacity of rate limit accumulationin controller move operations limit

Maximum capacity of rate limit accumulationin controller node management operations limit

Maximum capacity of rate limit accumulationin controller topic operations limit

Max time that will pass before we make an attempt to create a controller snapshot, after a new controller command appears

Timeout (ms) to wait for new topic creation

Default number of quota tracking windows

Default number of partitions per topic

Default replication factor for new topics

Default quota tracking window size in milliseconds

delete segments older than this - default 1 week

Disable batch cache in log manager

Disable registering metrics exposed on the internal metrics endpoint (/metrics)

Disable registering metrics exposed on the public metrics endpoint (/public_metrics)

Election timeout expressed in milliseconds

Enables limiting of controller log write rate

Enable idempotent producer

Enable automatic leadership rebalancing

Enable cluster metrics reporter

Enable pid file. You probably don't want to change this.

Enables rack-aware replica assignment

Enable SASL authentication for Kafka connections, authorization is required. see also kafka_enable_authorization

Enable Server Side Schema ID Validation.

Enable the storage space manager that coordinates and control space usage between log data and the cloud storage cache.

Enable transactions

Enables the usage tracking mechanism, storing windowed history of kafka/cloud_storage metrics over time

Whether new feature flags may auto-activate after upgrades (true) or must wait for manual activation via the admin API (false)

Maximum number of bytes returned in fetch request

Time to wait for next read in fetch request when requested min bytes wasn't reached

Minimum time before which unused session will get evicted from sessions. Maximum time after which inactive session will be deleted is two time given configuration valuecache

Time to wait for a response from tx_registry

Extra delay (ms) added to rebalance phase to wait for new members

The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.

The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating, which can overwhelm broker resources.

Timeout for new member joins

How often the system should check for expired group offsets.

Consumer group offset retention seconds. Offset retention can be disabled by setting this value to null.

Number of partitions in the internal group membership topic

How often the health manager runs

Max age of metadata cached in the health monitor of non controller node

Id allocator allocates messages in batches (each batch is a one log record) and then serves requests from memory without touching the log until the batch is exhausted.

Capacity of the id_allocator log in number of messages. Once it reached id_allocator_stm should compact the log.

Target replication factor for internal topics

Time between cluster join retries in milliseconds

Target quota rate (partition mutations per default_window_sec)

Maximum size of a batch processed by server. If batch is compressed the limit applies to compressed batch size

Maximum connections per second for one core

Maximum number of Kafka client connections per broker

Maximum number of Kafka client connections from each IP address, per broker

Enable authorization for Kafka connections. Values:- nil: Ignored. Authorization is enabled with enable_sasl: true; true: authorization is required; false: authorization is disabled. See also: enable_sasl and kafka_api[].authentication_method

Whether to include tiered storage as a special remote:// directory in DescribeLogDirs Kafka API requests.

Enable the Kafka partition reassignment API

Kafka group recovery timeout expressed in milliseconds

Limit fetch responses to this many bytes, even if total of partition bytes limits is higher

The size of the batch used to estimate memory consumption for Fetch requests, in bytes. Smaller sizes allow more concurrent fetch requests per shard, larger sizes prevent running out of memory because of too many concurrent fetch requests.

The share of kafka subsystem memory that can be used for fetch read buffers, as a fraction of kafka subsystem memory amount

Smoothing factor for kafka queue depth control depth tracking.

Update frequency for kafka queue depth control.

Enable kafka queue depth control.

Queue depth when idleness is detected in kafka queue depth control.

Smoothing parameter for kafka queue depth control latency tracking.

Maximum queue depth used in kafka queue depth control.

Max latency threshold for kafka queue depth control depth tracking.

Minimum queue depth used in kafka queue depth control.

Number of windows used in kafka queue depth control latency tracking.

Window size for kafka queue depth control latency tracking.

The lowest value of the throughput quota a shard can get in the process of quota balancing, in bytes/s. 0 means there is no minimum.

The lowest value of the throughput quota a shard can get in the process of quota balancing, expressed as a ratio of default shard quota. 0 means there is no minimum, 1 means no quota can be taken away by the balancer.

Intra-node throughput quota balancer invocation period, in milliseconds. Value of 0 disables the balancer and makes all the throughput quotas immutable.

Time window used to average current throughput measurement for quota balancer, in milliseconds

Maximum size of a single request processed via Kafka API

Userspace receive buffer max size in bytes

Kafka server TCP receive buffer size in bytes.

Kafka server TCP transmit buffer size in bytes.

Per-shard capacity of the cache for validating schema IDs.

Node wide throughput ingress limit - maximum kafka traffic throughput allowed on the ingress side of each node, in bytes/s. Default is no limit.

Node wide throughput egress limit - maximum kafka traffic throughput allowed on the egress side of each node, in bytes/s. Default is no limit.

Key-value store flush interval (ms)

Key-value maximum segment size (bytes)

Leadership rebalancing idle timeout

Leader balancer mode

Leadership rebalancing mute timeout

Per shard limit for in progress leadership transfers

Group offset retention is enabled by default in versions of Redpanda >= 23.1. To enable offset retention after upgrading from an older version set this option to true.

Permits the use of strings that may induct log injection/modification

Interval, in seconds, of how often a message informing the operator that unsafe strings are permitted

Default topic cleanup policy

How often do we trigger background compaction

Default topic compression type

Default topic messages timestamp type

Default log segment lifetime in ms for topics which do not set segment.ms

Upper bound on topic segment.ms: higher values will be clamped to this value

Lower bound on topic segment.ms: lower values will be clamped to this value

Default log segment size in bytes for topics which do not set segment.bytes

Random variation to the segment size limit used for each partition

Upper bound on topic segment.bytes: higher values will be clamped to this limit

Lower bound on topic segment.bytes: lower values will be clamped to this limit

Max compacted segment size after consolidation

Max cache size for pids which rm_stm stores inside internal state. In overflow rm_stm will delete old pids and clear their status

Fail-safe maximum throttle delay on kafka requests

Time between members backend reconciliation loop retries

If true, the redpanda process will terminate immediately when an allocation cannot be satisfied due to memory exhasution. If false, an exception is thrown instead.

Interaval for metadata dissemination batching

Number of attempts of looking up a topic's meta data like shard before failing a request

Delay before retry a topic lookup in a shard or other meta tables

Maximum time to wait in metadata request for cluster health to be refreshed

cluster metrics reporter report interval

Cluster metrics reporter tick interval

cluster metrics reporter url

How long after the last heartbeat request a node will wait before considering itself to be isolated

Timeout for executing node management operations

Time interval between two node status messages. Node status messages establish liveness status outside of the Raft protocol.

Number of partitions that can be reassigned at once

Disk usage threshold that triggers moving partitions from the node

Minimum size of partition that is going to be prioritized when rebalancing cluster due to disk size threshold being breached. By default this value is calculated automaticaly

Partition autobalancing mode

Total size of partitions that autobalancer is going to move in one batch

Node unavailability timeout that triggers moving partitions from the node

Partition autobalancer tick interval

If the number of scheduled tick moves drops by this ratio, a new tick is scheduled immediately. Valid values are (0, 1]. For #example, with a value of 0.2 and 100 scheduled moves in a tick, a new tick is scheduled when the inprogress moves are < 80.

Quota manager GC frequency in milliseconds

After how many failed heartbeats to forcibly close an unresponsive TCP connection. Set to 0 to disable force disconnection.

Milliseconds for raft leader heartbeats

raft heartbeat RPC timeout

Raft I/O timeout

Raft learner recovery rate limit in bytes per sec

Maximum number of concurrent append entries requests sent by leader to one follower

Max memory that can be used for reads in raft recovery process by default 15% of total memory

default size of read issued during raft follower recovery

Disables dynamic rate allocation in recovery throttle (advanced).

Max size of requests cached for replication

Maximum number of x-core requests pending in Raft seastar::smp group. (for more details look at seastar::smp_service_group documentation)

Timeout for a timeout now request

Timeout waiting for follower recovery when transferring leadership

Duration after which inactive readers will be evicted from cache

Free memory limit that will be kept by batch cache background reclaimer

Length of time in which reclaim sizes grow

Maximum batch cache reclaim size

Minimum batch cache reclaim size

Length of time above which growth is reset

Timeout for append entries requests issued while updating stale follower

Free cache when segments roll

Timeout for append entries requests issued while replicating entries

Default max bytes per partition on disk before triggering a compaction

Local retention size target for partitions of topics with cloud storage write enabled

Local retention time target for partitions of topics with cloud storage write enabled

Time to wait state catch up before rejecting a request

TCP connection queue length for Kafka server and internal RPC server

Internal RPC TCP receive buffer size in bytes.

Internal RPC TCP transmit buffer size in bytes.

Rate limit for controller acls and users operations

Rate limit for controller configuration operations

Rate limit for controller move operations

Rate limit for controller node management operations

Rate limit for controller topic operations

The location of the Kerberos krb5.conf file for Redpanda

The location of the Kerberos keytab file for Redpanda

The primary of the Kerberos Service Principal Name (SPN) for Redpanda

Maximum delay until buffered data is written

Size for segments fallocation

Minimum size of the seq table non affected by compaction

Maximum number of bytes that may be used on each shard by compactionindex writers

If set, timestamps more than this many seconds in the future relative tothe server's clock will be ignored for data retention purposes, and retention will act based on another timestamp in the same segment, or the mtime of the segment file if no valid timestamp is available

Maximum number of partitions' logs that will be replayed concurrently at startup, or flushed concurrently on shutdown.

Threshold of minimum bytes free space before rejecting producers.

Size of each read buffer (one per in-flight read, per log segment)

How many additional reads to issue ahead of current read location

The number of segments per partition that the system will attempt to reserve disk capcity for. For #example, if the maximum segment size is configured to be 100 MB, and the value of this option is 2, then in a system with 10 partitions Redpanda will attempt to reserve at least 2 GB of disk space.

Threshold of minimim bytes free space before setting storage space alert

Threshold of minimim percent free space before setting storage space alert

Requires that an empty file named .redpanda_data_dir be present in the data directory. Redpanda will refuse to start if it is not found.

Target bytes to replay from disk on startup after clean shutdown: controls frequency of snapshots and checkpoints

Target fetch size quota byte rate (bytes per second) - disabled default

Target request size quota byte rate (bytes per second) - 2GB default

Time to wait state catch up before rejecting a request

Required file handles per partition when creating topics

Required memory per partition when creating topics

Maximum number of partitions which may be allocated to one shard (CPU core)

Reserved partition slots on shard (CPU core) 0 on each node. If this is >= topic_partitions_per_core, no data partitions will be scheduled on shard 0

Cleanup policy for a transaction coordinator topic

delete segments older than this - default 1 week

How large in bytes should each log segment be (default 1G)

Amount of partitions for transactions coordinator

Producer ids are expired once this time has elapsed after the last write with the given producer id.

How often to log per partition tx stats, works only with debug logging enabled.

Delay before scheduling next check for timed out transactions

The interval in which all usage stats are written to disk

The number of windows to persist in memory and disk

The width of a usage window, tracking cloud and kafka ingress/egress traffic each interval

Use a separate scheduler group for fetch processing

Timeout (ms) to wait for leadership in metadata cache

Size of the zstd decompression workspace