What follows are the release notes for the YugabyteDB 2024.1 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB 2024.1 release series. For an RSS feed of all release series, point your feed reader to the [RSS feed for releases](../index.xml). {{}} YugabyteDB 2024.1.0.0 and newer releases do not support v7 Linux versions (CentOS7, Red Hat Enterprise Linux 7, Oracle Enterprise Linux 7.x), Amazon Linux 2, and Ubuntu 18. If you're currently using one of these Linux versions, upgrade to a supported OS version before installing YugabyteDB v2024.1.0. Refer to [Operating system support](/v2024.1/reference/configuration/operating-systems/) for the complete list of supported operating systems. {{}} {{< tip title="New memory division settings available" >}} YugabyteDB uses [memory division flags](/v2024.1/reference/configuration/yb-master/#memory-division-flags) to specify how memory should be divided between different processes (for example, [YB-TServer](/v2024.1/architecture/yb-tserver/) versus [YB-Master](/v2024.1/architecture/yb-master/)) on a YugabyteDB node as well as within processes. Using these flags, you can better allocate memory for PostgreSQL, making it more suitable for a wider range of use cases. For _new_ v2024.1.x universes, if you are expecting to use any nontrivial amount of [YSQL](/v2024.1/api/ysql/), it is strongly recommended to set [‑‑use_memory_defaults_optimized_for_ysql](/v2024.1/reference/configuration/yb-tserver/#use-memory-defaults-optimized-for-ysql). This changes the memory division defaults to better values for YSQL usage, and optimizes memory for the node size. If you are _upgrading_ a universe, you may want to instead review your memory division settings and adjust them if desired; see [best practices](/stable/develop/best-practices-develop/data-modeling-perf/#minimize-the-number-of-tablets-you-need). In future releases, the memory division settings will be used to determine how many tablet replicas a YB-TServer can safely support; this information will power new alerts warning you about overloading nodes with too many tablet replicas and allow blocking operations that would create too many tablet replicas. {{< /tip >}} ## Release announcements * [Introducing YugabyteDB with Enhanced Postgres Compatibility: Postgres Without Limits](https://www.yugabyte.com/blog/yugabytedb-enhanced-postgres-compatibility/) ## v2024.1.6.1 - July 23, 2025 {#v2024.1.6.1} **Build:** `2024.1.6.1-b2` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.6.1/yugabytedb-2024.1.6.1-b2-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.6.1/yugabytedb-anywhere-2024.1.6.1-b2-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.6.1-b2 ``` ### Bug fix #### DocDB * Ensures accurate replication for transactional writes on xCluster range partitioned tables. {{}} ## v2024.1.6.0 - April 25, 2025 {#v2024.1.6.0} **Build:** `2024.1.6.0-b53` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.6.0/yugabytedb-2024.1.6.0-b53-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.6.0/yugabytedb-anywhere-2024.1.6.0-b53-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.6.0-b53 ``` ### Change log

View the detailed changelog

### Improvements #### YSQL * Adds new flags for role management in YSQL backups and restores. {{}} * Enables viewing TCMalloc heap snapshots for PG backend processes via new YSQL functions. {{}} * Enhances `yb_servers` function to include `universe_uuid` for better cluster identification. {{}} #### DocDB * Enhances handling of expired snapshots by retrying deletion tasks automatically. {{}} #### CDC * Reduces logging frequency for certain CDC errors to avoid clutter. {{}} ### Bug fixes #### YSQL * Ensures Batched Nested Loop joins correctly check for hashable join clauses. {{}} * Reduces XID usage by generating one per `REFRESH MATERIALIZED VIEW CONCURRENTLY` operation. {{}} * Fixes potential negative cost calculations for large table scans by using `double` for estimates. {{}} * Enhances estimation of result tuple size in edge cases, preventing division by zero errors. {{}} * Fixes incorrect role name quoting in `ysql_dumpall`. {{}} * Fixes `\d` command for indexes with spaces in their names. {{}} * Modifies `ysql_dumpall` to combine `CREATE ROLE` and `ALTER ROLE` for effective role management. {{}} * Enhances index scans and partition pruning for BOOLEAN conditions. {{}} * Enhances accuracy of seek-next estimation for index scans and paged results. {{}} * Refines cost model tuning using server-side execution times for more accurate query optimization. {{}} * Prevents crashes by handling non-variable expressions in single-row updates or deletes. {{}} * Prevents relcache reference leaks in `yb_get_ybctid_width`. {{}} * Optimizes cost modeling for primary index scans to assume sequential disk block fetching. {{}} * Enhances `yb_index_check` to verify materialized view indexes' consistency. {{}} * Corrects estimations for inner table scans in Batched Nested Loop Joins. {{}} * Fixes integer overflow in BNL cost estimation, preventing negative values. {{}} #### DocDB * Adds mtime and checksum to block cache key prefix to prevent conflicts. {{}} * Enhances transactional xCluster accuracy by using majority replicated OpId for ApplySafeTime calculations. {{}} * Ensures accurate `WaitForReplicationDrain` behavior by not mislabeling tablets as drained. {{}} * Prevents false conflict detection in snapshot isolation operations. {{}} #### yugabyted * Preserves the universe key locally after enabling EAR for recovery scenarios. {{}}

## v2024.1.5.0 - February 18, 2025 {#v2024.1.5.0} **Build:** `2024.1.5.0-b26` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.5.0/yugabytedb-2024.1.5.0-b26-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.5.0/yugabytedb-anywhere-2024.1.5.0-b26-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.5.0-b26 ``` ### Change log

View the detailed changelog

### Improvements #### YSQL * Enhances initdb logging, correctly handles process termination, and fixes memory leaks on abort signals. {{}} #### yugabyted * Ensures cached plans correctly depend on user roles, addressing {{}}. {{}} * Adds support for IPv6. {{}} #### CDC * Limits CDC GetConsistentChanges response by byte size, allowing toggling with `cdc_use_byte_threshold_for_vwal_changes`. {{}} ### Bug fixes #### YSQL * Reduces memory usage during YugabyteDB connection startup. {{}} * Removes an erroneous log entry that appeared in crashes or errors. {{}} * Enables `pg_authid` table pre-fetching during login to reduce RPC calls and prevent errors. {{}} * Ensures correct row order for queries with IN clauses. {{}} * Fixes data race in regex pushdown by using thread-local variables. {{}} * Automatically restarts the postmaster to prevent stuck processes when a background worker crashes. {{}} * Reduces scope of `ALTER DATABASE` impact to only the targeted database. {{}} * Ensures DDL operations continue smoothly by retrying background tasks on polling errors. {{}} * Reverts `ALTER DATABASE RENAME` and `ALTER DATABASE OWNER` to global impact. {{}} * Fixes incorrect aggregate column references in flattened subqueries. {{}} #### DocDB * Prevents crash during transaction abort by adjusting lock handling. {{}} * Prevents crashes during catalog reloads by safely handling null `cluster_config_`. {{}} * Prevents xCluster failures caused by same-name table re-creation during non-DBScoped DR. {{}} #### CDC * Ensures newly created tables in logical replication are retained correctly during cleanup. {{}}

## v2024.1.4.0 - January 9, 2025 {#v2024.1.4.0} **Build:** `2024.1.4.0-b108` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.4.0/yugabytedb-2024.1.4.0-b108-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.4.0/yugabytedb-anywhere-2024.1.4.0-b108-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.4.0-b108 ``` ### New features #### CDC * Enables tablet splitting on tables under CDCSDK stream by default using the flag `enable_tablet_split_of_cdcsdk_streamed_tables`. {{}} ### Change log

View the detailed changelog

### Improvements #### YSQL * Updates advisory lock error message with a workaround hint instead of `not yet implemented.` {{}} * Allows preloading of foreign key lists in relcache, avoiding on-demand master fetches, controlled by `yb_enable_fkey_catcache`. {{}} * Streamlines sys table prefetching by automatically resetting catalog read time. {{}} * Prevents catalog version bumps on no-op `ALTER ROLE` commands, saving system resources. {{}} #### YCQL * Resolves timestamp precision mismatches in YCQL, aligning inserts and index scans to millisecond precision. Enables users to optionally revert to microsecond precision using the flag `cql_revert_to_partial_microsecond_support=false`. {{}},{{}} * Tracks microsecond precision timestamps in YCQL with new metric `cql_microseconds_timestamps_used`. This tracking occurs if the `cql_revert_to_partial_microsecond_support` flag is true. {{}} #### DocDB * Blocks writes based on total bytes being flushed, not just memtable count. {{}} * Enhances load balancer to prioritize under-replicated tablets first. {{}} * Limits concurrent remote bootstraps per TServer using `load_balancer_max_inbound_remote_bootstraps_per_tserver` flag. {{}} * Reduces unnecessary attribute map creations during Prometheus metric aggregation. {{}} * Disables db_max_flushing_bytes logic to enhance write-stalling based on memtable size. {{}} * Upgrades non-FIPS OpenSSL to 3.0.15 and removes CentOS 7 builds. {{}} * Prevents overloading by correctly categorizing bootstrapping tablets in load balancing. {{}} * Enhances RBS throttling by focusing on active fetch sessions and adjusts expiration logic. {{}},{{}} * Enhances stability by fixing a missing null check in tablet lookup processes. {{}} #### CDC * Disables automatic tablet splitting for CDC stream tables by default. {{}} * Enables dynamic table addition in logical replication by default. {{}} * Automatically removes metrics when a CDC stream is deleted. {{}} #### yugabyted * Ensures accurate CPU usage metrics in the YCQL table by updating `prev_ticks_` each iteration. {{}} ### Bug fixes #### YSQL * Modifies backup/restore process to skip column name checks for indexes, allowing for successful restoration even with renamed columns. {{}} * Enhances Postgres restart stability by refining shutdown handling and lock file management. Includes killing only Postgres-associated PIDs, using `SIGQUIT` for immediate shutdowns to reduce hang-ups, and monitoring for stuck processes on macOS {{}} * Streamlines `ysql_dump` process by eliminating faulty statements for colocated table unique constraint, enhancing database reliability. {{}} * Fixes the flaw in the current DDL atomicity workflow where only the first table's schema was compared, even if it doesn't change after a schema version increment. {{}} * Alters encoding setup order to ensure `UTF8` use instead of `SQL_ASCII` and adds `pg_collation` to preloaded tables to prevent specific errors. {{}} * Prevent incorrect loading of pg_collation catalog caches, eliminating associated performance issues. {{}} * Adds HELP and TYPE metadata to each metrics onto the YSQL Prometheus metrics endpoint for better metrics accessibility. {{}} * Blocks `REINDEX` and `TRUNCATE` on system tables using table rewrite. {{}} * Fixes out-of-bound memory access during index scan cost estimation ensuring stable query execution on system catalogs and expression-based indexes when using the `yb_enable_base_scans_cost_model` configuration {{}} * Ensures parent partitioned tables correctly rewrite during operations, preventing backup/restore errors. {{}} * Assumes indexes contain 1000 rows when statistics are unavailable, ensuring better query planning. {{}} * Enhances performance during nested loop joins by ensuring correct limit resets in colocated index scans. {{}} * Enhances indexing by allowing batching of `PlaceHolderVar` in parameterized conditions. {{}} * Reduces assert strictness in YSQL, preventing crashes when only some index key columns have unbatchable clauses. {{}} * Fixes foreign key checks to avoid errors with unique indexes. {{}} * Accurately estimates the cost of partial index scans by including index conditions. {{}} * Reduces memory usage spikes by using a new memory context for `RelationBuildTriggers`. {{}} * Allows Batch Nested Loop (BNL) joins on partitioned tables in outer joins. {{}},{{}} * Ensures accurate catalog version tracking during concurrent index creation. {{}} * Fixes inconsistency in index entries during concurrent writes in the backfill phase. {{}} #### YCQL * Allows users to control error suppression for `GROUP BY` in YCQL using `ycql_ignore_group_by_error` flag. {{}} * Eliminates unexpected error during shell type definition update across multiple sessions. {{}} * Prevents TServer crashes on executing specific queries by handling NULL pointers effectively. {{}} #### DocDB * Prevents tablet bootstrap from getting stuck when replaying a truncate operation. {{}} * Allows the packed row size to exceed its limit during repacking, preventing TServer crash after `alter table add column` commands with a default value. {{}} * Ensures heartbeat processing now checks sequence numbers, avoiding overwrites that could corrupt metadata during remote bootstrapping of new tablets. This change prevents loss of leader tracking and enhances data consistency. {{}} * TServers now reject remote bootstrap requests for previously deleted tablets. {{}} * Sets a default 10MB max size for RocksDB MANIFEST files, now configurable via flags. {{}} * Enhances visibility for unexpected rate limiter sleep times in RBS threads. {{}} * Prevents crashes when calling `ListMasterServers` RPC on a master by returning an error. {{}} * Ensures continuous xCluster task operation even if the master temporarily loses the lease but retains leadership. {{}} * Prevents crashes during compaction when `ysql_enable_packed_row` is off and existing packed rows are present. {{}} * Fixes error for select queries in large tables by adjusting backward scan bounds. {{}} * Ensures consistency between `Slice::Less` and `Slice::compare` methods. {{}} * Fixes shutdown path issues to prevent fatal errors during RPC completion. {{}} * Resolves scheduling of contentious waiters to the correct, larger RPC threadpool. {{}} * Reduces latency by enhancing early resume of RPCs blocked on aborted transactions. {{}} * Enables ASAN builds to use a specific older third party version to ensure compatibility and unblock backports. {{}} #### CDC * Fixes memory leaks in the walsender's handling of transactions and file nodes. {{}} * Enables removal of dropped tables from the replica identity map to prevent replication errors. {{}} * Excludes expired or irrelevant entries from metrics calculations for more accurate reporting. {{}} * Ensures accurate history retention for various replica identities in CDC. {{}}

## v2024.1.3.1 - November 20, 2024 {#v2024.1.3.1} **Build:** `2024.1.3.1-b8` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.3.1/yugabytedb-2024.1.3.1-b8-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.3.1/yugabytedb-anywhere-2024.1.3.1-b8-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.3.1-b8 ``` ### Bug fixes #### YSQL * Enhances Postgres restarts by better handling lock files, identifying stuck processes and uses `SIGQUIT` for immediate shutdowns {{}} * Fixes a crash during cost estimation of indexes on system columns and expressions when using the new cost model {{}} ## v2024.1.3.0 - October 8, 2024 {#v2024.1.3.0} **Build:** `2024.1.3.0-b105` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.3.0/yugabytedb-2024.1.3.0-b105-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.3.0/yugabytedb-anywhere-2024.1.3.0-b105-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.3.0-b105 ``` ### New features * [Bitmap scan support](../../../reference/configuration/yb-tserver/#enable-bitmapscan). Combine multiple indexes for more efficient scans. {{}} * [Voyager assessment visualisation in yugabyted UI](/stable/yugabyte-voyager/migrate/assess-migration/). Updates to the migrations screens - "Schema Migrate page" to better capture the schema issues, and "Data Migrations UX" to readily provide voyager commands to users. * [Disk full scenarios](/stable/troubleshoot/nodes/disk-full/). Fail writes on nodes, if the free disk space on the nodes becomes low. ### Change log

View the detailed changelog

### Improvements #### YSQL * Introduces the `yb_make_next_ddl_statement_nonincrementing` configuration parameter to reduce catalog version increments during many consecutive DDLs, lowering memory and latency spikes. {{}} * Added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables for yb_db_admin role. {{}} #### YCQL * Updates YCQL tables `system_schema.functions` and `system_schema.aggregates` to align with Cassandra, aiding Cortex migration. {{}} #### DocDB * Enables reduction of duplicate code and custom flag filtering through `GetFlagInfos` relocation. {{}} * Allows reduction of thread stack size to avoid Linux hugepage backing. {{}} * Adds `emergency_repair_mode` flag and yb-admin commands `dump_sys_catalog_entries` and `write_sys_catalog_entry` to address CatalogEntity data corruption issues. {{}} * Introduces a new flag `max_disk_throughput_mbps` for automated control of write rejections when disk is full, replacing `reject_writes_min_disk_space_aggressive_check_mb`. {{}} * Allows explicit addition of flags to `gflag_allowlist.txt` for secure call home data collection, plus `version_info` in TServer data. {{}} #### CDC * Allows dynamic table addition alongside table removal from CDC streams without needing to disable or restart. {{}} * Adds a yb-admin option to disable dynamic table addition when creating new streams. {{}} * Allows automatic cleanup for ineligible tables on CDC streams and improves the removal process by fetching tablet list from a local copy instead of the state table, improving performance. Updates include removal of flag protection for Step-1 and the replacement of the preview flag. {{}} * Syncs table removal from a CDC stream to a more performant async process, allowing both manual and automatic removal via the `RemoveUserTableFromCDCSDKStream` and `RemoveTablesFromCDCSDKStream` RPCs respectively. This change utilizes a background thread to enhance user table deletion while preserving data consistency. {{}} * Enables asynchronous removal of user tables from CDC streams by adjusting how background threads process and persist stream metadata. {{}} * Disables `wal_level` checks for logical replication in YSQL as PG WAL is not used. {{}} * Changes the default `wal_level` to `logical` to ensure compatibility with logical replication clients. {{}} #### yugabyted * Enables bitmap scans and removes size-based fetching in the PostgreSQL Parity feature by updating flags. {{}} * Enhances `yugabyted configure_read_replica` commands with checks to gracefully handle failures when `data_placement_constraint` lacks `:`. {{}} * Allows yugabyted UI metrics to display correctly when deployed with Kubernetes OSS operator. {{}} * Reduces data redundancy by stopping the sending of flag information from yugabyted in callhome diagnostics. {{}} ### Bug fixes #### YSQL * Fixes an issue related to regular expression pushdown causing inconsistency and crashes by introducing thread local cache. {{}} * Resets catalog read time after table prefetching to prevent outdated readings for on-demand loading from a master. {{}} * Corrects IndexScan's indextlist to improve accuracy of variable references in subquery scenarios. {{}} * Fixes crashes caused by using a RowComparisonExpression on a reordered primary key index. {{}} * Fixes repeated calls in `DeleteTableInternal`, preventing `YbAdminSnapshotScheduleTest.SysCatalogRetention` failure on asan builds. {{}} * Resolves potential memory corruption by mutating pushdown expressions outside the per tuple context. {{}} * Eliminates the publishing of empty metrics from the `statementType` enum on the YSQL Metrics endpoint. {{}} * Revamped `Storage Read Requests` metric now better reflects true RPC count in `PrecastRequestSender` usage. {{}},{{}} * Adds more logs to aid in debugging a transaction status mismatch issue. {{}} * Addresses a bug in YSQL that caused crashes while costing an index scan on a hash index with included columns using CBO. {{}} * Fixes the hanging issue when creating a colocated table by correcting the unconditional reading of the `relkind` column. {{}} * Fixes a bug that caused incorrect deducing of commit/abort status for transactions which do not change the table's schema. {{}} * Replaces deprecated function `shared_ptr::unique` to resolve compilation errors on macOS. {{}} * Improves cache re-invalidation for `ALTER TABLE` commands to avoid schema version mismatch errors in the same session. {{}} * Adds a new flag `TEST_check_catalog_version_overflow` that can detect and prevent catalog version overflow issues. {{}} * Removes unnecessary DCHECK for smoother rolling upgrades from release 2.20, enhancing `ysql_enable_db_catalog_version_mode` flag functionality. {{}} * Enables a node to successfully join using the `join` flag by implementing a retry mechanism. {{}} #### DocDB * Limits response size by omitting per-object details from SysSnapshotEntryPB in ListSnapshotSchedules RPC. {{}} * Reduces the risk of deadlocks when making active read/write requests to a table right after a tablet-split. {{}} * Simplifies handling of large RPC responses by catching errors earlier and appropriately using `narrow_cast`. {{}} * Switches write query duration from `CoarseMonoClock` to `MonoClock` for precise nanosecond granularity. {{}} * Refactors to remove an assumption that every TServer hosts tablets, preventing crashes in certain circumstances. {{}} * Sets the `close_timestamp_micros` field for partially copied log segments to prevent time-based policy violation. {{}} * Eliminates persistent logging issue after dropping a colocated table by removing unnecessary logs. {{}} * Corrects leader balancing for geopartitioned tables to prevent early exits and ensure accurate global leader sorting. {{}} * Enables batching of metric updates for YCQL reads to prevent performance slowdowns. {{}} * Ensures "create table" requests for colocated tables don't fail due to mistaken tablet limit checks. {{}} * Ensures node restarts successfully with `secure` enabled by introducing a timeout framework. {{}} * Promotes cluster start-up when master_webserver_port is customized by considering user input during the `validate_and_set_configs` step. {{}} #### CDC * Prevents TServer crash by giving each thread its own copy of `SchemaDetailsMap` during concurrent `GetChanges` calls. {{}},{{}} * Refactors CDCSDK to prevent data loss during workload with single shard transactions by ensuring WAL messages are committed before being replicated. {{}} * Reduces disk IO overhead by optimizing the tablet-meta update process in CDCService. {{}} * Introduces three new yb-admin commands for managing user tables in a CDC stream and addresses a race condition issue by removing an optimization in the `UpdateCheckpointForTabletEntriesInCDCState` method. {{}},{{}},{{}} * Obsoletes expired or unpolled tables from CDC stream and state table, reducing unnecessary resource retention and potential blocking of split tablet deletion. {{}} * Filters out records related to index tables in colocated databases to prevent GetChanges call failures. {{}} * Ensures old-tuples for updates are only sent when replica identity is FULL, matching PostgreSQL behavior. {{}} #### yugabyted * Offers corrected DB OID in the function `calculate_table_size`, now matching `YBCGetDatabaseOid(rel)` for improved reliability. {{}}

## v2024.1.2.0 - September 3, 2024 {#v2024.1.2.0} **Build:** `2024.1.2.0-b77` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.2.0/yugabytedb-2024.1.2.0-b77-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.2.0/yugabytedb-anywhere-2024.1.2.0-b77-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.2.0-b77 ``` ### New features * [Semi-automatic transactional xCluster setup](/v2024.1/deploy/multi-dc/async-replication/async-replication-transactional/). Provides operationally simpler setup and management of YSQL transactional xCluster replication, as well as simpler steps for performing DDL changes. {{}} * [Kubernetes readiness probe](/v2024.1/deploy/kubernetes/single-zone/oss/helm-chart/#readiness-probes). Added readiness probes for TServer and Master pods in YugabyteDB, supporting custom or default configurations, thereby enhancing stability by ensuring YSQL/YCQL and YB-Master pods are ready before traffic is routed. {{}} * yugabyted * [Voyager assessment visualisation in yugabyted UI](/stable/yugabyte-voyager/migrate/assess-migration/). Yugabyted UI provides a dashboard to allow the users to effectively plan the migrations based on the complexity and also be able to monitor the progress of each migration * [Backup/restore support with TLS enabled](/v2024.1/reference/configuration/yugabyted/#backup). In secure mode, yugabyted cluster supports taking full backup/restores. {{}} * [xCluster support](/v2024.1/reference/configuration/yugabyted/#set-up-xcluster-replication-between-clusters). yugabyted enables native support for setting up xCluster between two yugabyted deployed clusters. {{}} ### Change log

View the detailed changelog

### Improvements #### YSQL * Avoids renaming DocDB tables during legacy rewrite operations to prevent issues with backup/restore and improves handling of failed ADD/DROP primary key, ALTER TYPE operations. {{}} * Adds Save/Restore state functionality to `ConsistentReadPoint` using a new `Momento` class. {{}} #### DocDB * Allows asynchronous DNS cache updating and resolution retry upon failure to reduce RPC call delays and prevent unexpected leadership changes. {{}},{{}} * Enhances stack trace tracking endpoints usability by adding an access button, a reset tracking link, and timestamp details. {{}} * Introduces a new flag `enable_rwc_lock_debugging` to control slow lock debugging and fixes a bug in `rwc_lock.cc`. {{}} #### yugabyted * Enables better handling of multi-valued flags in yugabyted without duplication, making the system more maintainable. {{}} * Enables xCluster replication management between database clusters using new yugabyted commands. {{}} * Reduces `collect_logs` command failures by removing the yugabyted running check even when the yugabyted process is not running. {{}} * Introduces `upgrade_finalize` command for easy YugabyteDB version upgrades using yugabyted CLI and includes an `upgrade_ysql_timeout` flag. {{}} * Enhances `yugabyted configure_read_replica` commands with checks to gracefully handle failures when `data_placement_constraint` lacks `:`. {{}} ### Bug fixes #### YSQL * Allows YB Admins to run pg_locks without requiring superuser status. {{}} * Allows faster data inserts into tables with identity columns. {{}} * Prevents a crash related to memory release associated with TupleTableSlots in SubPlans during a Values Scan. {{}} * Reduces prefix length in the index when using distinct index scan with included columns. {{}} * Allows resetting of stats collected by the ANALYZE command, including `reltuples`, `pg_statistic` rows, and `pg_statistic_ext` values. {{}} * Eliminates unnecessary waiting for concurrent transactions in the DEFERRABLE mode for READ ONLY serializable transactions. {{}} * Streamlines the "drop column" operation process, preventing hindrance even if the alter schema RPC is missed. {{}} * Fixes TServer crash when pushing down certain SAOP operations like string_to_array. {{}} * Allows pg_replication_slots to return an empty response instead of an error when `ysql_yb_enable_replication_commands` flag is false. {{}} * Ensures the node restarts properly even with `secure` mode enabled by adding a timeout framework. {{}} #### DocDB * Allows restoring a snapshot schedule from a time just before the oldest snapshot, improving usability. {{}} * Fixes issue of scans not honoring timeouts, preventing indefinite reads and reducing CPU usage. {{}} * Adds validation to RPCs `DeleteSnapshot` and `RestoreSnapshot` to prevent deletion or use of snapshots involved in ongoing processes. {{}} * Prevents `unexpected leader` fatal errors by updating cached leader terms immediately after a leader change. {{}} * Allows the persistent mapping of source-target schema versions when a new table is added to a colocated database, thus preventing replication from stalling after TServer restarts. {{}} * Updates cotable IDs in flushed frontiers during a snapshot restore, preventing potential post-restore issues. {{}} * Ensures object drops correctly cascade to dependent columns in the DocDB table, preventing inconsistencies. {{}} * Allows large bytes to be requested on RateLimiter, preventing indefinite call stalling. {{}} * Restores the previously missing home icon in the master user interface. {{}} * Removes extra reads during the processing of `INSERT INTO ...RETURNS STATUS AS ROW` for CQL tables with a specific primary key, improving system load and efficiency. {{}} #### CDC * Introduces new auto flag `cdcsdk_enable_identification_of_non_eligible_tables` and three yb-admin commands to manage tables in CDC stream, enhancing control and reducing unnecessary resource usage. {{}},{{}},{{}} * Fixes the issue of getting either `0` or a random time as the server's system clock in XLogData from the logical replication stream. {{}} * Simple: Adds null checks and removes table removal optimization for CDC stream, ensuring proper arrangement and functioning of user-created tables. {{}},{{}},{{}} * Enables support for dynamically allotted OID data types in CDC to prevent system crashes. {{}} * Allows handling of non-eligible table cleanup in CDC stream loading even after table drop, preventing master crash. {{}}

## v2024.1.1.0 - July 31, 2024 {#v2024.1.1.0} **Build:** `2024.1.1.0-b137` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.1.0/yugabytedb-2024.1.1.0-b137-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.1.0/yugabytedb-anywhere-2024.1.1.0-b137-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.1.0-b137 ``` ### Highlight **PostgreSQL Logical Replication Protocol Support** {{}} We're excited to announce in the 2024.1.1.0 release support for the PostgreSQL Logical Replication Protocol for Change Data Capture (CDC), in addition to the existing native [gRPC Replication protocol](/v2024.1/additional-features/change-data-capture/using-yugabytedb-grpc-replication/). This feature allows you to manage CDC streams using [Publications](https://www.postgresql.org/docs/11/sql-createpublication.html) and [Replication Slots](https://www.postgresql.org/docs/11/logicaldecoding-explanation.html#LOGICALDECODING-REPLICATION-SLOTS), similar to native PostgreSQL. Additionally, a [new connector](/v2024.1/additional-features/change-data-capture/using-logical-replication/get-started/#get-started-with-yugabytedb-connector) is introduced that utilizes the logical replication protocol to consume the CDC streams via [Replication slots](https://www.postgresql.org/docs/current/logicaldecoding-explanation.html#LOGICALDECODING-REPLICATION-SLOTS). For more information, refer to [logical replication protocol](/v2024.1/additional-features/change-data-capture/using-logical-replication/). ### New features * [Automated SQL/CQL Shell binary](/v2024.1/api/ysqlsh/#installation). Along with full binary, added separate downloadable SQL/CQL Shell binary. * [Voyager assessment visualisation in yugabyted UI](/stable/yugabyte-voyager/migrate/assess-migration/). Yugabyted UI provides a dashboard to allow the users to effectively plan the migrations based on the complexity and also be able to monitor the progress of each migration {{}} ### Change log

View the detailed changelog

### Improvements #### YSQL * Eradicates misleading log message during table creation with DDL atomicity enabled. {{}} * Removes the unused `keep_order` field from `YbctidGenerator` for cleaner results processing. {{}} * Updates read time for each operation to simplify code and avoid applying used read time from obsolete operations. {{}} * Enhances logging for DDL atomicity and corrects a loop index in cross-db unit test for clearer test logs. {{}} #### DocDB * Offers a new stack trace tracking framework for improved visibility of disk I/O operations, configurable through the `track_stack_traces` flag. {{}} * Introduces utility to dump top heap stacks when memory limit is exceeded for enhanced debugging. {{}} * Adds `SCHECK_PB_FIELDS_NOT_EMPTY` macro for validating non-empty fields, with individual checks on list elements. {{}} * Disables the unused and unreliable NamespaceReplication tests temporarily for seamless operation. {{}} * Eliminates potential deadlock issue in DeleteOrHideTabletsAndSendRequests by removing unnecessary shared lock, and introduces a test for bug 22608. {{}} * Refreshes stack trace tracking UI endpoints to enable per-column sorting and optimize sorting script. {{}} * Introduces two new `SOURCE_UNREACHABLE` and `SYSTEM_ERROR` enums to enable more detailed error reporting from the Poller. {{}} * Refreshes stack trace tracking UI endpoints to enable per-column sorting and optimize sorting script. {{}} * Introduces two new `SOURCE_UNREACHABLE` and `SYSTEM_ERROR` enums to enable more detailed error reporting from the Poller. {{}} * Shifts tcmalloc profiling code to the Utils folder. {{}} * Enhances visibility of the `Hidden` state in Master/Tserver Tables UI by shifting its position more prominently to the `State` column. {{}} #### CDC * Prevents addition of tables with enum array column to the CDC stream to avoid crashes during consumption. {{}} * Transforms the flag `yb_enable_cdc_consistent_snapshot_streams` from a preview into a default true auto flag. {{}} * Introduces three new yb-admin commands to remove a user table from a CDCSDK stream, descend dynamic table addition in a CDC stream, and validate CDC state for a particular stream, enhancing control over CDC streams. {{}},{{}} * Adds a TServer flag, `ysql_yb_default_replica_identity`, for customizing default replica identity at table creation. {{}} * Introduces `cdcsdk_enable_dynamic_table_addition` flag to manage dynamic table additions in replication slot consumption model. {{}} * Introduces replication slot name for internal distinction between two consumption models in the code. {{}} * Allows creating an old model stream via yb-admin in upgraded environments, ensuring only one stream type per database. {{}} * Eliminates unnecessary NOTICE messages when setting yb_read_time from walsender, reducing message clutter. {{}} #### yugabyted * Allows to specify multiple data directories using the new `additional_data_dir` configuration. {{}} ### Bug fixes #### YSQL * Fixes an error that occurs when decoding null values from a boolean column sorted as NULLS LAST in a secondary index. {{}} * Fixes YSQL upgrade single connection mode error preventing new connection attempts before the old ones are released. {{}} * Fixes memory leaks in pg_constraint/pg_attrdef local cache by adding a missing `hash_destroy` call in `YbCleanupTupleCache`. {{}} * Fixes a memory leak in catalog cache refresh during upgrade that led to higher memory consumption. {{}} * Resolves remaining memory leaks in CacheMemoryContext to stabilize cache memory after every catalog cache refresh. {{}} * Stops Batched Nest Loop (BNL) crashes by ensuring better indexing condition checks. {{}} * Refines the logic to accurately push down join clauses to batched index scans without causing conflicts. {{}} * Re-enables rechecking for RowCompareExpressions to accurately handle NULL inputs in scan bound calculations. {{}} * Grants BNL hashtable its own expression context to prevent data overwrites during query execution. {{}} * Allows for an enhanced readability and performance of yb_cost_index code, aiding in merging with the pg15 branch. {{}} * Fixes a bug in the index tuple width calculation for better YB base scans cost model. {{}} * Makes `yb_get_range_split_clause` robust using PG TRY CATCH block, ensuring YB backup doesn't fail. {{}} * Prevents coredumps by ensuring YSQL webserver destruction upon receiving a termination signal. {{}} * Reduces test iterations of `PgCatalogVersionTest.SqlCrossDBLoadWithDDL` in tsan/asan build to prevent timeouts. {{}} * Fixes memory leaks in ybcFetchNextHeapTuple by properly freeing the YBCStatus. {{}} * Replaces `TRUNCATE` with `DELETE FROM` in TestPgTransparentRestarts, enhancing its stability and reducing flakiness. {{}} * Allows `CreateNamespaceIfNotExists` function to retry on "already exists" error, preventing race conditions. {{}} * Removes the unused function `Catalocustomeranager::WaitForDdlVerificationToFinish` for clarity. {{}} * Resolves potential database OID collision with `system_postgres` by excluding reserved OID 65535 in allocation. {{}} * Prevents the `IN` expressions on single column from wrongly taking the tuple path, ensuring correct data processing. {{}} * Ensures bitmap scans correctly recheck all results and avoid excluding rows, improving accuracy of outcomes. {{}} * Fixes incorrect access to the scan plan's bind descriptor during tuple IN condition rechecks. {{}} * Allows the creation of new shared relations during YSQL upgrade to have a global impact by incrementing the catalog version across every database. {{}} * Corrects the YbGetOrdinaryColumnsNeedingPgRecheck function to return table column numbers instead of index numbers, preventing unnecessary data fetches and potential crashes or errors after dropping a column. {{}} * Fixes the issue when a separately created and later attached partition does not properly inherit the parent's primary key using `ALTER TABLE ...ATTACH PARTITION`. {{}} * Now allows for correct backward prefix-based scanning by eliminating the problematic `kGroupEnd` marker that was leading to inaccurate seek results. {{}} * Reduces unexpected log messages by not invoking `YsqlDdlTxnCompleteCallback` if all table 'pb_txn_id's in the DDL transaction verifier state are already cleared, avoiding potential deadlock situations in DDL atomicity. {{}} * Reduces sequence cache collision by incorporating both database and sequence OIDs as the entry key. {{}} * Resolves a detected deadlock during ALTER TABLE operations, enhancing test stability. {{}} * Prevents a crash related to memory release associated with TupleTableSlots in SubPlans during a Values Scan. {{}} * Eliminates the occurrence of "schema version mismatch" error following a DROP INDEX statement by introducing a delay in index deletion. {{}} #### DocDB * Eliminates possible deadlock during setup replication by fixing the order in which locks are acquired. {{}} * Removes unnecessary flush during snapshot deletion, preventing write blocks. {{}} * Resolves a heartbeat metrics issue ensuring full xCluster error information is sent to the new master even during a leader failover, and makes `tserver_heartbeat_metrics_interval_ms` runtime updatable. {{}} * Converted the `ysql_skip_row_lock_for_update` to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{}} * Prevents duplicate entries in the sequence data filter used by Snapshot schedules, ensuring successful operations. {{}} * Resolves the issue of `pg_locks` query failure due to missing host node UUID in distributed transactions. {{}} * Clarifies memory division flags to reflect they are percentage of the process's hard memory limit, not total available memory. {{}} * Eliminates latency spikes in conflicting workloads by preventing redundant ProbeTransactionDeadlock RPCs. {{}} * Prevents premature metric destruction during Prometheus scrapes, resolving non-UTF8 character issues. {{}} * Ensures failed xCluster setup if the xCluster stream state update to `ACTIVE` does not occur. {{}} #### CDC * Ensures deletion of MemoryContext after each GetChanges RPC to prevent memory leaks. {{}} * Prevents newly created indexes, materialized views, and non-user tables from being added to the Chang Data Capture (CDC) stream metadata. {{}} * Reduces resource usage by removing non-eligible tables, like indexes, from existing CDC SDK stream metadata, and releasing retention barriers. This change requires the master flag `enable_cleanup_of_non_eligible_tables_from_cdcsdk_stream` and limits processing to two non-eligible tables per namespace per run with `cdcsdk_table_processing_limit_per_run`. Introduces three yb-admin commands for managing CDC streams. {{}},{{}},{{}} * Fixes the issue of getting either `0` or a random time as the server's system clock in XLogData from the logical replication stream. {{}} * Fixes the segmentation fault in walsender for dynamic table addition by refreshing stored replica identities and preventing a race condition when creating dynamic tables. {{}} * Solves an issue where CDCSDK incorrectly deduces tablets as not interesting for stream before reaching the configured time limit. {{}} * Refines the logic to remove `BEGIN` record when no DML records are added, preventing potential virtual WAL crashes. {{}} * Addresses a race condition in dynamic table creation, enhancing stability during table and tablet initialization. {{}} * Increases timeout for deletion of slot entry from the state table to correct test failures in TSAN builds. {{}} * Resolves "could not open relation" error by updating slot creation method and simplifying yb_read_time logic. {{}} * Adds support for data types with dynamic OIDs in CDC to prevent crashes and allows tables with such columns to join the stream. {{}} #### yugabyted * Eliminates inconsistent failures in TestSessionParameters.java by resetting roles to default instead of dropping them. {{}}

## v2024.1.0.0 - June 4, 2024 {#v2024.1.0.0} **Build:** `2024.1.0.0-b129` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.0.0/yugabytedb-2024.1.0.0-b129-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.0.0/yugabytedb-anywhere-2024.1.0.0-b129-third-party-licenses.html) ### Downloads

**Docker:** ```sh docker pull yugabytedb/yugabyte:2024.1.0.0-b129 ``` ### Highlights **Enhanced Postgres Compatibility Mode** {{}} We're pleased to announce the early access of the new [Enhanced Postgres Compatibility Mode](/v2024.1/reference/configuration/postgresql-compatibility/) in the 2024.1.0.0 release. This mode enables you to take advantage of many new improvements in both PostgreSQL compatibility and performance parity, making it even easier to lift and shift your applications from PostgreSQL to YugabyteDB. When this mode is turned on, YugabyteDB uses the [Read-Committed](/v2024.1/architecture/transactions/read-committed/) isolation mode, the [Wait-on-Conflict](/v2024.1/architecture/transactions/concurrency-control/#wait-on-conflict) concurrency mode for predictable P99 latencies, and the new Cost Based Optimizer {{}} that takes advantage of the distributed storage layer architecture and includes query pushdowns, LSM indexes, and [batched nested loop joins](/v2024.1/architecture/query-layer/join-strategies/#batched-nested-loop-join-bnl) to offer PostgreSQL-like performance. **Rollback after upgrade** Rolling back to the pre-upgrade version if you're not satisfied with the upgraded version is now {{}}. Refer to the [Rollback phase](/v2024.1/manage/upgrade-deployment/#b-rollback-phase) for more information. ### New features * [yugabyted](/v2024.1/reference/configuration/yugabyted/) * Set preferred regions. The preferred region handles all read and write requests from clients. Use the [yugabyted configure data_placement](/v2024.1/reference/configuration/yugabyted/#data-placement) command to specify preferred regions for clusters. * Connection management integration. With connection management enabled, the **Nodes** page of yugabyted UI displays the split of physical and logical connections. * [Docker-based deployments](/v2024.1/reference/configuration/yugabyted/#create-a-multi-region-cluster-in-docker). Improves the yugabyted Docker user experience for RF-3 deployments and docker container/host restarts. * Simplified [PITR configuration](/v2024.1/reference/configuration/yugabyted/#restore). * Perform all admin operations using a [pass through mechanism](/v2024.1/reference/configuration/yugabyted/#admin-operation) to execute yb-admin commands. * DocDB Availability * Speed up local bootstrap. Faster rolling upgrades and restarts by minimizing table bootstrap time. * Hardening Raft. Reduced time window for re-tryable requests by honoring write RPC timeouts. * [Batched nested loop joins](/v2024.1/architecture/query-layer/join-strategies/#batched-nested-loop-join-bnl). A join execution strategy that is an improvement on Nested Loop joins that sends one request to the inner table per batch of outer table tuples instead of once per individual outer table tuple. * [Tablet splitting on range-sharded tables](/v2024.1/architecture/docdb-sharding/tablet-splitting/#range-sharded-tables). Optimized the tablet split thresholds to speed up data ingestion. * [Catalog Caching](/v2024.1/reference/configuration/yb-tserver/#catalog-flags). Reduce master requests during PostgreSQL system catalog refresh by populating YB-TServer catalog cache. {{}} * [Catalog Caching](/v2024.1/reference/configuration/yb-tserver/#ysql-yb-toast-catcache-threshold). Use TOAST compression to reduce PG catalog cache memory. Compressed catalog tuples when storing in the PG catalog cache to reduce the memory consumption. {{}} * [Index backfill](/v2024.1/explore/ysql-language-features/indexes-constraints/index-backfill/) stability improvements. Ensure timely notification to all nodes and PostgreSQL backends before initiating index backfill to prevent missing entries during index creation. * Support for CDC with atomic DDL. In case of DDL being rolled back, CDC will not send records with rolled back schema. * [Wait-On Conflict Concurrency Control](/v2024.1/architecture/transactions/concurrency-control/#wait-on-conflict). Cross-tablet fairness in resuming "waiters". Resume waiters in a consistent order across tablets, when a set of transactions simultaneously wait on more than one intent/lock on various tablets. * YSQL * [Cost-based optimizer](/v2024.1/reference/configuration/yb-tserver/#yb-enable-base-scans-cost-model). Added support for cost based optimizer for YSQL. {{}} * [DDL concurrency](/v2024.1/reference/configuration/yb-tserver/#ddl-concurrency-flags). Support for isolating DDLs per database. Specifically, a DDL in one database does not cause catalog cache refreshes or aborts transactions due to breaking change in another database. * [DDL atomicity](/v2024.1/reference/configuration/yb-tserver/#ddl-atomicity-flags). Ensures that YSQL DDLs are fully atomic between YSQL and DocDB layers, that is in case of any errors, they are fully rolled back, and in case of success they are applied fully. Currently, such inconsistencies are rare but can happen. * [ALTER TABLE support](/v2024.1/api/ysql/the-sql-language/statements/ddl_alter_table/#add-column-if-not-exists-column-name-data-type-constraint-constraints). Adds support for the following variants of ALTER TABLE ADD COLUMN: * with a SERIAL data type * with a volatile DEFAULT * with a PRIMARY KEY * [Lower latency for large scans with size-based fetching](/v2024.1/reference/configuration/yb-tserver/#ysql-yb-fetch-size-limit). A static size based fetch limit value to control how many rows can be returned in one request from DocDB. {{}} * [Tablet limits](/v2024.1/architecture/docdb-sharding/tablet-splitting/#tablet-limits). Depending on the available nodes and resources such as memory and CPU, YugabyteDB can automatically limit the total number of tables that can be created to ensure that the system can be stable and performant. * Truncate support with [PITR](/v2024.1/manage/backup-restore/point-in-time-recovery/). The TRUNCATE command is now allowed for databases with PITR enabled. * DocDB memory tracking enhancements. Memory tracking in DocDB to account for 90% of memory used. * [Enhanced Explain Analyze output](/v2024.1/launch-and-manage/monitor-and-alert/query-tuning/explain-analyze/). Explain Analyze when used with DIST option will also show the rows read from the storage layer, which can help diagnosing the query performance. * Upgrade OpenSSL to 3.0.8 from 1.1.1. OpenSSL 1.1.1 is out of support. This feature upgrades YugabyteDB to FIPS compliant OpenSSL version 3.0.8. ### Change log

View the detailed changelog

### Improvements #### YSQL * Enhances logging for DDL transaction conflicts and PG catalog version mismatches by including the DDL command tag and specific log details outside of the `log_ysql_catalog_versions` flag. {{}} * Displays distinct prefix keys explicitly in the explain output, enhancing the clarity of indexing for users. {{}} * Adds auto flag `ysql_yb_enable_ddl_atomicity_infra` to control DDL atomicity feature during the upgrade phase. {{}} * Allows YbInitPinnedCacheIfNeeded to only load the shared pinned cache, enhancing concurrent handling of DDLs in various databases. {{}} * Now logs global-impact DDL statements that increment all database catalog versions. {{}} * Adds a new YSQL view for YCQL statement metrics, allowing it to be joined with YCQL wait events in the `yb_active_universe_history` table. {{}} * Reduces per-backend memory consumption by reinstating TOAST compression for catalogue tables. {{}} * Avoids schema version mismatch errors during ALTER TABLE operations in cases where DDL atomicity is enabled. {{}} * Resolves schema version mismatch errors that occur after an ALTER TABLE operation due to DDL transaction verification in non-debug builds. {{}} * Introduces a new YSQL configuration parameter `yb_enable_parallel_append` to disable the unannounced feature `parallel append`. {{}} * Adds new columns to localhost:13000/statements for more comprehensive database management, including user and database IDs along with varied block level statistics. {{}} * Enables DDL atomicity feature by default by altering `ysql_yb_ddl_rollback_enabled`, `report_ysql_ddl_txn_status_to_master`, and `ysql_ddl_transaction_wait_for_ddl_verification` flags' defaults. {{}} #### YCQL * Now throws an error when using the unsupported GROUP BY clause in YCQL with autoflag `ycql_suppress_group_by_error` available for compatibility issues. {{}} #### DocDB * Introduces a new flag to toggle on or off recommended memory defaults for increased control over individual memory settings. {{}} * Boosts error messaging clarity when preview flags are not set in `allowed_preview_flags_csv`. {{}} * Shifts xCluster-related functions from `Catalocustomeranager` to `XClusterSourceManager` for easier management. {{}} * Adjusts TServer memory percentage from 50% to 48% for (4,8]GiB boxes and sets new recommendations for boxes over 16 GiB. {{}} * Returns the original error messages from CreateTable and CreateTablegroup instead of an incorrect "Database not found" message. {{}} * Updates `AreNodesSafeToTakeDown` to return earlier and deliver a readable error message when hitting a timeout, using a control flag, `are_nodes_safe_to_take_down_timeout_buffer_ms` with a default setting of 2 seconds. {{}} * Allows for faster failover in xCluster DR by skipping the cleanup process when `skip_producer_stream_deletion` is set on `DeleteUniverseReplicationRequestPB`. Safe for upgrades and rollbacks. {{}} * Adds a flag `FLAGS_tablet_split_min_size_ratio` to control tablet splitting based on SST file sizes, ensuring better control over tablet size imbalance. {{}} #### CDC * Preserves CDC stream even when all associated tables are dropped, tying its lifecycle to the database. {{}} * Allows modification of the publication refresh interval using the `cdcsdk_publication_list_refresh_interval_secs` flag. {{}} * Introduces replica identity in CDC to populate before image records, allowing table-level before image information fetching and retaining in stream metadata. {{}} * Eliminates unnecessary NOTICE messages when setting yb_read_time from walsender, reducing message clutter. {{}} * Enables transaction state to be cleared promptly after a table is deleted, preventing table deletion from getting stuck and resulting in faster functionality. {{}} #### yugabyted * Directly enables `yb_enable_read_committed_isolation` and `ysql_enable_read_request_caching` on yb-master and yb-tserver processes. {{}} * Simplifies yugabyted by dropping Python2 support and transitioning the script to use Python3, replacing deprecated distutils package with shutil. {{}},{{}} ### Bug fixes #### YSQL * Avoids failure when upgrading from version 2.14/2.16 to 2.20 by introducing a check to ensure pggate can handle RPC metrics sidecar before sending Scanned Rows count. {{}} * Safeguards against an out-of-bounds memory write error in HashAggregate plan node with duplicate columns by rectifying array size allocation. Imported tests ensure the fix's robustness. {{}} * Disables bitmap scan by default to prevent unwarranted selection due to lower CBO costs. {{}} * Allows YSQL DDL operations to wait for rollback/roll-forward operations to finish before proceeding. {{}} * Reduces unnecessary log messages when `catalog_version_table_in_perdb_mode` is set to true. {{}} * Corrects an issue where certain unbatchable filters weren't detected during indexpath formation when indexpath accepted batched values from multiple relations. Requires backports to 2.20 and 2.18. {{}} * Adds network latency cost to startup cost, yielding more accurate cost calculations in small tables. {{}} * Allows more accurate modeling of base scan costs by taking into account the impact of storage index filters on secondary index. {{}} * Renames the YSQL configuration parameter `ddl_rollback_enabled` to `yb_ddl_rollback_enabled` for specificity. {{}} * Resolves colocation option issues in table creation linked to table rewriting and partitioning. Enhances the `defGetBoolean` function to parse string values "0" and "1" as false and true respectively, and shifts a verification step to a earlier spot in the CREATE TABLE execution path, ensuring successful table partition creation. {{}},{{}} * Exposes the YSQL configuration parameter `yb_enable_optimizer_statistics` as a flag `ysql_yb_enable_optimizer_statistics`. {{}} * Corrects checks in YbIsScanCompatible to ensure the right-hand side (RHS) of all bound index conditions, not just inequalities, fits into the left-hand side (LHS) datatype. {{}} * The deadlock issue occurring when both a table and its index are deleted concurrently in yb-master has been resolved. {{}} * Fixes a bug that caused incorrect setting of global catalog version mode on TServer start. {{}} * Deprecates the unused `enable_pg_savepoints` flag to prevent irregular system behaviour. {{}} * Refines the `YbGetOrdinaryColumnsNeedingPgRecheck` condition to align with the `ybIsTupMismatch` implementation, ensuring Postgres rechecks index conditions when the "preliminary check" is skipped due to an invalid target key attnum. {{}} * Reduces the frequency of `schema version mismatch` errors during consecutive DDL operations by ensuring the up-to-date schema is fetched. {{}} * Allows usage of `YsqlDdlRollbackEnabled` in pggate C++ code by correctly passing the result of `YbDdlRollbackEnabled`, reducing DDL atomicity g-flag issues in RF 1 clusters. {{}} * Ensures pushed down `RowCompareExpressions` correctly enforce non-null column references, rectifying previous behavior and enhancing data accuracy. {{}} * The postgres process no longer crashes when running a "show all" command due to correct placement of the `yb_enable_ddl_atomicity_infra` description. {{}} * Adds a new flag `ysql_min_new_version_ignored_count` to prevent a TServer crash caused by the downward shift in yb-master's catalog version, often surfacing post a PITR restore operation. {{}} * Adjusts the value of `YB_AT_REWRITE_ALTER_PRIMARY_KEY` to prevent flag clashes and accommodate future upstream PG flags. {{}} * Fills in the "relation" column in `pg_locks` with the correct table OID after a table rewrite. {{}} * Prevents query layer retries for multi-statement queries to avoid redoing whole queries, ensuring idempotence. {{}} * Prevents unnecessary CPU cycles and log flooding by not reading `pg_yb_catalog_version` when `enable_ysql=false`. {{}} * Corrects the log message for successful column drop operations, ensuring accurate representation of DDL operations. {{}} #### YCQL * Allows the deletion of the Cassandra role in ycqlsh without it regenerating upon cluster restart, by adding a flag to mark if the role was previously created. {{}} #### DocDB * Ensures `Create Table` operation fails if `Alter Replication` encounters an error, enhancing the reliability of replication setup. {{}} * Converted the `ysql_skip_row_lock_for_update` to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{}} * Modifies memory consumption calculations for pending operations to ensure accurate rejection of new writes at bootstrap, preventing loading failures. {{}} * Trims large error messages in AsyncRpc::Failed to prevent hitting memory limit and resulting unavailability. {{}} * Renames and updates the description of the flag `min_secustomerent_size_to_rollover_at_flush` for clarity. {{}} * Changes the class of `enable_automatic_tablet_splitting` flag from `kLocalPersisted` (class 2) to `kExternal` (class 4) to eliminate setup issues with xCluster configurations. {{}} * Fixes a segmentation fault in yb-master by checking for a null pointer before dereferencing it, addressing an issue in the CDC run on `2.23.0.0-b37-arm`. {{}} * Allows DML operations on non-replicated databases and blocks DML only on databases in transactional xCluster replication STANDBY mode. Now only databases part of an inbound transactional xCluster replication group in the xCluster safe time map will have DML operations blocked. Also, certain attributes are moved from TServer to `TserverXClusterContext`. {{}} * Adds a TSAN suppression to manage the apparent race condition in the function `boost::regex_match`. {{}} * Eliminates potential FATAL errors during reported tabletPB creation by ensuring retrieval of schema version is atomic. {{}} * Enables the session to outlive the callback by holding a shared pointer to it, preventing potential crashes during concurrent DML queries. {{}} * Prevents yb-master crash by ensuring background task isn't deleted before the callback is invoked. {{}} * Enables callback completion wait in `PollTransactionStatusBase` during shutdown to prevent unexpected process termination. {{}} * Allows viewing of the RPC bind addresses in the master leader UI, especially beneficial in cases like k8s where the RPC bind address with the pod DNS is more useful than the broadcast address. {{}} * Reduces unnecessary logging during checkpoint operations by lowering INFO level logs to DEBUG_LEVEL, enhancing log readability. {{}} * Prevents fatal errors by skipping ReserveMarker/AsyncAppend if the tablet peer has already been shut down. {{}} * Enhances YSQL operation by refining task shutdown procedures and avoiding unnecessary task aborts. {{}} * Enhances load balancer efficiency by refining validation logic to block tablet replica additions only for those with a pending delete in progress on the same server, avoiding potential slowdowns during mass tablet replica moves. {{}} * Avoids multiple destruction of the same database connection, preventing system crashes due to simultaneous connection failures. {{}} * Stops fatal errors caused by the re-use of remote log anchor session during remote bootstrap from a non-leader peer. This fix ensures shared pointers are accurately tracked for `tablet_peer` objects using the `=` operator, preventing unintentional destruction of underlying objects. {{}} * Corrects a bug causing some tablet metrics to display incorrect metric_type attribute. {{}} * Enables the `skip_table_tombstone_check` for colocated tables to prevent errors. {{}} * Initializes `prev_op` to `UNKNOWN` to prevent AlmaLinux 8 fastdebug gcc11 compilation failures. {{}} * Delays `min_running_ht` initialization until after the successful completion of tablet bootstrap to prevent unexpected behaviors. {{}} * Resolves the issue of `pg_locks` query failure due to missing host node UUID in distributed transactions. {{}} * Eliminates latency spikes in conflicting workloads by preventing redundant `ProbeTransactionDeadlock` RPCs. {{}} #### CDC * Ensures deletion of MemoryContext after each GetChanges RPC to prevent memory leaks. {{}} * Fixes a memory leakage issue in the walsender process by deep freeing the cached record batch after streaming to the client. {{}} * Adds more debug logs in the walsender to aid in investigating issues like linked data loss. {{}} * Adds more debug logs for stress run debugging, skips `RollbackToSubTransaction` RPC to local TServer if not needed, and enhances debugging of the ListReplicationSlots function. {{}},{{}},{{}} * Removes table level attributes from CDCSDK metrics to avoid TServer crash due to failed DCHECK assertion. {{}} * Fixes the segmentation fault in walsender for dynamic table addition by refreshing stored replica identities and preventing a race condition when creating dynamic tables. {{}} * Solves an issue where CDCSDK incorrectly deduces tablets as not interesting for stream before reaching the configured time limit. {{}} * Enables support for streaming update operations via walsender, enhancing PG compatible logical replication support. Now executes schema changes in the logical replication protocol and maintains a record of changes in each table's read_time_ht hybrid time in the PG catalog. Includes handling late ALTER TABLE responses and addressing incomplete cleanup in the case of a stream creation failure. This feature is disabled under test flag `ysql_TEST_enable_replication_slot_consumption`. {{}} * Prevents failures in decoding change events by refreshing `cached_schema_details` when executing a new `GetChanges` request if the client indicates a necessity for the schema. {{}} #### yugabyted * Prevents SyntaxWarning and exceptions when incorrect advertise_address is given by adjusting string literals and adding check for errors. {{}},{{}},{{}} #### Other * Fixes hidden split parent tablets wrongly appearing as leaderless in the master's leaderless tablet endpoint, ensuring accurate load balance status and preventing potential issues with Point-in-Time Recovery (PITR) operations. {{}}