• YugabyteDB
• YugabyteDB Anywhere

## Release announcements * [Introducing YugabyteDB 2.18: Unlock Kubernetes and Simplify Operations](https://www.yugabyte.com/blog/release-218-announcement/) ## Release notes {{< warning title="Upgrade to release 2.18.7.0 or later" >}} To avoid a rare race condition that can be encountered in v2.18.5-2.18.6, upgrade to release v2.18.7.0 or later. {{< /warning >}} What follows are the release notes for the YugabyteDB v2.18 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.18 release series. ## v2.18.9.0 - August 15, 2024 {#v2.18.9.0} **Build:** `2.18.9.0-b17` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.18.9.0/yugabytedb-2.18.9.0-b17-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.18.9.0/yugabytedb-anywhere-2.18.9.0-b17-third-party-licenses.html) ### Downloads

• macOS
• Linux x86
• Linux ARM

### Docker ```sh docker pull yugabytedb/yugabyte:2.18.9.0-b17 ``` ### Improvements #### YSQL * Ensures failed ADD/DROP pkey, ALTER TYPE operations no longer leave DocDB table renamed to _temp_old, avoiding backup/restore issues. {{}} #### DocDB * Allows asynchronous DNS cache updating and resolution retry upon failure to reduce RPC call delays and prevent unexpected leadership changes. {{}},{{}} ### Bug fixes #### YSQL * Prevents the `IN` expressions on single column from wrongly taking the tuple path, ensuring correct data processing. {{}} * Reduces sequence cache collision by incorporating both database and sequence OIDs as the entry key. {{}} * Prevents a crash related to memory release associated with TupleTableSlots in SubPlans during a Values Scan. {{}} #### DocDB * Adds validation to prevent deletion or restoration of snapshots in use by other operations. {{}} * Prevents inconsistencies between DocDB and PG by ensuring column drops cascade correctly when dependent objects are dropped. {{}} * Allows large byte requests on RateLimiter by breaking them into multiple smaller requests, ensuring smooth xCluster operation. {{}} #### CDC * Prevents newly created indexes, materialized views, and non-user tables from being added to the Change Data Capture (CDC) stream metadata. {{}} ## v2.18.8.1 - July 18, 2024 {#v2.18.8.1} **Build:** `2.18.8.1-b3` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.18.8.1/yugabytedb-2.18.8.1-b3-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.18.8.1/yugabytedb-anywhere-2.18.8.1-b3-third-party-licenses.html) ### Downloads

• macOS
• Linux x86
• Linux ARM

### Docker ```sh docker pull yugabytedb/yugabyte:2.18.8.1-b3 ``` ### Improvements #### DocDB * Allows asynchronous DNS cache updating and resolution retry upon failure to reduce RPC call delays and prevent unexpected leadership changes. {{}},{{}} ## v2.18.8.0 - June 21, 2024 {#v2.18.8.0} **Build:** `2.18.8.0-b42` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.18.8.0/yugabytedb-2.18.8.0-b42-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.18.8.0/yugabytedb-anywhere-2.18.8.0-b42-third-party-licenses.html) ### Downloads

• macOS
• Linux x86
• Linux ARM

### Docker ```sh docker pull yugabytedb/yugabyte:2.18.8.0-b42 ``` ### Improvements #### YSQL * Introduces `yb_silence_advisory_locks_not_supported_error` as a temporary solution for users to avoid disruption when using advisory locks without actual lock acquisition. {{}} * Adds new columns to localhost:13000/statements for more comprehensive database management, including user and database IDs along with varied block level statistics. {{}} #### YCQL * Now throws an error when using the unsupported GROUP BY clause in YCQL with autoflag `ycql_suppress_group_by_error` available for compatibility issues. {{}} #### DocDB * Solves the potential deadlock in DeleteTabletListAndSendRequests, while also preventing xrepl tablets deletion due to multiple calls to the function. {{}},{{}} * Allows GarbageCollector subclasses to compile without declaring a virtual destructor. {{}} #### yugabyted * Allows a smooth restart of the second node in a cluster using the `join` flag without throwing any errors. {{}} ### Bug fixes #### YSQL * Fixes YSQL upgrade single connection mode error preventing new connection attempts before the old ones are released. {{}} * Safeguards against an out-of-bounds memory write error in HashAggregate plan node with duplicate columns by rectifying array size allocation. Imported tests ensure the fix's robustness. {{}} * Corrects an issue where certain unbatchable filters weren't detected during indexpath formation when indexpath accepted batched values from multiple relations. {{}} * Resolves colocation option issues in table creation linked to table rewriting and partitioning. Enhances the `defGetBoolean` function to parse string values "0" and "1" as false and true respectively, and shifts a verification step to a earlier spot in the CREATE TABLE execution path, ensuring successful table partition creation. {{}},{{}} * Corrects checks in YbIsScanCompatible to ensure the right-hand side (RHS) of all bound index conditions, not just inequalities, fits into the left-hand side (LHS) datatype. {{}} * Refines the YbGetOrdinaryColumnsNeedingPgRecheck condition to align with the ybIsTupMismatch implementation, ensuring Postgres rechecks index conditions when the "preliminary check" is skipped due to an invalid target key attnum. {{}} * Resolves compile errors in 2.18 build associated with MemTracker and reinstates the `GetDbBlockCacheNumShardBits` definition. {{}} * Ensures pushed down RowCompareExpressions correctly enforce non-null column references, rectifying previous behavior and enhancing data accuracy. {{}} * Stops Batched Nest Loop (BNL) crashes by ensuring better indexing condition checks. {{}} * Refines the logic to accurately push down join clauses to batched index scans without causing conflicts. {{}} * Grants BNL hashtable its own expression context to prevent data overwrites during query execution. {{}} * Allows `yb_get_range_split_clause` to handle `GinNull` decoding failure in GIN Index partitions without blocking backups. {{}} * Prevents coredumps by ensuring YSQL webserver destruction upon receiving a termination signal. {{}} * Prevents the `IN` expressions on single column from wrongly taking the tuple path, ensuring correct data processing. {{}} #### YCQL * Allows the deletion of the Cassandra role in YCQLsh without it regenerating upon cluster restart, by adding a flag to mark if the role was previously created. {{}} #### DocDB * Now ensures tablets are cleared from the pending deletes state even on task failures, preventing load balancer operations issues. Additionally, fixes a race condition to prevent indefinite retention of tablets in the set after task completion. {{}} * Fix ensures heartbeat processing doesn't blindly overwrite tablet replica state, avoiding potential data corruption. {{}} * Removes unnecessary flush during snapshot deletion, preventing write blocks. {{}} * Ensures `Create Table` operation fails if `Alter Replication` encounters an error, enhancing the reliability of replication setup. {{}} * Modifies memory consumption calculations for pending operations to ensure accurate rejection of new writes at bootstrap, preventing loading failures. {{}} * Fixes a segmentation fault in yb-master by checking for a null pointer before dereferencing it, addressing an issue in the CDC run on `2.23.0.0-b37-arm`. {{}} * Reduces unnecessary logging during checkpoint operations by lowering INFO level logs to DEBUG_LEVEL, enhancing log readability. {{}} * Enhances load balancer efficiency by refining validation logic to block tablet replica additions only for those with a pending delete in progress on the same server, avoiding potential slowdowns during mass tablet replica moves. {{}} * Reduces disruptions by throttling the master process log messages related to "tablet server has a pending delete" into 20-second intervals. {{}} * Fixes hidden split parent tablets wrongly appearing as leaderless in the master's leaderless tablet endpoint, ensuring accurate load balance status and preventing potential issues with Point-in-Time Recovery (PITR) operations. {{}} #### CDC * Prevents failures in decoding change events by refreshing `cached_schema_details` when executing a new `GetChanges` request if the client indicates a necessity for the schema. {{}} ## v2.18.7.0 - April 1, 2024 {#v2.18.7.0} **Build:** `2.18.7.0-b30` **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.18.7.0/yugabytedb-2.18.7.0-b30-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.18.7.0/yugabytedb-anywhere-2.18.7.0-b30-third-party-licenses.html) ### Downloads

• macOS
• Linux x86
• Linux ARM

### Docker ```sh docker pull yugabytedb/yugabyte:2.18.7.0-b30 ``` ### Improvements #### YSQL * Treats `REFRESH MATERIALIZED VIEW` as a non-disruptive change, preventing unnecessary transaction terminations. The default option, `REFRESH MATERIALIZED VIEW NONCONCURRENTLY`, modifies metadata but without making a disruptive alteration. {{}} ### Bug fixes #### YSQL * Ensures the Linux `PDEATH_SIG` mechanism signals child processes of their parent process's exit, by correctly configuring all PG backends immediately after their fork from the postmaster process. {{}} * Fixes table rewrite issue on non-colocated tables/matviews in colocated DB, ensuring the new table uses the original table's colocation setting. Includes a workaround for GH issue 20914. {{}} * Prevents simultaneous send of read and write operations in the same RPC request that could lead to inconsistent read results, by ensuring that, in case of multiple operations, all buffered ones are flushed first. {{}} * Increases the schema version of the default partition whenever you create a new partition, preventing erroneous data insertion into the default partition due to cache refresh issues. {{}} * Adjusts heartbeat mechanism to shut down when an "Unknown Session" error occurs, reducing log alerts. This benefits idle connections with expired sessions. {{}} * Allows BNL's on outer and inner tables, even if the inner table has "unbatchable" join restrictions that can't accept batches of inputs, enhancing queries with complex join conditions. {{}} * Corrects error in creating an index using both SPLIT AT and INCLUDE by properly considering included columns in computing split rows. {{}} #### DocDB * Corrects the RPATH setting for certain OpenLDAP libraries, ensuring correct versions are used and eliminating potential system conflicts. {{}} * Corrects block cache metrics discrepancy by ensuring Statistics object passes into LRUCache from TableCache for accurate updates. {{}} * Resolves issues in the under-replicated endpoint algorithm, ensuring correct counting of replicas only when the block's minimum number of replicas has not been fulfilled yet, hence offering accurate replica tally for placement blocks. {{}} * Validates the use of two arguments for `disable_tablet_splitting`, addressing a previous condition where only one was required, thereby enhancing backup process reliability. {{}} * Updates the condition for HT lease reporting to ensure accurate leaderless tablet detection in RF-1 setup, preventing false alarms. {{}} #### CDC * Reduces tablet split data loss risk by updating `cdc_sdk_safe_time` only when a GetChanges explicit checkpoint acknowledgment is received, improving data reliability. {{}} * Fixed the decoding of NUMERIC value in CDC records to prevent precision loss by ensuring that the decoded string is not converted to scientific notation if its length is more than 20 characters. Additionally, the fix involves using the string representation with no limit on length and employing the Postgres numeric_out method for decoding, which is identical to the decoding of numerics in a PG query. {{}} ## v2.18.6.0 - February 6, 2024 {#v2.18.6.0} {{< warning title="Use 2.18.7.0 or later.">}} {{< /warning >}} ### Improvements #### YSQL * Issues a notice for unsafe ALTER TABLE operations, including for `ADD COLUMN...DEFAULT`, to indicate existing rows won't be backfilled with the default value, enhancing user awareness. Suppression possible by setting `ysql_suppress_unsafe_alter_notice` gflag to true. {{}} * Added sorting capabilities to BatchNestedLoopJoin to return the rows in the same order as NestedLoopJoin {{}} * Replaces the `ysql_max_read_restart_attempts` and `ysql_max_write_restart_attempts` gflags with `yb_max_query_layer_retries`, applies limit to Read Committed isolation statement retries, and adjusts `retry_backoff_multiplier` and `retry_min_backoff` defaults. {{}} #### DocDB * Allows tracing of UpdateConsensus API by enabling the `collect_update_consensus_traces` GFlag, providing trace logs for UpdateConsensus from remote followers while also ignoring `request.trace_requested` if both leader and follower doesn't support this change. {{}} * Allows customizing retryable request timeouts to respect client-side YCQL and YSQL timings, optimizing log replay and preventing the tserver from rejecting requests that exceed durations. Adjusts default retryable request timeout to 660 seconds and offers a configuration to eliminate server-side retention of retryable requests with `FLAGS_retryable_request_timeout_secs =0`. {{}} * Logs all instances of tablet metadata creation/updation, enabling additional insights for troubleshooting in cases of multiple meta records for the same tablet. {{}} * Automates recovery of index tables impacted by a bug, preventing performance degradation and disk size leak, by ensuring `schema.table_properties.retain_delete_markers` is reset to false when index backfilling is done. {{}} * Offers redesigned server level aggregation for metrics, thus introducing more metrics for enhanced debugging. Removes several unused URL parameters and makes the new output compatible with YugabyteDB Anywhere and YugabyteDB Aeon, preventing double-counting issues in charts. Drops unused JSON and Prometheus callbacks from MetricEntity for a cleaner design. {{}} * Enhances debugging by adding verbose logging for load balancer global and per-table state changes and including global state information when printing sorted load. {{}} * Reduces server initialization time by eliminating the accumulation of deleted tablet superblocks during startup, through a modification in the `DeleteTablet` operation. {{}} * Enables automatic recovery of index tables affected by a bug, verifying their backfilling status and correcting the `retain_delete_markers` property to enhance performance. {{}} * Integrates hard and soft server memory limit metrics into Prometheus, enabling better visualization of memory allocation and usage data, specifically in non-default value universes. {{}} * Enables control over the batching of non-deferred indexes during backfill via a new flag, improving index management. {{}} ### Bug fixes #### YSQL * Eliminates the issue where killing a background worker consumed a `Proc` struct without returning it, effectively limiting webserver restarts to eight times. {{}} * Introduces enhanced logging in YSQL webserver for better debugging of failures, by adding two new flags that log endpoint access and print basic tcmalloc stats respectively. The flags can be updated by sending a SIGHUP signal. Additionally, offers more visibility into the webserver's memory usage through the newly created `:13000/memz` and `:13000/webserver-heap-prof`. {{}} * Rectifies a segmentation fault issue in the Postmaster process in rare lock acquisition cases. {{}} * Enables postmaster restart when a backend process is unexpectedly terminated while in a critical section, preventing potential data corruption and system hangs. {{}} * Establishes `:13000/memz` and `:13000/webserver-heap-prof` endpoints to improve visibility into the webserver memory usage while managing tcmalloc stats and current allocations. Integrates a new runtime variable `yb_pg_metrics.webserver_profiler_sample_freq_bytes` to control tcmalloc sampling frequency. {{}} * Introduces the `pg_stat_statements.yb_qtext_size_limit` flag to control the maximum file size read into memory, preventing system crashes due to oversized or corrupt qtext files. {{}} * Limits the number of attempts to read inconsistent backend entries to 1000 and logs every hundredth attempt to prevent potentially infinite waiting loop. {{}} * Fixes a segmentation fault during cleanup phase in the webserver by ensuring `MyLatch` is used consistently throughout the process lifecycle. {{}} * Reduces excessive memory consumption during secondary index scans {{}} * Refines the computation of required batched/un-batched path parameters, eliminating the need to manually track non-batchable relations and simplifying logic. Fixes a bug affecting Nested Loop joins. {{}},{{}} * Upgrades "Unknown session" error to FATAL permitting drivers to instantly terminate stale connections, minimizing manual user intervention. {{}} * Rectifies correctness issue when nested correlated subqueries uses sequential scan {{}} * Rectifies correctness issues when BatchNestedLoop join is used and the join condition contained a mix of equality and non-equality filters. {{}} * Rectifies correctness issue when join on inequality condition and join columns contains NULL values {{}} * Rectifies correctness issue when queries involving outer joins and aggregate uses BNL. {{}} * Corrects the Batch Nested Loop's optimization logic for proper handling of cases where the given limit matches the outer table's exact size, ensuring accurate query results. {{}} * Addresses a bug that caused backup failure due to the absence of `yb_catalog_version`, by ensuring the function's existence post-normal migration. {{}} #### DocDB * Mitigates the issue of uneven tablet partitions and multiple pollers writing to the same consumer tablet by only applying intents on the consumer that match the producer tablet's key range. If some keys/values are filtered out from a batch, it will not delete the consumer's intents, as they may be needed by subsequent applications. Also guarantees idempotency, even if some `apply` records stutter and fetch older changes. {{}} * Reduces chances of transaction conflicts upon promotion by delaying the sending of UpdateTransactionStatusLocation RPCs until after the first PROMOTED heartbeat response is received, enhancing transaction consistency and accuracy. {{}} * Resolves potential `WriteQuery` leak issue in CQL workloads, ensuring proper execution and destruction of queries, while preventing possible tablet shutdown blockages during conflict resolution failure. {{}} * Unblocks single shard waiters once a blocking sub-transaction rolls back, by applying identical conflict check logic for both distributed transactions and single shard transactions. {{}} * Eliminates a race condition that can occur when simultaneous calls to `SendAbortToOldStatusTabletIfNeeded` try to send the abort RPC, thus preventing avoidable FATALs for failed geo promotions. {{}} * Deprecates the `enable_process_lifetime_heap_sampling` flag and allows to control tcmalloc sampling via the `profiler_sample_freq_bytes` flag alone, enhancing user control over database sampling settings. {{}} * Resolves instances of the leaderless tablet endpoint incorrectly reporting a tablet as leaderless post-leader change, by tweaking the detection logic to depend on the last occurrence of a valid leader, ensuring more accurate tablet reporting. {{}} * Allows early termination of ReadCommitted transactions with a `kConflict` error, enhancing overall system throughput by eliminating unnecessary blockages without waiting for the next RPC restart. {{}} * Modifies SysCatalog tablet's retryable request retention duration to consider both YQL and YSQL client timeouts, reducing the likelihood of `request is too old` errors during YSql DDLs. {{}} * Fixes FATAL errors occurring during tablet participant shutdown due to in-progress RPCs by ensuring `rpcs_.Shutdown` is invoked after all status resolvers have been shut down. {{}} * Handles backfill responses gracefully even when they overlap across multiple operations, reducing risks of crashes and errors due to network delays or slow masters. {{}} #### CDC * Corrects the computation of the `cdcsdk_sent_lag` metric to prevent steep, disproportionate increases by updating the `last_sent_record_time` when a `SafePoint` record is spotted, in addition to DMLs and `READ` ops. {{}} * Adjusts table drop process to shift CDCSDK streams from ACTIVE to DELETING_METADATA state, enhancing the ability to add new tables to streams even after a table drop. {{}} #### Other * Adjusts `tserver start` and `tserver stop` scripts to successfully terminate all running PG processes, irrespective of their PID digit count. {{}} ## v2.18.5.2 - January 17, 2024 {#v2.18.5.2} {{< warning title="Use 2.18.7.0 or later.">}} {{< /warning >}} ### Bug fixes #### YSQL * Fix BNL local join lookup equality function. {{}} #### CDC * Fix addition of new tables to stream metadata after drop table. {{}} ## v2.18.5.1 - January 10, 2024 {#v2.18.5.1} {{< warning title="Use 2.18.7.0 or later.">}} {{< /warning >}} ### Improvements #### YSQL * In Read Committed Isolation, limit the number of retry attempts when the aborted query is retried. {{}} ### Bug fixes #### YSQL * Return correct results when Batch Nested Loop join is used for queries involving Nested LEFT JOINs on LATERAL views. {{}},{{}} #### DocDB * In Read Committed Isolation, immediately abort transactions when conflict is detected. {{}} ## v2.18.5.0 - December 21, 2023 {#v2.18.5.0} {{< warning title="Use 2.18.7.0 or later.">}} {{< /warning >}} ### New features #### CDC * Introduces a flag: `enable_tablet_split_of_cdcsdk_streamed_tables` (default:`false`) to toggle automatic tablet splitting for tables in a CDCSDK stream, enhancing user control over replication processes. {{}} ### Improvements #### YSQL * Enables the display of unique DIST statistics through the auto_explain module, available with a new option `auto_explain.log_dist` and can be turned off as desired. {{}} * Enables the modification of `is_single_row_txn` for finer control over non-transactional writes required by `COPY`, index backfill, or when `yb_disable_transactional_writes` is set, preventing issues during non-bufferable operations for single row transactions. {{}} * Modifies YSQLDump to generate `create index NONCONCURRENTLY`, preventing automated index backfilling during the backup-restore process, making it quicker and more effective. {{}} * Verifies error return of px_cipher_decrypt in pgcrypto, a crucial step towards supporting OpenSSL 3.0+. {{}} * Enables a prerequisite import of upstream PG commit `Disable OpenSSL EVP digest padding in pgcrypto`, paving the way for OpenSSL 3.0+ upgrade. {{}} * Allows for the import of upstream PG commit `Add alternative output for OpenSSL 3 without legacy loaded`, aiding OpenSSL 3.0+ upgrade. {{}} * Prevents potential SQL injections in extension scripts by rejecting substitutions of extension schemas or owners, mitigating vulnerability CVE-2023-39417. {{}} * Mitigates potential security issues by importing upstream Postgres commit from REL_11_STABLE for future support of `ALTER