Binlog Commit Optimization for Large Transactions

[SongLibing]

Binlog Commit Optimization for Large Transactions
Description

When a transaction commits, it copies the binlog events from
the binlog cache to the binlog file. Very large transactions
(e.g., gigabytes) prevent other transactions from writing to
the binary log for a long time.

The solution is to rename the binlog cache file to the new
binlog file instead of copying its content, if the committing
transaction has a large binlog cache. Since copying is avoided
and most of the I/O is outside the protection of LOCK_log, the
commits of large transactions are as fast as small transactions.

Design

• binlog_large_commit_threshold
  type: ulonglong
  scope: global
  dynamic: yes
  default: 128 MB

  Binlog cache of transactions larger than the threshold trigger
  this feature. This feature will be disabled if
  binlog_large_commit_threshold is set to 0.

• #binlog_cache_files directory
  To support renaming, all binlog cache temporary files are managed
  as regular files now. The #binlog_cache_files directory is in
  the same directory as binlog files. It is created at server
  startup if it doesn't exist. Otherwise, all files with ML_
  prefix in the directory are deleted at startup.

  The temporary files are named with the ML_ prefix and the memory
  address of the binlog_cache_data object.

• Reserve space
  The cache files must reserve enough space at the beginning
  for header events and GTID event.

  • IO_CACHE_binlog_cache_storage::m_file_reserved_bytes
    Stores the bytes reserved at the beginning of the cache file.
    The reserved space is hidden from callers. Thus, there is no
    change for callers. For example,

    • get_byte_position() still returns the length of binlog data
      written to the cache, but not the file length.
    • truncate(0) truncates the file to m_file_reserved_bytes rather than 0.

  • Empty_log_event
    It is used to pad the unused space after the rename. Empty_log_event is
    immediately after Previous_gtids_log_event. Dump thread doesn't send
    the event to replicas for compatibility reasons.

    +-----------------+
    | Magic |
    +-----------------+
    | FDE |
    +-----------------+
    | Prev_gtid |
    +-----------------+
    | Empty_log_event |
    +-----------------+
    | GTID |
    +-----------------+

• Binlog_commit_by_rotate
  It is used to encapsulate the code for renaming a binlog cache
  temporary file to a binlog file.

  • should_commit_by_rotate()
    Checks whether a binlog cache should be renamed to a binlog file.

  • commit()
    This is the entry point to rename a binlog cache and commit the
    transaction. Two rotations are performed to guarantee the renamed
    binlog file includes only one transaction in the renamed binlog file.

    • acquire LOCK_log
    • rotate 1: renaming happens here
    • commit to the storage engine.
    • rotate 2: prevent other transactions from being written.
    • release LOCK_log

  • replace_binlog_file()
    Renaming happens in the middle of a rotation. After the new binlog file
    is generated, replace_binlog_file() is called to:

    • copy the header bytes from the new binlog file to the binlog cache file.
    • delete the binlog file.
    • rename the binlog cache file to the binlog file.
    • write Empty event, Gtid event and Xid event.

    Writing the transaction's events during the rotation is more
    convenient for the binlog backup system which reads the binlog
    file directly.

• Limits

  • Neither binlog encryption nor binlog cache encryption is supported.
    Renaming will not be triggered.
  • In the renamed binlog file, checksum and compression are disabled.

Copyright (c) 2026, Oracle and/or its affiliates.

What does this change do?

mysql/mysql-community#77

Why is it needed?

Improve MySQL's stability

How was it tested?

• Added/updated MTR tests under mysql-test/
• scripts/ci/mtr.sh passes locally
• Ran the relevant full suite (name it): ______

Contributor checklist

• I have signed the OCA with the email on these commits
• Code is formatted (scripts/ci/format.sh)
• Commits are focused with descriptive messages

AI assistance

• I did not use AI assistance for this contribution
• I used AI assistance for this contribution merge code into mysql-trunk

If AI assistance was used, describe the tool(s) and extent of use:

Areas touched

Binlog Replication