Skip to content

separate write db and read db - #121

Open
sredxny wants to merge 13 commits into
mainfrom
allow-write-and-read-connection
Open

separate write db and read db#121
sredxny wants to merge 13 commits into
mainfrom
allow-write-and-read-connection

fix schema test

bbab3b8
Select commit
Loading
Failed to load commit list.
probelabs / Visor: overview succeeded Sep 30, 2025 in 5m 5s

✅ Check Passed

overview check completed successfully with no issues found.

Details

📊 Summary

  • Total Issues: 1

🐛 Issues by Category

📚 Documentation (1)

  • ℹ️ AI_RESPONSE:1 - This is an excellent pull request that introduces a crucial feature for database scalability. Here is a comprehensive analysis:

1. Change Impact Analysis

What this PR accomplishes

This pull request refactors the PostgreSQL database driver to support read/write splitting. It introduces the ability to configure separate database connections for write operations (like INSERT, UPDATE, DELETE) and read operations (SELECT). This allows the application to direct write traffic to a primary database and offload read traffic to one or more replica databases.

This is a significant architectural improvement that enables:

  • Scalability: Read-heavy workloads can be scaled by adding more read replicas without impacting the performance of the primary write database.
  • Performance: Reduces the load on the primary database, leading to faster query execution for both reads and writes.
  • High Availability: Can be part of a high-availability strategy, allowing the application to continue serving read requests even if the primary database is temporarily unavailable.

The implementation is backward-compatible. If a separate read connection string is not provided, the driver defaults to using the primary (write) connection for all operations, ensuring existing deployments continue to work without any configuration changes.

Key technical changes introduced

  • New Configuration Option: A ReadConnectionString field has been added to the ClientOpts struct (persistent/internal/types/client_options.go), allowing users to specify a connection string for a read replica.

  • Dual Connection Management: The connection management logic in persistent/internal/driver/postgres/lifecycle.go has been fundamentally updated. The lifeCycle struct now holds two separate gorm.DB connections: writeDB and readDB. The Connect method establishes both connections if two different connection strings are provided, and the Close and Ping methods are updated to manage the lifecycle of both.

  • Intelligent Query Routing: All database operations within the PostgreSQL driver have been systematically routed to the appropriate connection:

    • Write Operations: All functions that modify data or schema (e.g., Insert, Update, Delete, Migrate, CreateIndex) now exclusively use the writeDB connection. These changes are primarily in basic_operations.go, indexes.go, and schema.go.
    • Read Operations: All functions that only retrieve data (e.g., Query, Count, GetIndexes, DBTableStats, HasTable) now use the readDB connection. These changes are found in query.go, indexes.go, and schema.go.
  • Code Refactoring: Common helper functions (ensureID, cloneDBObject, mergeQueryFields) and their tests were moved from the basic_operations files into new, dedicated utils.go and utils_test.go files. This improves code organization and reusability.

Affected system components

The changes are well-encapsulated within the PostgreSQL driver located at persistent/internal/driver/postgres/. While the modifications are localized to this driver, any part of the application that uses the persistent storage layer with a PostgreSQL backend can now leverage this new capability to build more scalable and resilient database architectures.

2. Architecture Visualization

The following diagram illustrates the new database connection architecture. It shows how the PostgresDriver now acts as a router, directing queries to either the primary database for writes or a read replica for reads.

graph TD
    title PostgreSQL Read/Write Connection Separation

    subgraph Application
        PostgresDriver
    end

    subgraph "Database Infrastructure"
        PrimaryDB["Primary DB (Writes)"]
        ReplicaDB["Read Replica (Reads)"]
    end

    PostgresDriver -- "writeDB Connection (INSERT, UPDATE, DELETE, etc.)" --> PrimaryDB
    PostgresDriver -- "readDB Connection (SELECT, COUNT, etc.)" --> ReplicaDB

    PrimaryDB -- "Data Replication" --> ReplicaDB

    classDef note fill:#f9f,stroke:#333,stroke-width:2px;
    
    subgraph Note
        direction LR
        A["If ReadConnectionString is not provided,<br/>the readDB connection points to the Primary DB,<br/>using the same connection as writeDB."]
    end
    class Note note
Loading

Generated by Visor - AI-powered code review

Annotations

Check notice on line 1 in AI_RESPONSE

See this annotation in the file changed.

@probelabs probelabs / Visor: overview

documentation Issue

This is an excellent pull request that introduces a crucial feature for database scalability. Here is a comprehensive analysis:

### **1. Change Impact Analysis**

#### **What this PR accomplishes**

This pull request refactors the PostgreSQL database driver to support read/write splitting. It introduces the ability to configure separate database connections for write operations (like `INSERT`, `UPDATE`, `DELETE`) and read operations (`SELECT`). This allows the application to direct write traffic to a primary database and offload read traffic to one or more replica databases.

This is a significant architectural improvement that enables:
*   **Scalability:** Read-heavy workloads can be scaled by adding more read replicas without impacting the performance of the primary write database.
*   **Performance:** Reduces the load on the primary database, leading to faster query execution for both reads and writes.
*   **High Availability:** Can be part of a high-availability strategy, allowing the application to continue serving read requests even if the primary database is temporarily unavailable.

The implementation is backward-compatible. If a separate read connection string is not provided, the driver defaults to using the primary (write) connection for all operations, ensuring existing deployments continue to work without any configuration changes.

#### **Key technical changes introduced**

*   **New Configuration Option:** A `ReadConnectionString` field has been added to the `ClientOpts` struct (`persistent/internal/types/client_options.go`), allowing users to specify a connection string for a read replica.

*   **Dual Connection Management:** The connection management logic in `persistent/internal/driver/postgres/lifecycle.go` has been fundamentally updated. The `lifeCycle` struct now holds two separate `gorm.DB` connections: `writeDB` and `readDB`. The `Connect` method establishes both connections if two different connection strings are provided, and the `Close` and `Ping` methods are updated to manage the lifecycle of both.

*   **Intelligent Query Routing:** All database operations within the PostgreSQL driver have been systematically routed to the appropriate connection:
    *   **Write Operations:** All functions that modify data or schema (e.g., `Insert`, `Update`, `Delete`, `Migrate`, `CreateIndex`) now exclusively use the `writeDB` connection. These changes are primarily in `basic_operations.go`, `indexes.go`, and `schema.go`.
    *   **Read Operations:** All functions that only retrieve data (e.g., `Query`, `Count`, `GetIndexes`, `DBTableStats`, `HasTable`) now use the `readDB` connection. These changes are found in `query.go`, `indexes.go`, and `schema.go`.

*   **Code Refactoring:** Common helper functions (`ensureID`, `cloneDBObject`, `mergeQueryFields`) and their tests were moved from the `basic_operations` files into new, dedicated `utils.go` and `utils_test.go` files. This improves code organization and reusability.

#### **Affected system components**

The changes are well-encapsulated within the **PostgreSQL driver** located at `persistent/internal/driver/postgres/`. While the modifications are localized to this driver, any part of the application that uses the `persistent` storage layer with a PostgreSQL backend can now leverage this new capability to build more scalable and resilient database architectures.

### **2. Architecture Visualization**

The following diagram illustrates the new database connection architecture. It shows how the `PostgresDriver` now acts as a router, directing queries to either the primary database for writes or a read replica for reads.

```mermaid
graph TD
    title PostgreSQL Read/Write Connection Separation

    subgraph Application
        PostgresDriver
    end

    subgraph "Database Infrastructure"
        PrimaryDB["Primary DB (Writes)"]
        ReplicaDB["Read Replica (Reads)"]
    end

    PostgresDriver -- "writeDB Connection (INSERT, UPDATE, DELETE, etc.)" --> PrimaryDB
    PostgresDriver -- "readDB Connection (SELECT, COUNT, etc.)" --> ReplicaDB

    PrimaryDB -- "Data Replication" --> ReplicaDB

    classDef note fill:#f9f,stroke:#333,stroke-width:2px;
    
    subgraph Note
        direction LR
        A["If ReadConnectionString is not provided,<br/>the readDB connection points to the Primary DB,<br/>using the same connection as writeDB."]
    end
    class Note note
```