[Proposal]: Architectural Refactoring & Directory Restructuring

[SB2318]

Problem Statement

The current backend codebase relies on a monolithic index.js file (~1,500+ lines) that combines multiple distinct responsibilities: Express app configuration, database bootstrap, Kafka producer/consumer initialization, inline route handlers, WebSocket logic, and HTTP server listener. Additionally, server.js is marked as a deprecated file, and routing endpoints across routes/ lack a standardized versioning convention (v1/v2).

This monolithic pattern causes several key maintenance & reliability issues:

1. Testing Friction: Importing index.js in test suites immediately triggers database connections, Kafka workers, and open HTTP listener handles.
2. High Code Complexity: Inline endpoints and business logic inside index.js blur the boundaries between presentation, routing, and domain services.
3. API Versioning Inconsistency: Route files are flatly stored in routes/, making it difficult to manage breaking changes or introduce /api/v1 and /api/v2 endpoints cleanly.

---

🎯 Proposed Solution & Architectural Blueprint

1. Separation of Responsibilities (app.js vs server.js)

Split index.js into two clear entry points:

• app.js (Express App Factory / Builder)

  • Configures global middleware (CORS, Helmet, Rate Limiter, Body Parsers, Compression).
  • Mounts API router (/api/v1, /api/v2, /api/health).
  • Registers global error handling middleware (errorHandler).
  • Exports the app instance without starting HTTP listener or connecting to DB/Kafka.

• server.js (Infrastructure Bootstrapper & Entry Point)

  • Loads environment variables & secret validation (config/secrets.js).
  • Initializes database connections (dbConnect), messaging queues (Kafka producers/consumers), and background workers.
  • Starts HTTP and Socket.io server listener on PORT.
  • Implements graceful shutdown logic (SIGTERM, SIGINT).

---

2. Directory Structure & API Routing Versioning (v1 / v2)

Refactor the routes/ and controllers/ directories to support clean versioning:

src/ (or root)
├── app.js                   # Express application setup & middleware stack
├── server.js                # Main entry point & infrastructure bootstrapper
├── config/                  # Environment & infrastructure configuration
│   ├── database.js
│   ├── firebase.js
│   ├── kafka.js
│   └── secrets.js           # Secret loader & validator
├── routes/                  # Centralized API Routing
│   ├── index.js             # Root router aggregator (/api/v1, /api/v2)
│   ├── v1/                  # Version 1 API Routes
│   │   ├── index.js         # Aggregator for v1 routes
│   │   ├── authRoutes.js
│   │   ├── userRoutes.js
│   │   ├── articleRoutes.js
│   │   ├── podcastRoutes.js
│   │   └── wellnessRoutes.js
│   └── v2/                  # Version 2 API Routes
│       ├── index.js         # Aggregator for v2 routes
│       ├── userRoutes.js
│       └── articleRoutes.js
├── controllers/             # Controller Presentation Layer
│   ├── v1/                  # Controllers for v1 routes
│   ├── v2/                  # Controllers for v2 routes
│   └── admin/               # Admin specific controllers
├── services/                # Business & Domain Logic Layer
│   ├── mqueue/              # Kafka producers and consumer workers
│   └── security/            # Token management & authentication logic
├── middleware/              # Express Middlewares
│   ├── errorHandler.js
│   ├── ratelimit.js
│   └── validator.js
├── loaders/                 # Infrastructure loaders (DB, Kafka, Socket)
│   ├── dbLoader.js
│   ├── kafkaLoader.js
│   └── socketLoader.js
└── utils/                   # Helpers & Redaction utilities

---

3. Modular Router Aggregation Pattern

Instead of individually requiring 20+ route files inside index.js, use a nested router aggregator pattern:

routes/index.js

const express = require('express');
const v1Router = require('./v1');
const v2Router = require('./v2');

const router = express.Router();

router.use('/v1', v1Router);
router.use('/v2', v2Router);

module.exports = router;

routes/v1/index.js

const express = require('express');
const userRoutes = require('./userRoutes');
const articleRoutes = require('./articleRoutes');
const podcastRoutes = require('./podcastRoutes');

const v1Router = express.Router();

v1Router.use('/users', userRoutes);
v1Router.use('/articles', articleRoutes);
v1Router.use('/podcasts', podcastRoutes);

module.exports = v1Router;

Mounting in app.js

const apiRouter = require('./routes');
app.use('/api', apiRouter); // Routes accessible via /api/v1/... and /api/v2/...

---

4. Background Loader Modularization (loaders/)

Move Kafka consumers, producers, and database connections out of index.js into dedicated loader modules:

• loaders/dbLoader.js: Database connection lifecycle.
• loaders/kafkaLoader.js: Kafka topics initialization and consumer listeners setup.
• loaders/socketLoader.js: Socket.io server connection and event handlers.

---

5. Graceful Shutdown Implementation

In server.js, handle process termination signals cleanly:

const gracefulShutdown = async (signal) => {
  console.log(`Received ${signal}. Initiating graceful shutdown...`);
  
  server.close(() => {
    console.log('HTTP server closed.');
  });

  try {
    await db.disconnect();
    await kafkaProducer.disconnect();
    console.log('Database and messaging queue connections closed.');
    process.exit(0);
  } catch (err) {
    console.error('Error during graceful shutdown:', err);
    process.exit(1);
  }
};

process.on('SIGTERM', () => gracefulShutdown('SIGTERM'));
process.on('SIGINT', () => gracefulShutdown('SIGINT'));

---

Action Items / Tasks

• Create app.js and extract Express middleware setup, route mounting, and error handler from index.js.
• Refactor server.js to serve as the main server bootstrap script (DB, Kafka, HTTP listener).
• Remove inline route/socket handlers from index.js and move them into appropriate controller/service files.
• Restructure routes/ into routes/v1/ and routes/v2/ with central router index files.
• Reorganize controllers/ into v1/ and v2/ subdirectories to align with API routes.
• Create loaders/ for Database, Kafka, and Socket.io setup.
• Implement graceful process shutdown handling (SIGTERM/SIGINT).
• Update Jest unit and integration tests to import app.js instead of starting a live HTTP server.

---

Benefits

• Clean Architecture & Separation of Concerns: Decouples application logic from server startup and infrastructure listeners.
• Improved Testability: Test suites can import app.js and use supertest without open handles or port binding conflicts.
• Scalable API Versioning: Clear separation between /api/v1 and /api/v2 endpoints.
• Maintainable Codebase: Solves the 1,500+ line monolithic index.js anti-pattern.

[github-actions]

Thank you for creating this issue! We'll look into it as soon as possible. Your contributions are highly appreciated! 😊

---

🤖 Assignment Guidelines

To get assigned to this issue, simply leave a comment with or .
The AI bot will automatically assign you if you meet the following eligibility criteria:

1. You do not currently have any other active assigned issues in this repository.
2. If you were previously assigned an issue, you must have submitted a Pull Request for it before requesting a new one.

[github-actions]

This issue has been triaged as a backend task (level:advanced).

Expected Effect & Scope: The issue provides a clear problem statement and a detailed architectural proposal for backend refactoring, making it valid and in scope. It's classified as a backend issue due to its focus on APIs, server structure, and infrastructure. The proposed architectural changes and large-scale refactoring warrant an 'advanced' difficulty level and require escalation to a maintainer.

This requires backend investigation and has been routed to @SB2318 for review and prioritization.