BettyBlocks Runtime

A dynamic data management system with schema compilation, REST API, and web interface.

Overview

This application consists of three main services:

• DataCompiler: Compiles schemas and creates database tables dynamically
• Backend (DataAPI): Provides REST API for CRUD operations with filtering, sorting, and pagination
• Frontend: React/TypeScript web application for schema management and data querying

Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│    Frontend     │───▶│   DataCompiler  │───▶│     Backend     │
│   (React/TS)    │    │   (Schema API)  │    │   (Data API)    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         │                       ▼                       ▼
         │              ┌─────────────────┐    ┌─────────────────┐
         └─────────────▶│      Redis      │    │   PostgreSQL    │
                        │   (Schemas)     │    │     (Data)      │
                        └─────────────────┘    └─────────────────┘

Features

✅ Core Requirements

• Schema compilation stored in Redis
• Dynamic table creation in PostgreSQL
• REST API for data operations
• Web interface for data management

✅ Bonus Features

• Advanced filtering (text fields only, numeric fields excluded)
• Sorting on all field types
• Pagination with proper count handling
• Full CRUD operations
• Real-time data updates

✅ Senior-Level Standards

• Proper error handling with HTTP status codes
• Input validation using Pydantic models
• Type safety throughout the application
• Configuration management with environment variables
• Comprehensive logging
• Health check endpoints
• API documentation with OpenAPI/Swagger
• Docker containerization

Quick Start

Prerequisites

• Docker and Docker Compose
• Node.js 18+ (for development)
• Python 3.11+ (for development)

Running with Docker (Recommended)

# Clone and start all services
git clone <repository-url>
cd bettyblocks-runtime
docker-compose up -d

# Access the applications
# Frontend: http://localhost:3000
# DataCompiler API: http://localhost:8000/docs
# Backend API: http://localhost:8001/docs

Development Setup

# Backend services
cd services/backend
pip install -r requirements.txt
uvicorn main:app --reload --port 8001

cd services/datacompiler
pip install -r requirements.txt
uvicorn main:app --reload --port 8000

# Frontend
cd services/frontend
npm install
npm run dev  # Runs on http://localhost:5173

API Documentation

DataCompiler API (Port 8000)

Compile Schema

POST /api/compile
Content-Type: application/json

{
  "table_name": "users",
  "fields": [
    {
      "name": "id",
      "type": "integer",
      "nullable": false
    },
    {
      "name": "name",
      "type": "string",
      "nullable": false
    },
    {
      "name": "email",
      "type": "string",
      "nullable": false
    },
    {
      "name": "age",
      "type": "integer",
      "nullable": true
    }
  ]
}

List Schemas

GET /api/schemas

Get Schema

GET /api/schemas/{table_name}

Backend API (Port 8001)

Query Data

GET /api/data/{table_name}?page=0&limit=10&sort_by=name&sort_order=asc&filters={"name":"john"}

Create Record

POST /api/data/{table_name}
Content-Type: application/json

{
  "name": "John Doe",
  "email": "john@example.com",
  "age": 30
}

Update Record

PUT /api/data/{table_name}/{record_id}
Content-Type: application/json

{
  "name": "John Smith"
}

Delete Record

DELETE /api/data/{table_name}/{record_id}

Configuration

Environment variables can be set in .env files or Docker environment:

# Database
DATABASE_URL=postgresql+asyncpg://bettyblocks:bettyblocks@postgres:5432/bettyblocks

# Redis
REDIS_URL=redis://redis:6379

# Logging
LOG_LEVEL=DEBUG

# CORS
CORS_ORIGINS=["http://localhost:3000","http://localhost:5173"]

Data Types

Supported field types:

• string: Text data (supports filtering)
• integer: Whole numbers (no filtering, sorting enabled)
• number: Decimal numbers (no filtering, sorting enabled)
• boolean: True/false values
• date: Date values
• datetime: Date and time values

Filtering Rules

• Text fields: Support case-insensitive partial matching
• Numeric fields: Filtering is disabled (as per requirements)
• Other fields: No filtering implemented

Testing

# Run frontend tests
cd services/frontend
npm test

# Run E2E tests
npm run test:e2e

Architecture Decisions

1. Microservices: Separated DataCompiler and Backend for clear responsibility boundaries
2. Redis for Schema Storage: Fast access to compiled schemas
3. Dynamic SQLAlchemy Models: Runtime table creation without migrations
4. React Query: Efficient state management and caching in frontend
5. Pydantic Models: Type safety and validation in Python services
6. Material-UI: Consistent and professional UI components

Security Considerations

• Input validation on all endpoints
• SQL injection prevention through SQLAlchemy ORM
• CORS configuration for cross-origin requests
• Error message sanitization
• Request rate limiting (recommended for production)

Performance Optimizations

• Redis caching for schema lookups
• Database connection pooling
• React Query for frontend caching
• Pagination to limit data transfer
• Efficient SQL query generation

Monitoring and Health Checks

Health check endpoints available:

• DataCompiler: GET /health
• Backend: GET /health

Contributing

1. Follow the established code patterns
2. Add proper type annotations
3. Include error handling
4. Update tests for new features
5. Document API changes

License

MIT License