From c2623c3cf6ecd7551a94f72050a9f06aae4b3788 Mon Sep 17 00:00:00 2001 From: Nagy Daniel Date: Mon, 27 Oct 2025 11:18:32 +0100 Subject: [PATCH 1/7] OBB WIP. --- OBB_IMPLEMENTATION.md | 167 ++++++ docker/Dockerfile | 27 +- docker/build.sh | 29 ++ docker/run.sh | 33 ++ src/ai/RoboflowAPIObjectDetector.ts | 2 +- src/data/ExportFormatData.ts | 8 +- src/data/enums/LabelType.ts | 3 +- src/data/info/LabelToolkitData.ts | 7 + src/logic/actions/EditorActions.ts | 4 + src/logic/actions/LabelActions.ts | 19 +- src/logic/export/OBBLabelsExporter.ts | 82 +++ src/logic/render/OBBRenderEngine.ts | 474 ++++++++++++++++++ src/store/labels/types.ts | 8 + src/utils/ImageDataUtil.ts | 2 + src/utils/LabelUtil.ts | 14 +- .../LabelsToolkit/LabelsToolkit.tsx | 11 +- .../OBBLabelsList/OBBLabelsList.scss | 12 + .../OBBLabelsList/OBBLabelsList.tsx | 147 ++++++ .../ExportLabelsPopup/ExportLabelPopup.tsx | 4 + 19 files changed, 1041 insertions(+), 12 deletions(-) create mode 100644 OBB_IMPLEMENTATION.md create mode 100755 docker/build.sh create mode 100755 docker/run.sh create mode 100644 src/logic/export/OBBLabelsExporter.ts create mode 100644 src/logic/render/OBBRenderEngine.ts create mode 100644 src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.scss create mode 100644 src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.tsx diff --git a/OBB_IMPLEMENTATION.md b/OBB_IMPLEMENTATION.md new file mode 100644 index 000000000..fb3b065ce --- /dev/null +++ b/OBB_IMPLEMENTATION.md @@ -0,0 +1,167 @@ +# YOLO OBB (Oriented Bounding Box) Implementation + +## Overview +This document describes the implementation of YOLO Oriented Bounding Box (OBB) support in the make-sense image labeling tool. + +## What is YOLO OBB? +YOLO OBB (Oriented Bounding Box) is a format used by Ultralytics YOLO models for annotating rotated objects. Unlike regular axis-aligned bounding boxes, OBBs can be rotated to better fit objects at any angle, making them particularly useful for aerial/satellite imagery and other scenarios where objects are not always upright. + +### YOLO OBB Format Specification +According to the Ultralytics documentation, the YOLO OBB format is: +``` +class_index x1 y1 x2 y2 x3 y3 x4 y4 +``` + +Where: +- `class_index`: The class ID of the object +- `x1 y1`, `x2 y2`, `x3 y3`, `x4 y4`: Four corner points of the oriented bounding box +- All coordinates are normalized to [0, 1] range (divided by image width/height) + +## Implementation Details + +### 1. Core Type System +**Files Modified:** +- `src/data/enums/LabelType.ts` - Added `OBB = 'OBB'` to LabelType enum +- `src/store/labels/types.ts` - Added `LabelOBB` type and `labelOBBs` array to `ImageData` + +The `LabelOBB` type includes: +```typescript +export type LabelOBB = Annotation & { + vertices: IPoint[]; + isCreatedByAI: boolean; + status: LabelStatus; + suggestedLabel: string; +} +``` + +### 2. Label Toolkit Integration +**Files Modified:** +- `src/data/info/LabelToolkitData.ts` - Added OBB option to the label toolkit menu + +Users can now select "OBB" as a labeling mode alongside Rect, Point, Line, and Polygon. + +### 3. Rendering Engine +**Files Created:** +- `src/logic/render/OBBRenderEngine.ts` - Complete rendering engine for drawing and editing OBBs + +**Features:** +- Interactive drawing: Users draw OBBs by clicking 3 points: + 1. First point: Start of first edge + 2. Second point: End of first edge (defines one side of the box) + 3. Third point: Defines the width/height by projecting perpendicular to the first edge +- Vertex manipulation: Users can drag any of the 4 corners to reshape the OBB +- Visual feedback: Active and highlighted OBBs show anchors at corners +- Semi-transparent fill for better visibility + +**Files Modified:** +- `src/logic/actions/EditorActions.ts` - Integrated OBBRenderEngine into the editor + +### 4. Export Functionality +**Files Created:** +- `src/logic/export/OBBLabelsExporter.ts` - Exports OBB annotations in YOLO OBB format + +**Export Format:** +- Creates a .zip file containing .txt files (one per image) +- Each line in the .txt file represents one OBB annotation +- Format: `class_index x1 y1 x2 y2 x3 y3 x4 y4` +- All coordinates normalized to [0, 1] + +**Files Modified:** +- `src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx` - Added OBB export option +- `src/data/ExportFormatData.ts` - Added YOLO OBB format description + +### 5. Label Management +**Files Created:** +- `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.tsx` - List view for OBB labels +- `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.scss` - Styles for OBB list + +**Files Modified:** +- `src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx` - Added OBB tab and list +- `src/logic/actions/LabelActions.ts` - Added `deleteOBBLabelById()` and OBB support in visibility toggle +- `src/utils/LabelUtil.ts` - Added `createLabelOBB()` method +- `src/utils/ImageDataUtil.ts` - Initialize `labelOBBs: []` in ImageData creation + +## How to Use + +### Drawing OBBs: +1. Load your images into the tool +2. Create or select label classes +3. Select "OBB" from the label toolkit on the right sidebar +4. Click to place the first corner of your box +5. Click to place the second corner (defines one edge) +6. Click to define the width of the box (the tool auto-completes the rectangle) + +### Editing OBBs: +1. Click on an existing OBB to select it +2. Drag any of the 4 corner anchors to reshape the box +3. The OBB remains a quadrilateral but can be freely adjusted + +### Exporting: +1. Click "Export Labels" (or use the export action) +2. Select "OBB" as the label type +3. Choose "YOLO" as the export format +4. Download the .zip file containing your annotations + +### File Structure: +``` +annotations.zip +├── image1.txt +├── image2.txt +└── image3.txt +``` + +Each .txt file contains lines like: +``` +0 0.512 0.345 0.678 0.389 0.634 0.556 0.468 0.512 +1 0.234 0.123 0.345 0.234 0.298 0.401 0.187 0.290 +``` + +## Training with Ultralytics YOLO + +Once you have your OBB annotations, you can use them to train YOLO models: + +```python +from ultralytics import YOLO + +# Load a model +model = YOLO('yolo11n-obb.yaml') + +# Train the model +results = model.train(data='your_dataset.yaml', epochs=100, imgsz=640) +``` + +Your dataset.yaml should specify: +```yaml +path: /path/to/dataset +train: images/train +val: images/val + +names: + 0: class_name_1 + 1: class_name_2 +``` + +## Files Changed Summary + +### New Files Created (7): +1. `src/logic/render/OBBRenderEngine.ts` +2. `src/logic/export/OBBLabelsExporter.ts` +3. `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.tsx` +4. `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.scss` + +### Files Modified (11): +1. `src/data/enums/LabelType.ts` +2. `src/store/labels/types.ts` +3. `src/data/info/LabelToolkitData.ts` +4. `src/utils/LabelUtil.ts` +5. `src/logic/actions/EditorActions.ts` +6. `src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx` +7. `src/data/ExportFormatData.ts` +8. `src/utils/ImageDataUtil.ts` +9. `src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx` +10. `src/logic/actions/LabelActions.ts` + +## References +- [Ultralytics OBB Documentation](https://docs.ultralytics.com/datasets/obb/) +- YOLO OBB format uses 4 corner points normalized to [0, 1] +- Internally, YOLO processes in xywhr format (center x, center y, width, height, rotation) diff --git a/docker/Dockerfile b/docker/Dockerfile index 69029205f..283aaef20 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,12 +1,27 @@ FROM node:16.16.0 -RUN apt-get update && apt-get -y install git && rm -rf /var/lib/apt/lists/* +# Accept build arguments for user creation +ARG USRNM=developer +ARG USRUID=1000 +ARG USRGID=1000 -COPY ./ /make-sense +# Create group and user with specified UID/GID +RUN groupadd --gid ${USRGID} ${USRNM} || true && \ + useradd --uid ${USRUID} --gid ${USRGID} -m -s /bin/bash ${USRNM} || true -RUN cd /make-sense && \ - npm install +# Create app directory +RUN mkdir -p /app && chown ${USRUID}:${USRGID} /app -WORKDIR /make-sense +# Copy package files and install dependencies as root +COPY package.json package-lock.json* /app/ +RUN cd /app && npm install && chown -R ${USRUID}:${USRGID} /app/node_modules -ENTRYPOINT ["npm", "run", "dev"] +# Switch to non-root user +USER ${USRNM} + +WORKDIR /app + +# Expose development server port +EXPOSE 3000 + +CMD ["/bin/bash"] diff --git a/docker/build.sh b/docker/build.sh new file mode 100755 index 000000000..f6488e34f --- /dev/null +++ b/docker/build.sh @@ -0,0 +1,29 @@ +#!/bin/bash + +# Determine script location +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# Navigate to project root (parent directory) +cd "$SCRIPT_DIR/.." + +echo "Building make-sense Docker image..." +echo "Current directory: $(pwd)" +echo "User: $(whoami) (UID: $(id -u), GID: $(id -g))" + +# Build the Docker image with user arguments +docker build \ + --build-arg USRNM=$(whoami) \ + --build-arg USRUID=$(id -u) \ + --build-arg USRGID=$(id -g) \ + -t make-sense:dev \ + -f docker/Dockerfile \ + . + +if [ $? -eq 0 ]; then + echo "✅ Build successful! Image: make-sense:dev" + echo "" + echo "To run the container, use: ./docker/run.sh" +else + echo "❌ Build failed!" + exit 1 +fi diff --git a/docker/run.sh b/docker/run.sh new file mode 100755 index 000000000..1092a7a67 --- /dev/null +++ b/docker/run.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +# Determine script location +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# Navigate to project root +cd "$SCRIPT_DIR/.." + +echo "Starting make-sense development container..." +echo "Project directory: $(pwd)" +echo "Container will be accessible at: http://localhost:9500" +echo "" +echo "Volume mount: $(pwd) -> /app" +echo "Running as: $(whoami) (UID: $(id -u), GID: $(id -g))" +echo "" +echo "Inside the container, run:" +echo " npm run dev - Start development server" +echo " npm run build - Build for production" +echo " npm test - Run tests" +echo "" + +# Run the container +docker run -it \ + --name make-sense-dev \ + --rm \ + -p 9500:3000 \ + -v "$(pwd)":/app \ + -w /app \ + --user $(id -u):$(id -g) \ + make-sense:dev + +# Note: The container will be removed when stopped due to --rm flag +# If you want it to persist, remove the --rm flag above diff --git a/src/ai/RoboflowAPIObjectDetector.ts b/src/ai/RoboflowAPIObjectDetector.ts index 2378b392a..301efdeca 100644 --- a/src/ai/RoboflowAPIObjectDetector.ts +++ b/src/ai/RoboflowAPIObjectDetector.ts @@ -59,7 +59,7 @@ export class RoboflowAPIObjectDetector { FileUtil.loadImageBase64(imageData.fileData).then((data) => { axios({ method: 'POST', - url: 'https://detect.roboflow.com/' + roboflowAPIDetails.model, + url: '' + roboflowAPIDetails.model, params: { api_key: roboflowAPIDetails.key }, diff --git a/src/data/ExportFormatData.ts b/src/data/ExportFormatData.ts index 540536a58..d88764f62 100644 --- a/src/data/ExportFormatData.ts +++ b/src/data/ExportFormatData.ts @@ -34,13 +34,19 @@ export const ExportFormatData: ExportFormatDataMap = { [LabelType.POLYGON]: [ { type: AnnotationFormatType.VGG, - label: 'Single file in VGG JSON format.' + label: 'A .zip package containing files in VGG JSON format.' }, { type: AnnotationFormatType.COCO, label: 'Single file in COCO JSON format.' } ], + [LabelType.OBB]: [ + { + type: AnnotationFormatType.YOLO, + label: 'A .zip package containing files in YOLO OBB format.' + } + ], [LabelType.IMAGE_RECOGNITION]: [ { type: AnnotationFormatType.CSV, diff --git a/src/data/enums/LabelType.ts b/src/data/enums/LabelType.ts index 87be4c62a..b4b9ed1e8 100644 --- a/src/data/enums/LabelType.ts +++ b/src/data/enums/LabelType.ts @@ -3,5 +3,6 @@ export enum LabelType { POINT = 'POINT', RECT = 'RECT', POLYGON = 'POLYGON', - LINE = 'LINE' + LINE = 'LINE', + OBB = 'OBB' } \ No newline at end of file diff --git a/src/data/info/LabelToolkitData.ts b/src/data/info/LabelToolkitData.ts index 9b886503e..2dd01d822 100644 --- a/src/data/info/LabelToolkitData.ts +++ b/src/data/info/LabelToolkitData.ts @@ -45,4 +45,11 @@ export const LabelToolkitData: ILabelToolkit[] = [ imageAlt: 'polygon', projectType: ProjectType.OBJECT_DETECTION, }, + { + labelType: LabelType.OBB, + headerText: 'OBB', + imageSrc: 'ico/polygon.png', + imageAlt: 'obb', + projectType: ProjectType.OBJECT_DETECTION, + }, ]; \ No newline at end of file diff --git a/src/logic/actions/EditorActions.ts b/src/logic/actions/EditorActions.ts index ab71e34e1..b359c8c82 100644 --- a/src/logic/actions/EditorActions.ts +++ b/src/logic/actions/EditorActions.ts @@ -3,6 +3,7 @@ import {EditorModel} from "../../staticModels/EditorModel"; import {RectRenderEngine} from "../render/RectRenderEngine"; import {PointRenderEngine} from "../render/PointRenderEngine"; import {PolygonRenderEngine} from "../render/PolygonRenderEngine"; +import {OBBRenderEngine} from "../render/OBBRenderEngine"; import {IRect} from "../../interfaces/IRect"; import {RectUtil} from "../../utils/RectUtil"; import {EditorData} from "../../data/EditorData"; @@ -41,6 +42,9 @@ export class EditorActions { case LabelType.POLYGON: EditorModel.supportRenderingEngine = new PolygonRenderEngine(EditorModel.canvas); break; + case LabelType.OBB: + EditorModel.supportRenderingEngine = new OBBRenderEngine(EditorModel.canvas); + break; default: EditorModel.supportRenderingEngine = null; break; diff --git a/src/logic/actions/LabelActions.ts b/src/logic/actions/LabelActions.ts index d47f2e7dc..f59d37208 100644 --- a/src/logic/actions/LabelActions.ts +++ b/src/logic/actions/LabelActions.ts @@ -1,5 +1,5 @@ import {LabelsSelector} from '../../store/selectors/LabelsSelector'; -import {ImageData, LabelLine, LabelName, LabelPoint, LabelPolygon, LabelRect} from '../../store/labels/types'; +import {ImageData, LabelLine, LabelName, LabelPoint, LabelPolygon, LabelRect, LabelOBB} from '../../store/labels/types'; import {filter} from 'lodash'; import {store} from '../../index'; import {updateImageData, updateImageDataById} from '../../store/labels/actionCreators'; @@ -24,6 +24,9 @@ export class LabelActions { case LabelType.POLYGON: LabelActions.deletePolygonLabelById(imageId, labelId); break; + case LabelType.OBB: + LabelActions.deleteOBBLabelById(imageId, labelId); + break; } } @@ -71,6 +74,17 @@ export class LabelActions { store.dispatch(updateImageDataById(imageData.id, newImageData)); } + public static deleteOBBLabelById(imageId: string, labelOBBId: string) { + const imageData: ImageData = LabelsSelector.getImageDataById(imageId); + const newImageData = { + ...imageData, + labelOBBs: filter(imageData.labelOBBs || [], (currentLabel: LabelOBB) => { + return currentLabel.id !== labelOBBId; + }) + }; + store.dispatch(updateImageDataById(imageData.id, newImageData)); + } + public static toggleLabelVisibilityById(imageId: string, labelId: string) { const imageData: ImageData = LabelsSelector.getImageDataById(imageId); const newImageData = { @@ -87,6 +101,9 @@ export class LabelActions { labelLines: imageData.labelLines.map((labelLine: LabelLine) => { return labelLine.id === labelId ? LabelUtil.toggleAnnotationVisibility(labelLine) : labelLine }), + labelOBBs: (imageData.labelOBBs || []).map((labelOBB: LabelOBB) => { + return labelOBB.id === labelId ? LabelUtil.toggleAnnotationVisibility(labelOBB) : labelOBB + }), }; store.dispatch(updateImageDataById(imageData.id, newImageData)); } diff --git a/src/logic/export/OBBLabelsExporter.ts b/src/logic/export/OBBLabelsExporter.ts new file mode 100644 index 000000000..7509f22b5 --- /dev/null +++ b/src/logic/export/OBBLabelsExporter.ts @@ -0,0 +1,82 @@ +import {AnnotationFormatType} from '../../data/enums/AnnotationFormatType'; +import {ImageData, LabelName, LabelOBB} from '../../store/labels/types'; +import {ImageRepository} from '../imageRepository/ImageRepository'; +import JSZip from 'jszip'; +import { saveAs } from 'file-saver'; +import {LabelsSelector} from '../../store/selectors/LabelsSelector'; +import {ExporterUtil} from '../../utils/ExporterUtil'; +import {findIndex} from 'lodash'; +import {ISize} from '../../interfaces/ISize'; +import {NumberUtil} from '../../utils/NumberUtil'; + +export class OBBLabelsExporter { + public static export(exportFormatType: AnnotationFormatType): void { + switch (exportFormatType) { + case AnnotationFormatType.YOLO: + OBBLabelsExporter.exportAsYOLO(); + break; + default: + return; + } + } + + private static exportAsYOLO(): void { + const zip = new JSZip(); + LabelsSelector.getImagesData() + .forEach((imageData: ImageData) => { + const fileContent: string = OBBLabelsExporter.wrapOBBLabelsIntoYOLO(imageData); + if (fileContent) { + const fileName : string = imageData.fileData.name.replace(/\.[^/.]+$/, '.txt'); + try { + zip.file(fileName, fileContent); + } catch (error) { + throw new Error(error as string); + } + } + }); + + try { + zip.generateAsync({type:'blob'}) + .then((content: Blob) => { + saveAs(content, `${ExporterUtil.getExportFileName()}.zip`); + }); + } catch (error) { + throw new Error(error as string); + } + } + + public static wrapOBBLabelIntoYOLO(labelOBB: LabelOBB, labelNames: LabelName[], imageSize: ISize): string { + const snapAndFix = (value: number) => NumberUtil.snapValueToRange(value, 0, 1).toFixed(6); + const classIdx: string = findIndex(labelNames, {id: labelOBB.labelId}).toString(); + + // YOLO OBB format: class_index x1 y1 x2 y2 x3 y3 x4 y4 + // All coordinates normalized to [0, 1] + const normalizedCoords: number[] = []; + + labelOBB.vertices.forEach((vertex) => { + normalizedCoords.push(vertex.x / imageSize.width); + normalizedCoords.push(vertex.y / imageSize.height); + }); + + // Snap all coordinates to [0, 1] range + const processedCoords = normalizedCoords.map((value: number) => snapAndFix(value)); + + return [classIdx, ...processedCoords].join(' '); + } + + private static wrapOBBLabelsIntoYOLO(imageData: ImageData): string { + if (!imageData.labelOBBs || imageData.labelOBBs.length === 0 || !imageData.loadStatus) + return null; + + const labelNames: LabelName[] = LabelsSelector.getLabelNames(); + const image: HTMLImageElement = ImageRepository.getById(imageData.id); + const imageSize: ISize = {width: image.width, height: image.height}; + + const labelOBBsString: string[] = imageData.labelOBBs + .filter((labelOBB: LabelOBB) => labelOBB.labelId !== null) + .map((labelOBB: LabelOBB) => { + return OBBLabelsExporter.wrapOBBLabelIntoYOLO(labelOBB, labelNames, imageSize); + }); + return labelOBBsString.join('\n'); + } +} diff --git a/src/logic/render/OBBRenderEngine.ts b/src/logic/render/OBBRenderEngine.ts new file mode 100644 index 000000000..eaa675c0b --- /dev/null +++ b/src/logic/render/OBBRenderEngine.ts @@ -0,0 +1,474 @@ +import {store} from '../../index'; +import {RectUtil} from '../../utils/RectUtil'; +import {updateCustomCursorStyle} from '../../store/general/actionCreators'; +import {CustomCursorStyle} from '../../data/enums/CustomCursorStyle'; +import {EditorData} from '../../data/EditorData'; +import {BaseRenderEngine} from './BaseRenderEngine'; +import {RenderEngineSettings} from '../../settings/RenderEngineSettings'; +import {IPoint} from '../../interfaces/IPoint'; +import {DrawUtil} from '../../utils/DrawUtil'; +import {IRect} from '../../interfaces/IRect'; +import {ImageData, LabelOBB} from '../../store/labels/types'; +import {LabelsSelector} from '../../store/selectors/LabelsSelector'; +import { + updateActiveLabelId, + updateFirstLabelCreatedFlag, + updateHighlightedLabelId, + updateImageDataById +} from '../../store/labels/actionCreators'; +import {MouseEventUtil} from '../../utils/MouseEventUtil'; +import {EventType} from '../../data/enums/EventType'; +import {RenderEngineUtil} from '../../utils/RenderEngineUtil'; +import {LabelType} from '../../data/enums/LabelType'; +import {EditorActions} from '../actions/EditorActions'; +import {GeneralSelector} from '../../store/selectors/GeneralSelector'; +import {LabelUtil} from '../../utils/LabelUtil'; +import {PointUtil} from '../../utils/PointUtil'; +import {LabelStatus} from '../../data/enums/LabelStatus'; + +export class OBBRenderEngine extends BaseRenderEngine { + + // ================================================================================================================= + // STATE + // ================================================================================================================= + + private activePath: IPoint[] = []; + private resizeAnchorIndex: number = null; + + public constructor(canvas: HTMLCanvasElement) { + super(canvas); + this.labelType = LabelType.OBB; + } + + // ================================================================================================================= + // EVENT HANDLERS + // ================================================================================================================= + + public update(data: EditorData): void { + if (!!data.event) { + switch (MouseEventUtil.getEventType(data.event)) { + case EventType.MOUSE_MOVE: + this.mouseMoveHandler(data); + break; + case EventType.MOUSE_UP: + this.mouseUpHandler(data); + break; + case EventType.MOUSE_DOWN: + this.mouseDownHandler(data); + break; + default: + break; + } + } + } + + public mouseDownHandler(data: EditorData): void { + const isMouseOverCanvas: boolean = RenderEngineUtil.isMouseOverCanvas(data); + if (isMouseOverCanvas) { + if (this.isCreationInProgress()) { + this.updateActivelyCreatedLabel(data); + } else { + const obbUnderMouse: LabelOBB = this.getOBBUnderMouse(data); + if (!!obbUnderMouse) { + const anchorIndex: number = obbUnderMouse.vertices.reduce( + (indexUnderMouse: number, anchor: IPoint, index: number) => { + if (indexUnderMouse === null) { + const anchorOnCanvas: IPoint = RenderEngineUtil.transferPointFromImageToViewPortContent(anchor, data); + if (this.isMouseOverAnchor(data.mousePositionOnViewPortContent, anchorOnCanvas)) { + return index; + } + } + return indexUnderMouse; + }, null); + + if (anchorIndex !== null) { + this.startExistingLabelResize(data, obbUnderMouse.id, anchorIndex); + } else { + store.dispatch(updateActiveLabelId(obbUnderMouse.id)); + } + } else { + this.updateActivelyCreatedLabel(data); + } + } + } + } + + public mouseUpHandler(data: EditorData): void { + if (this.isResizeInProgress()) + this.endExistingLabelResize(data); + } + + public mouseMoveHandler(data: EditorData): void { + if (!!data.viewPortContentImageRect && !!data.mousePositionOnViewPortContent) { + const isOverImage: boolean = RenderEngineUtil.isMouseOverImage(data); + if (isOverImage && !this.isCreationInProgress()) { + const labelOBB: LabelOBB = this.getOBBUnderMouse(data); + if (!!labelOBB && !this.isResizeInProgress()) { + if (LabelsSelector.getHighlightedLabelId() !== labelOBB.id) { + store.dispatch(updateHighlightedLabelId(labelOBB.id)) + } + } else { + if (LabelsSelector.getHighlightedLabelId() !== null) { + store.dispatch(updateHighlightedLabelId(null)); + } + } + } + } + } + + // ================================================================================================================= + // RENDERING + // ================================================================================================================= + + public render(data: EditorData) { + const activeLabelId: string = LabelsSelector.getActiveLabelId(); + const imageData: ImageData = LabelsSelector.getActiveImageData(); + if (imageData) { + imageData.labelOBBs?.forEach((labelOBB: LabelOBB) => { + if (labelOBB.isVisible) { + if (labelOBB.status === LabelStatus.ACCEPTED && labelOBB.id === activeLabelId) { + this.drawActiveOBB(labelOBB, data) + } else { + this.drawInactiveOBB(labelOBB, data); + } + } + }); + this.drawCurrentlyCreatedOBB(data); + this.updateCursorStyle(data); + } + } + + private drawCurrentlyCreatedOBB(data: EditorData) { + if (this.activePath.length > 0) { + const mousePositionSnapped: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + + let pathToDraw: IPoint[] = [...this.activePath]; + + if (this.activePath.length === 1) { + // Drawing first edge + pathToDraw.push(mousePositionSnapped); + } else if (this.activePath.length === 2) { + // Complete the rectangle + const p1 = this.activePath[0]; + const p2 = this.activePath[1]; + + // Calculate perpendicular vector + const edge = PointUtil.subtract(p2, p1); + const perpendicular = { x: -edge.y, y: edge.x }; + + // Project mouse position onto perpendicular direction + const toMouse = PointUtil.subtract(mousePositionSnapped, p1); + const projection = (toMouse.x * perpendicular.x + toMouse.y * perpendicular.y) / + (perpendicular.x * perpendicular.x + perpendicular.y * perpendicular.y); + + const offset = { x: perpendicular.x * projection, y: perpendicular.y * projection }; + const p3 = PointUtil.add(p2, offset); + const p4 = PointUtil.add(p1, offset); + + pathToDraw = [p1, p2, p3, p4]; + } + + const lineColor: string = BaseRenderEngine.resolveLabelLineColor(null, true); + DrawUtil.drawPolygon(this.canvas, pathToDraw, lineColor, RenderEngineSettings.LINE_THICKNESS); + + // Draw anchors + pathToDraw.forEach((point: IPoint) => { + DrawUtil.drawCircleWithFill( + this.canvas, + point, + RenderEngineSettings.anchorSize.width / 2, + BaseRenderEngine.resolveLabelAnchorColor(true) + ); + }); + } + } + + private drawInactiveOBB(labelOBB: LabelOBB, data: EditorData) { + const pathOnCanvas: IPoint[] = RenderEngineUtil.transferPolygonFromImageToViewPortContent(labelOBB.vertices, data); + const highlightedLabelId: string = LabelsSelector.getHighlightedLabelId(); + const displayAsActive: boolean = labelOBB.status === LabelStatus.ACCEPTED && labelOBB.id === highlightedLabelId; + const lineColor: string = BaseRenderEngine.resolveLabelLineColor(labelOBB.labelId, displayAsActive); + const anchorColor: string = BaseRenderEngine.resolveLabelAnchorColor(displayAsActive); + this.renderOBB(pathOnCanvas, displayAsActive, lineColor, anchorColor); + } + + private drawActiveOBB(labelOBB: LabelOBB, data: EditorData) { + let pathOnCanvas: IPoint[] = RenderEngineUtil.transferPolygonFromImageToViewPortContent(labelOBB.vertices, data); + + if (this.isResizeInProgress()) { + const snappedMousePosition: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + pathOnCanvas = this.calculateRectangleCorners(pathOnCanvas, this.resizeAnchorIndex, snappedMousePosition); + } + + const lineColor: string = BaseRenderEngine.resolveLabelLineColor(labelOBB.labelId, true); + const anchorColor: string = BaseRenderEngine.resolveLabelAnchorColor(true); + this.renderOBB(pathOnCanvas, true, lineColor, anchorColor); + } + + private renderOBB(pathOnCanvas: IPoint[], isActive: boolean, lineColor: string, anchorColor: string) { + DrawUtil.drawPolygonWithFill(this.canvas, pathOnCanvas, DrawUtil.hexToRGB(lineColor, 0.2)); + DrawUtil.drawPolygon(this.canvas, pathOnCanvas, lineColor, RenderEngineSettings.LINE_THICKNESS); + + if (isActive) { + pathOnCanvas.forEach((point: IPoint) => { + DrawUtil.drawCircleWithFill(this.canvas, point, RenderEngineSettings.anchorSize.width / 2, anchorColor); + }); + } + } + + private updateCursorStyle(data: EditorData) { + if (!!this.canvas && !!data.mousePositionOnViewPortContent && !GeneralSelector.getImageDragModeStatus()) { + const obbUnderMouse: LabelOBB = this.getOBBUnderMouse(data); + if ((!!obbUnderMouse && obbUnderMouse.status === LabelStatus.ACCEPTED) || this.isInProgress()) { + store.dispatch(updateCustomCursorStyle(CustomCursorStyle.MOVE)); + this.canvas.style.cursor = 'none'; + return; + } + else if (RenderEngineUtil.isMouseOverCanvas(data)) { + RenderEngineUtil.wrapDefaultCursorStyleInCancel(data); + this.canvas.style.cursor = 'none'; + } else { + this.canvas.style.cursor = 'default'; + } + } + } + + // ================================================================================================================= + // HELPERS + // ================================================================================================================= + + public isInProgress(): boolean { + return this.isCreationInProgress() || this.isResizeInProgress(); + } + + private isCreationInProgress(): boolean { + return this.activePath.length > 0 && this.activePath.length < 3; + } + + private isResizeInProgress(): boolean { + return this.resizeAnchorIndex !== null; + } + + private updateActivelyCreatedLabel = (data: EditorData) => { + if (this.isCreationInProgress()) { + const mousePositionSnapped: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + this.activePath.push(mousePositionSnapped); + + if (this.activePath.length === 3) { + this.addLabelAndFinishCreation(data); + } + } else { + const isMouseOverImage: boolean = RenderEngineUtil.isMouseOverImage(data); + if (isMouseOverImage) { + this.startLabelCreation(data); + } + } + }; + + private startLabelCreation = (data: EditorData) => { + const mousePositionSnapped: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + this.activePath.push(mousePositionSnapped); + store.dispatch(updateActiveLabelId(null)); + EditorActions.setViewPortActionsDisabledStatus(true); + }; + + private addLabelAndFinishCreation = (data: EditorData) => { + const mousePositionSnapped: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + + // Complete the rectangle + const p1 = this.activePath[0]; + const p2 = this.activePath[1]; + + // Calculate perpendicular vector + const edge = PointUtil.subtract(p2, p1); + const perpendicular = { x: -edge.y, y: edge.x }; + + // Project mouse position onto perpendicular direction + const toMouse = PointUtil.subtract(mousePositionSnapped, p1); + const projection = (toMouse.x * perpendicular.x + toMouse.y * perpendicular.y) / + (perpendicular.x * perpendicular.x + perpendicular.y * perpendicular.y); + + const offset = { x: perpendicular.x * projection, y: perpendicular.y * projection }; + const p3 = PointUtil.add(p2, offset); + const p4 = PointUtil.add(p1, offset); + + const pathOnImage: IPoint[] = [p1, p2, p3, p4].map((point: IPoint) => { + return RenderEngineUtil.transferPointFromViewPortContentToImage(point, data); + }); + + this.addOBBLabel(pathOnImage); + this.endLabelCreation(); + }; + + private endLabelCreation() { + this.activePath = []; + EditorActions.setViewPortActionsDisabledStatus(false); + } + + private addOBBLabel = (vertices: IPoint[]) => { + const activeLabelId = LabelsSelector.getActiveLabelNameId(); + const imageData: ImageData = LabelsSelector.getActiveImageData(); + const labelOBB: LabelOBB = LabelUtil.createLabelOBB(activeLabelId, vertices); + + if (!imageData.labelOBBs) { + imageData.labelOBBs = []; + } + + imageData.labelOBBs.push(labelOBB); + store.dispatch(updateImageDataById(imageData.id, imageData)); + store.dispatch(updateFirstLabelCreatedFlag(true)); + store.dispatch(updateActiveLabelId(labelOBB.id)); + }; + + private getOBBUnderMouse(data: EditorData): LabelOBB { + const labelOBBs: LabelOBB[] = LabelsSelector.getActiveImageData().labelOBBs || []; + + for (const labelOBB of labelOBBs) { + if (labelOBB.isVisible && this.isMouseOverOBB(labelOBB.vertices, data)) { + return labelOBB; + } + } + return null; + } + + private isMouseOverOBB(vertices: IPoint[], data: EditorData): boolean { + const pathOnCanvas: IPoint[] = RenderEngineUtil.transferPolygonFromImageToViewPortContent(vertices, data); + + // Check if mouse is over any anchor + for (let i = 0; i < pathOnCanvas.length; i++) { + if (this.isMouseOverAnchor(data.mousePositionOnViewPortContent, pathOnCanvas[i])) { + return true; + } + } + + // Check if mouse is inside the OBB or near edges + return this.isPointInsidePolygon(data.mousePositionOnViewPortContent, pathOnCanvas); + } + + private isPointInsidePolygon(point: IPoint, polygon: IPoint[]): boolean { + let inside = false; + for (let i = 0, j = polygon.length - 1; i < polygon.length; j = i++) { + const xi = polygon[i].x, yi = polygon[i].y; + const xj = polygon[j].x, yj = polygon[j].y; + + const intersect = ((yi > point.y) !== (yj > point.y)) + && (point.x < (xj - xi) * (point.y - yi) / (yj - yi) + xi); + if (intersect) inside = !inside; + } + return inside; + } + + private isMouseOverAnchor(mouse: IPoint, anchor: IPoint): boolean { + const anchorRect: IRect = RectUtil.getRectWithCenterAndSize(anchor, RenderEngineSettings.anchorHoverSize); + return RectUtil.isPointInside(anchorRect, mouse); + } + + private calculateRectangleCorners(originalVertices: IPoint[], draggedIndex: number, newPosition: IPoint): IPoint[] { + // The opposite corner is the anchor (stays fixed) + const anchorIndex = (draggedIndex + 2) % 4; + const anchor = originalVertices[anchorIndex]; + + // Get the two adjacent corners + const adj1Index = (anchorIndex + 1) % 4; + const adj2Index = (anchorIndex + 3) % 4; + + // Calculate edge vectors from anchor in the original rectangle + const edge1 = PointUtil.subtract(originalVertices[adj1Index], anchor); + const edge2 = PointUtil.subtract(originalVertices[adj2Index], anchor); + + // Vector from anchor to new dragged position + const toNew = PointUtil.subtract(newPosition, anchor); + + // Normalize edge directions + const edge1Length = Math.sqrt(edge1.x * edge1.x + edge1.y * edge1.y); + const edge2Length = Math.sqrt(edge2.x * edge2.x + edge2.y * edge2.y); + + if (edge1Length === 0 || edge2Length === 0) { + return originalVertices; + } + + const edge1Norm = { x: edge1.x / edge1Length, y: edge1.y / edge1Length }; + const edge2Norm = { x: edge2.x / edge2Length, y: edge2.y / edge2Length }; + + // Project toNew onto both edge directions + const proj1 = toNew.x * edge1Norm.x + toNew.y * edge1Norm.y; + const proj2 = toNew.x * edge2Norm.x + toNew.y * edge2Norm.y; + + // Calculate new positions for all corners + const newVertices = new Array(4); + newVertices[anchorIndex] = anchor; + newVertices[adj1Index] = { + x: anchor.x + edge1Norm.x * proj1, + y: anchor.y + edge1Norm.y * proj1 + }; + newVertices[adj2Index] = { + x: anchor.x + edge2Norm.x * proj2, + y: anchor.y + edge2Norm.y * proj2 + }; + newVertices[draggedIndex] = { + x: anchor.x + edge1Norm.x * proj1 + edge2Norm.x * proj2, + y: anchor.y + edge1Norm.y * proj1 + edge2Norm.y * proj2 + }; + + return newVertices; + } + + private startExistingLabelResize = (data: EditorData, labelId: string, anchorIndex: number) => { + store.dispatch(updateActiveLabelId(labelId)); + this.resizeAnchorIndex = anchorIndex; + EditorActions.setViewPortActionsDisabledStatus(true); + }; + + private endExistingLabelResize = (data: EditorData) => { + const activeLabelOBB: LabelOBB = LabelsSelector.getActiveImageData().labelOBBs.find( + (label: LabelOBB) => label.id === LabelsSelector.getActiveLabelId() + ); + + if (!!activeLabelOBB) { + const snappedMousePosition: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + const imageData: ImageData = LabelsSelector.getActiveImageData(); + + imageData.labelOBBs = imageData.labelOBBs.map((labelOBB: LabelOBB) => { + if (labelOBB.id === activeLabelOBB.id) { + // Calculate new rectangle corners in viewport coordinates + const verticesOnCanvas = RenderEngineUtil.transferPolygonFromImageToViewPortContent(labelOBB.vertices, data); + const newVerticesOnCanvas = this.calculateRectangleCorners(verticesOnCanvas, this.resizeAnchorIndex, snappedMousePosition); + + // Transfer back to image coordinates + const newVertices = newVerticesOnCanvas.map((point: IPoint) => + RenderEngineUtil.transferPointFromViewPortContentToImage(point, data) + ); + + return { + ...labelOBB, + vertices: newVertices + }; + } + return labelOBB; + }); + + store.dispatch(updateImageDataById(imageData.id, imageData)); + } + + this.resizeAnchorIndex = null; + EditorActions.setViewPortActionsDisabledStatus(false); + }; +} diff --git a/src/store/labels/types.ts b/src/store/labels/types.ts index 38397f7e8..ab3741e64 100644 --- a/src/store/labels/types.ts +++ b/src/store/labels/types.ts @@ -33,6 +33,13 @@ export type LabelLine = Annotation & { line: ILine; } +export type LabelOBB = Annotation & { + vertices: IPoint[]; + isCreatedByAI: boolean; + status: LabelStatus; + suggestedLabel: string; +} + export type LabelName = { name: string; id: string; @@ -47,6 +54,7 @@ export type ImageData = { labelPoints: LabelPoint[]; labelLines: LabelLine[]; labelPolygons: LabelPolygon[]; + labelOBBs: LabelOBB[]; labelNameIds: string[]; // YOLO diff --git a/src/utils/ImageDataUtil.ts b/src/utils/ImageDataUtil.ts index fab0fcf54..f75e290ef 100644 --- a/src/utils/ImageDataUtil.ts +++ b/src/utils/ImageDataUtil.ts @@ -13,6 +13,7 @@ export class ImageDataUtil { labelPoints: [], labelLines: [], labelPolygons: [], + labelOBBs: [], labelNameIds: [], isVisitedByYOLOObjectDetector: false, isVisitedBySSDObjectDetector: false, @@ -28,6 +29,7 @@ export class ImageDataUtil { labelPoints: [], labelLines: [], labelPolygons: [], + labelOBBs: [], labelNameIds: [] } } diff --git a/src/utils/LabelUtil.ts b/src/utils/LabelUtil.ts index feec03d61..3daf92f93 100644 --- a/src/utils/LabelUtil.ts +++ b/src/utils/LabelUtil.ts @@ -1,4 +1,4 @@ -import {Annotation, LabelName, LabelPoint, LabelPolygon, LabelRect} from '../store/labels/types'; +import {Annotation, LabelName, LabelPoint, LabelPolygon, LabelRect, LabelOBB} from '../store/labels/types'; import { v4 as uuidv4 } from 'uuid'; import {find} from 'lodash'; import {IRect} from '../interfaces/IRect'; @@ -37,6 +37,18 @@ export class LabelUtil { } } + public static createLabelOBB(labelId: string, vertices: IPoint[]): LabelOBB { + return { + id: uuidv4(), + labelId, + vertices, + isVisible: true, + isCreatedByAI: false, + status: LabelStatus.ACCEPTED, + suggestedLabel: null + } + } + public static createLabelPoint(labelId: string, point: IPoint): LabelPoint { return { id: uuidv4(), diff --git a/src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx b/src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx index 5e6b8b1ae..5124e530c 100644 --- a/src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx +++ b/src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx @@ -14,6 +14,7 @@ import {Settings} from "../../../../settings/Settings"; import RectLabelsList from "../RectLabelsList/RectLabelsList"; import PointLabelsList from "../PointLabelsList/PointLabelsList"; import PolygonLabelsList from "../PolygonLabelsList/PolygonLabelsList"; +import OBBLabelsList from "../OBBLabelsList/OBBLabelsList"; import {ContextManager} from "../../../../logic/context/ContextManager"; import {ContextType} from "../../../../data/enums/ContextType"; import {EventType} from "../../../../data/enums/EventType"; @@ -53,7 +54,8 @@ class LabelsToolkit extends React.Component { LabelType.RECT, LabelType.POINT, LabelType.LINE, - LabelType.POLYGON + LabelType.POLYGON, + LabelType.OBB ]; const activeTab: LabelType = props.activeLabelType ? props.activeLabelType : this.tabs[0]; @@ -162,6 +164,13 @@ class LabelsToolkit extends React.Component { }} imageData={imagesData[activeImageIndex]} />} + {labelType === LabelType.OBB && } {labelType === LabelType.IMAGE_RECOGNITION && any; + activeLabelId: string; + highlightedLabelId: string; + updateActiveLabelNameIdAction: (activeLabelId: string) => any; + labelNames: LabelName[]; + updateActiveLabelIdAction: (activeLabelId: string) => any; +} + +const OBBLabelsList: React.FC = ( + { + size, + imageData, + updateImageDataByIdAction, + labelNames, + updateActiveLabelNameIdAction, + activeLabelId, + highlightedLabelId, + updateActiveLabelIdAction + } +) => { + const labelInputFieldHeight = 40; + const listStyle: React.CSSProperties = { + width: size.width, + height: size.height + }; + const labelOBBs = imageData.labelOBBs || []; + const listStyleContent: React.CSSProperties = { + width: size.width, + height: labelOBBs.length * labelInputFieldHeight + }; + + const deleteOBBLabelById = (labelOBBId: string) => { + LabelActions.deleteOBBLabelById(imageData.id, labelOBBId); + }; + + const toggleOBBLabelVisibilityById = (labelOBBId: string) => { + LabelActions.toggleLabelVisibilityById(imageData.id, labelOBBId); + }; + + const updateOBBLabel = (labelOBBId: string, labelNameId: string) => { + const newImageData = { + ...imageData, + labelOBBs: (imageData.labelOBBs || []) + .map((labelOBB: LabelOBB) => { + if (labelOBB.id === labelOBBId) { + return { + ...labelOBB, + labelId: labelNameId, + status: LabelStatus.ACCEPTED + } + } else { + return labelOBB + } + }) + }; + updateImageDataByIdAction(imageData.id, newImageData); + updateActiveLabelNameIdAction(labelNameId); + }; + + const onClickHandler = () => { + updateActiveLabelIdAction(null); + }; + + const getChildren = () => { + return labelOBBs + .filter((labelOBB: LabelOBB) => labelOBB.status === LabelStatus.ACCEPTED) + .map((labelOBB: LabelOBB) => { + return + }); + }; + + return ( +
+ {labelOBBs.filter((labelOBB: LabelOBB) => labelOBB.status === LabelStatus.ACCEPTED).length === 0 ? + : + +
+ {getChildren()} +
+ + } +
+ ); +}; + +const mapDispatchToProps = { + updateImageDataByIdAction: updateImageDataById, + updateActiveLabelNameIdAction: updateActiveLabelNameId, + updateActiveLabelIdAction: updateActiveLabelId +}; + +const mapStateToProps = (state: AppState) => ({ + activeLabelId: state.labels.activeLabelId, + highlightedLabelId: state.labels.highlightedLabelId, + labelNames : state.labels.labels +}); + +export default connect( + mapStateToProps, + mapDispatchToProps +)(OBBLabelsList); diff --git a/src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx b/src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx index bfe38a138..c248137bd 100644 --- a/src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx +++ b/src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx @@ -9,6 +9,7 @@ import { PolygonLabelsExporter } from '../../../logic/export/polygon/PolygonLabe import { PopupActions } from '../../../logic/actions/PopupActions'; import { LineLabelsExporter } from '../../../logic/export/LineLabelExport'; import { TagLabelsExporter } from '../../../logic/export/TagLabelsExport'; +import { OBBLabelsExporter } from '../../../logic/export/OBBLabelsExporter'; import GenericLabelTypePopup from '../GenericLabelTypePopup/GenericLabelTypePopup'; import { ExportFormatData } from '../../../data/ExportFormatData'; import { AppState } from '../../../store'; @@ -36,6 +37,9 @@ const ExportLabelPopup: React.FC = ({ activeLabelType }) => { case LabelType.POLYGON: PolygonLabelsExporter.export(exportFormatType); break; + case LabelType.OBB: + OBBLabelsExporter.export(exportFormatType); + break; case LabelType.IMAGE_RECOGNITION: TagLabelsExporter.export(exportFormatType); break; From b74d61ccf0d5cd0f18ea443149c1ccd88e463bb2 Mon Sep 17 00:00:00 2001 From: Nagy Daniel Date: Mon, 27 Oct 2025 11:37:33 +0100 Subject: [PATCH 2/7] OBB WIP 2 --- src/logic/render/OBBRenderEngine.ts | 139 ++++++++++++++++++++++------ 1 file changed, 113 insertions(+), 26 deletions(-) diff --git a/src/logic/render/OBBRenderEngine.ts b/src/logic/render/OBBRenderEngine.ts index eaa675c0b..429d979b6 100644 --- a/src/logic/render/OBBRenderEngine.ts +++ b/src/logic/render/OBBRenderEngine.ts @@ -383,46 +383,133 @@ export class OBBRenderEngine extends BaseRenderEngine { const anchorIndex = (draggedIndex + 2) % 4; const anchor = originalVertices[anchorIndex]; - // Get the two adjacent corners + // Get the two adjacent corners (these share an edge with the anchor) const adj1Index = (anchorIndex + 1) % 4; const adj2Index = (anchorIndex + 3) % 4; - // Calculate edge vectors from anchor in the original rectangle - const edge1 = PointUtil.subtract(originalVertices[adj1Index], anchor); - const edge2 = PointUtil.subtract(originalVertices[adj2Index], anchor); + // Calculate original edge vectors from anchor + const origEdge1 = PointUtil.subtract(originalVertices[adj1Index], anchor); + const origEdge2 = PointUtil.subtract(originalVertices[adj2Index], anchor); + const origDiagonal = PointUtil.subtract(originalVertices[draggedIndex], anchor); - // Vector from anchor to new dragged position - const toNew = PointUtil.subtract(newPosition, anchor); + const origEdge1Length = Math.sqrt(origEdge1.x * origEdge1.x + origEdge1.y * origEdge1.y); + const origEdge2Length = Math.sqrt(origEdge2.x * origEdge2.x + origEdge2.y * origEdge2.y); + const origDiagLength = Math.sqrt(origDiagonal.x * origDiagonal.x + origDiagonal.y * origDiagonal.y); - // Normalize edge directions - const edge1Length = Math.sqrt(edge1.x * edge1.x + edge1.y * edge1.y); - const edge2Length = Math.sqrt(edge2.x * edge2.x + edge2.y * edge2.y); + if (origEdge1Length === 0 || origEdge2Length === 0 || origDiagLength === 0) { + return originalVertices; + } + + // Calculate which edge is more aligned with the diagonal (using dot product) + const origEdge1Norm = { x: origEdge1.x / origEdge1Length, y: origEdge1.y / origEdge1Length }; + const origEdge2Norm = { x: origEdge2.x / origEdge2Length, y: origEdge2.y / origEdge2Length }; + const origDiagNorm = { x: origDiagonal.x / origDiagLength, y: origDiagonal.y / origDiagLength }; + + const dot1 = origEdge1Norm.x * origDiagNorm.x + origEdge1Norm.y * origDiagNorm.y; + const dot2 = origEdge2Norm.x * origDiagNorm.x + origEdge2Norm.y * origDiagNorm.y; - if (edge1Length === 0 || edge2Length === 0) { + // Determine which edge is primary (more aligned with diagonal) + let primaryEdge: IPoint, secondaryEdge: IPoint; + let primaryIndex: number, secondaryIndex: number; + let primaryLength: number, secondaryLength: number; + + if (Math.abs(dot1) >= Math.abs(dot2)) { + primaryEdge = origEdge1; + secondaryEdge = origEdge2; + primaryIndex = adj1Index; + secondaryIndex = adj2Index; + primaryLength = origEdge1Length; + secondaryLength = origEdge2Length; + } else { + primaryEdge = origEdge2; + secondaryEdge = origEdge1; + primaryIndex = adj2Index; + secondaryIndex = adj1Index; + primaryLength = origEdge2Length; + secondaryLength = origEdge1Length; + } + + // Calculate the angle between the primary edge and the diagonal in the original rectangle + // Using: cos(angle) = dot(primary, diagonal) / (|primary| * |diagonal|) + const cosAngle = (primaryEdge.x * origDiagonal.x + primaryEdge.y * origDiagonal.y) / (primaryLength * origDiagLength); + const sinAngle = Math.sqrt(1 - cosAngle * cosAngle); + + // tan(angle) = opposite/adjacent = secondaryLength / primaryLength (from right triangle) + // This angle must be preserved to maintain the rectangle geometry + + // New diagonal + const newDiagonal = PointUtil.subtract(newPosition, anchor); + const newDiagLength = Math.sqrt(newDiagonal.x * newDiagonal.x + newDiagonal.y * newDiagonal.y); + + if (newDiagLength === 0) { return originalVertices; } - const edge1Norm = { x: edge1.x / edge1Length, y: edge1.y / edge1Length }; - const edge2Norm = { x: edge2.x / edge2Length, y: edge2.y / edge2Length }; + // New edge lengths (scaled proportionally based on diagonal change) + const scaleFactor = newDiagLength / origDiagLength; + const newPrimaryLength = primaryLength * scaleFactor; + const newSecondaryLength = secondaryLength * scaleFactor; - // Project toNew onto both edge directions - const proj1 = toNew.x * edge1Norm.x + toNew.y * edge1Norm.y; - const proj2 = toNew.x * edge2Norm.x + toNew.y * edge2Norm.y; + // Get the angle of the new diagonal + const newDiagAngle = Math.atan2(newDiagonal.y, newDiagonal.x); - // Calculate new positions for all corners - const newVertices = new Array(4); - newVertices[anchorIndex] = anchor; - newVertices[adj1Index] = { - x: anchor.x + edge1Norm.x * proj1, - y: anchor.y + edge1Norm.y * proj1 + // Get the original angle of the diagonal + const origDiagAngle = Math.atan2(origDiagonal.y, origDiagonal.x); + + // Get the original angle of the primary edge + const origPrimaryAngle = Math.atan2(primaryEdge.y, primaryEdge.x); + + // Calculate the offset angle between primary edge and diagonal + let angleOffset = origPrimaryAngle - origDiagAngle; + + // New primary edge angle + const newPrimaryAngle = newDiagAngle + angleOffset; + + // New primary edge vector + const newPrimaryEdge = { + x: Math.cos(newPrimaryAngle) * newPrimaryLength, + y: Math.sin(newPrimaryAngle) * newPrimaryLength }; - newVertices[adj2Index] = { - x: anchor.x + edge2Norm.x * proj2, - y: anchor.y + edge2Norm.y * proj2 + + // Calculate the original angle between primary and secondary edges + const origSecondaryAngle = Math.atan2(secondaryEdge.y, secondaryEdge.x); + const origAngleBetweenEdges = origSecondaryAngle - origPrimaryAngle; + + // New secondary edge angle maintains the same angle relationship + const newSecondaryAngle = newPrimaryAngle + origAngleBetweenEdges; + const newSecondaryEdge = { + x: Math.cos(newSecondaryAngle) * newSecondaryLength, + y: Math.sin(newSecondaryAngle) * newSecondaryLength }; + + // Assign edges to correct vertices + const newVertices = new Array(4); + newVertices[anchorIndex] = anchor; + + if (primaryIndex === adj1Index) { + newVertices[adj1Index] = { + x: anchor.x + newPrimaryEdge.x, + y: anchor.y + newPrimaryEdge.y + }; + newVertices[adj2Index] = { + x: anchor.x + newSecondaryEdge.x, + y: anchor.y + newSecondaryEdge.y + }; + } else { + newVertices[adj2Index] = { + x: anchor.x + newPrimaryEdge.x, + y: anchor.y + newPrimaryEdge.y + }; + newVertices[adj1Index] = { + x: anchor.x + newSecondaryEdge.x, + y: anchor.y + newSecondaryEdge.y + }; + } + + // Dragged corner is at the sum of both edges newVertices[draggedIndex] = { - x: anchor.x + edge1Norm.x * proj1 + edge2Norm.x * proj2, - y: anchor.y + edge1Norm.y * proj1 + edge2Norm.y * proj2 + x: anchor.x + newPrimaryEdge.x + newSecondaryEdge.x, + y: anchor.y + newPrimaryEdge.y + newSecondaryEdge.y }; return newVertices; From 98ab79f41889a26b86f4ae774dfa5434b223da7b Mon Sep 17 00:00:00 2001 From: Nagy Daniel Date: Mon, 27 Oct 2025 11:42:30 +0100 Subject: [PATCH 3/7] OBB WIP 3 --- src/logic/render/OBBRenderEngine.ts | 154 +++++++++++++++++++++++++++- 1 file changed, 151 insertions(+), 3 deletions(-) diff --git a/src/logic/render/OBBRenderEngine.ts b/src/logic/render/OBBRenderEngine.ts index 429d979b6..cf4ee5480 100644 --- a/src/logic/render/OBBRenderEngine.ts +++ b/src/logic/render/OBBRenderEngine.ts @@ -34,6 +34,7 @@ export class OBBRenderEngine extends BaseRenderEngine { private activePath: IPoint[] = []; private resizeAnchorIndex: number = null; + private resizeEdgeIndex: number = null; public constructor(canvas: HTMLCanvasElement) { super(canvas); @@ -70,6 +71,7 @@ export class OBBRenderEngine extends BaseRenderEngine { } else { const obbUnderMouse: LabelOBB = this.getOBBUnderMouse(data); if (!!obbUnderMouse) { + // Check corner anchors first const anchorIndex: number = obbUnderMouse.vertices.reduce( (indexUnderMouse: number, anchor: IPoint, index: number) => { if (indexUnderMouse === null) { @@ -84,7 +86,13 @@ export class OBBRenderEngine extends BaseRenderEngine { if (anchorIndex !== null) { this.startExistingLabelResize(data, obbUnderMouse.id, anchorIndex); } else { - store.dispatch(updateActiveLabelId(obbUnderMouse.id)); + // Check edge anchors + const edgeIndex = this.getEdgeUnderMouse(obbUnderMouse.vertices, data); + if (edgeIndex !== null) { + this.startExistingLabelEdgeResize(data, obbUnderMouse.id, edgeIndex); + } else { + store.dispatch(updateActiveLabelId(obbUnderMouse.id)); + } } } else { this.updateActivelyCreatedLabel(data); @@ -96,6 +104,8 @@ export class OBBRenderEngine extends BaseRenderEngine { public mouseUpHandler(data: EditorData): void { if (this.isResizeInProgress()) this.endExistingLabelResize(data); + if (this.isEdgeResizeInProgress()) + this.endExistingLabelEdgeResize(data); } public mouseMoveHandler(data: EditorData): void { @@ -103,7 +113,7 @@ export class OBBRenderEngine extends BaseRenderEngine { const isOverImage: boolean = RenderEngineUtil.isMouseOverImage(data); if (isOverImage && !this.isCreationInProgress()) { const labelOBB: LabelOBB = this.getOBBUnderMouse(data); - if (!!labelOBB && !this.isResizeInProgress()) { + if (!!labelOBB && !this.isResizeInProgress() && !this.isEdgeResizeInProgress()) { if (LabelsSelector.getHighlightedLabelId() !== labelOBB.id) { store.dispatch(updateHighlightedLabelId(labelOBB.id)) } @@ -204,6 +214,12 @@ export class OBBRenderEngine extends BaseRenderEngine { data.viewPortContentImageRect ); pathOnCanvas = this.calculateRectangleCorners(pathOnCanvas, this.resizeAnchorIndex, snappedMousePosition); + } else if (this.isEdgeResizeInProgress()) { + const snappedMousePosition: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + pathOnCanvas = this.calculateRectangleFromEdgeDrag(pathOnCanvas, this.resizeEdgeIndex, snappedMousePosition); } const lineColor: string = BaseRenderEngine.resolveLabelLineColor(labelOBB.labelId, true); @@ -216,9 +232,20 @@ export class OBBRenderEngine extends BaseRenderEngine { DrawUtil.drawPolygon(this.canvas, pathOnCanvas, lineColor, RenderEngineSettings.LINE_THICKNESS); if (isActive) { + // Draw corner anchors pathOnCanvas.forEach((point: IPoint) => { DrawUtil.drawCircleWithFill(this.canvas, point, RenderEngineSettings.anchorSize.width / 2, anchorColor); }); + + // Draw edge anchors (midpoints) + for (let i = 0; i < pathOnCanvas.length; i++) { + const nextIndex = (i + 1) % pathOnCanvas.length; + const edgeMidpoint = { + x: (pathOnCanvas[i].x + pathOnCanvas[nextIndex].x) / 2, + y: (pathOnCanvas[i].y + pathOnCanvas[nextIndex].y) / 2 + }; + DrawUtil.drawRectWithFill(this.canvas, RectUtil.getRectWithCenterAndSize(edgeMidpoint, RenderEngineSettings.anchorSize), anchorColor); + } } } @@ -244,7 +271,7 @@ export class OBBRenderEngine extends BaseRenderEngine { // ================================================================================================================= public isInProgress(): boolean { - return this.isCreationInProgress() || this.isResizeInProgress(); + return this.isCreationInProgress() || this.isResizeInProgress() || this.isEdgeResizeInProgress(); } private isCreationInProgress(): boolean { @@ -255,6 +282,10 @@ export class OBBRenderEngine extends BaseRenderEngine { return this.resizeAnchorIndex !== null; } + private isEdgeResizeInProgress(): boolean { + return this.resizeEdgeIndex !== null; + } + private updateActivelyCreatedLabel = (data: EditorData) => { if (this.isCreationInProgress()) { const mousePositionSnapped: IPoint = RectUtil.snapPointToRect( @@ -558,4 +589,121 @@ export class OBBRenderEngine extends BaseRenderEngine { this.resizeAnchorIndex = null; EditorActions.setViewPortActionsDisabledStatus(false); }; + + private getEdgeUnderMouse(vertices: IPoint[], data: EditorData): number | null { + const pathOnCanvas: IPoint[] = RenderEngineUtil.transferPolygonFromImageToViewPortContent(vertices, data); + + for (let i = 0; i < pathOnCanvas.length; i++) { + const nextIndex = (i + 1) % pathOnCanvas.length; + const edgeMidpoint = { + x: (pathOnCanvas[i].x + pathOnCanvas[nextIndex].x) / 2, + y: (pathOnCanvas[i].y + pathOnCanvas[nextIndex].y) / 2 + }; + + if (this.isMouseOverAnchor(data.mousePositionOnViewPortContent, edgeMidpoint)) { + return i; + } + } + + return null; + } + + private startExistingLabelEdgeResize = (data: EditorData, labelId: string, edgeIndex: number) => { + store.dispatch(updateActiveLabelId(labelId)); + this.resizeEdgeIndex = edgeIndex; + EditorActions.setViewPortActionsDisabledStatus(true); + }; + + private endExistingLabelEdgeResize = (data: EditorData) => { + const activeLabelOBB: LabelOBB = LabelsSelector.getActiveImageData().labelOBBs.find( + (label: LabelOBB) => label.id === LabelsSelector.getActiveLabelId() + ); + + if (!!activeLabelOBB) { + const snappedMousePosition: IPoint = RectUtil.snapPointToRect( + data.mousePositionOnViewPortContent, + data.viewPortContentImageRect + ); + const imageData: ImageData = LabelsSelector.getActiveImageData(); + + imageData.labelOBBs = imageData.labelOBBs.map((labelOBB: LabelOBB) => { + if (labelOBB.id === activeLabelOBB.id) { + // Calculate new rectangle from edge drag in viewport coordinates + const verticesOnCanvas = RenderEngineUtil.transferPolygonFromImageToViewPortContent(labelOBB.vertices, data); + const newVerticesOnCanvas = this.calculateRectangleFromEdgeDrag(verticesOnCanvas, this.resizeEdgeIndex, snappedMousePosition); + + // Transfer back to image coordinates + const newVertices = newVerticesOnCanvas.map((point: IPoint) => + RenderEngineUtil.transferPointFromViewPortContentToImage(point, data) + ); + + return { + ...labelOBB, + vertices: newVertices + }; + } + return labelOBB; + }); + + store.dispatch(updateImageDataById(imageData.id, imageData)); + } + + this.resizeEdgeIndex = null; + EditorActions.setViewPortActionsDisabledStatus(false); + }; + + private calculateRectangleFromEdgeDrag(originalVertices: IPoint[], edgeIndex: number, mousePosition: IPoint): IPoint[] { + // Edge is defined by edgeIndex and the next vertex + const v1Index = edgeIndex; + const v2Index = (edgeIndex + 1) % 4; + const v3Index = (edgeIndex + 2) % 4; + const v4Index = (edgeIndex + 3) % 4; + + // The dragged edge is v1->v2, opposite edge is v3->v4 + const v1 = originalVertices[v1Index]; + const v2 = originalVertices[v2Index]; + const v3 = originalVertices[v3Index]; + const v4 = originalVertices[v4Index]; + + // Edge direction (normalized) + const edgeVector = PointUtil.subtract(v2, v1); + const edgeLength = Math.sqrt(edgeVector.x * edgeVector.x + edgeVector.y * edgeVector.y); + + if (edgeLength === 0) { + return originalVertices; + } + + const edgeDir = { x: edgeVector.x / edgeLength, y: edgeVector.y / edgeLength }; + + // Perpendicular to edge (pointing inward toward rectangle) + const perpendicular = { x: -edgeDir.y, y: edgeDir.x }; + + // Determine which perpendicular direction points into the rectangle + const toOppositeEdge = PointUtil.subtract(v3, v1); + const perpDot = toOppositeEdge.x * perpendicular.x + toOppositeEdge.y * perpendicular.y; + + if (perpDot < 0) { + perpendicular.x = -perpendicular.x; + perpendicular.y = -perpendicular.y; + } + + // Project mouse position onto the perpendicular direction from the edge + const toMouse = PointUtil.subtract(mousePosition, v1); + const perpDistance = toMouse.x * perpendicular.x + toMouse.y * perpendicular.y; + + // Translate the dragged edge + const offset = { x: perpendicular.x * perpDistance, y: perpendicular.y * perpDistance }; + + const newV1 = { x: v1.x + offset.x, y: v1.y + offset.y }; + const newV2 = { x: v2.x + offset.x, y: v2.y + offset.y }; + + // Opposite edge stays in place + const newVertices = new Array(4); + newVertices[v1Index] = newV1; + newVertices[v2Index] = newV2; + newVertices[v3Index] = v3; + newVertices[v4Index] = v4; + + return newVertices; + } } From e463e3772088f4847348f9afb541c1921c2269b0 Mon Sep 17 00:00:00 2001 From: Nagy Daniel Date: Mon, 27 Oct 2025 11:49:30 +0100 Subject: [PATCH 4/7] OBB WIP 4 --- src/logic/context/EditorContext.ts | 4 ++++ src/logic/render/OBBRenderEngine.ts | 5 +++++ 2 files changed, 9 insertions(+) diff --git a/src/logic/context/EditorContext.ts b/src/logic/context/EditorContext.ts index 14ee56aeb..7ef26a1bf 100644 --- a/src/logic/context/EditorContext.ts +++ b/src/logic/context/EditorContext.ts @@ -11,6 +11,7 @@ import {Direction} from "../../data/enums/Direction"; import {PlatformUtil} from "../../utils/PlatformUtil"; import {LabelActions} from "../actions/LabelActions"; import {LineRenderEngine} from "../render/LineRenderEngine"; +import {OBBRenderEngine} from "../render/OBBRenderEngine"; export class EditorContext extends BaseContext { public static actions: HotKeyAction[] = [ @@ -35,6 +36,9 @@ export class EditorContext extends BaseContext { case LabelType.LINE: (EditorModel.supportRenderingEngine as LineRenderEngine).cancelLabelCreation(); break; + case LabelType.OBB: + (EditorModel.supportRenderingEngine as OBBRenderEngine).cancelLabelCreation(); + break; } } EditorActions.fullRender(); diff --git a/src/logic/render/OBBRenderEngine.ts b/src/logic/render/OBBRenderEngine.ts index cf4ee5480..feb8fe3a3 100644 --- a/src/logic/render/OBBRenderEngine.ts +++ b/src/logic/render/OBBRenderEngine.ts @@ -351,6 +351,11 @@ export class OBBRenderEngine extends BaseRenderEngine { EditorActions.setViewPortActionsDisabledStatus(false); } + public cancelLabelCreation() { + this.activePath = []; + EditorActions.setViewPortActionsDisabledStatus(false); + } + private addOBBLabel = (vertices: IPoint[]) => { const activeLabelId = LabelsSelector.getActiveLabelNameId(); const imageData: ImageData = LabelsSelector.getActiveImageData(); From 718d5bb505d56ccc8f97007c957884886552dc84 Mon Sep 17 00:00:00 2001 From: Nagy Daniel Date: Mon, 27 Oct 2025 12:11:53 +0100 Subject: [PATCH 5/7] OBB support: fix imported annotations --- src/data/ImportFormatData.ts | 6 +++ src/logic/export/OBBLabelsExporter.ts | 11 +++++ src/logic/export/RectLabelsExporter.ts | 12 +++++ src/logic/import/yolo/YOLOImporter.ts | 30 +++++++++--- src/logic/import/yolo/YOLOUtils.ts | 65 +++++++++++++++++++++++++- 5 files changed, 117 insertions(+), 7 deletions(-) diff --git a/src/data/ImportFormatData.ts b/src/data/ImportFormatData.ts index c2ae4f16b..6222fafed 100644 --- a/src/data/ImportFormatData.ts +++ b/src/data/ImportFormatData.ts @@ -27,5 +27,11 @@ export const ImportFormatData: ImportFormatDataMap = { label: 'Single file in COCO JSON format.' } ], + [LabelType.OBB]: [ + { + type: AnnotationFormatType.YOLO, + label: 'Multiple files in YOLO OBB format along with labels names definition - labels.txt file.' + } + ], [LabelType.IMAGE_RECOGNITION]: [] } diff --git a/src/logic/export/OBBLabelsExporter.ts b/src/logic/export/OBBLabelsExporter.ts index 7509f22b5..9393422a5 100644 --- a/src/logic/export/OBBLabelsExporter.ts +++ b/src/logic/export/OBBLabelsExporter.ts @@ -22,6 +22,17 @@ export class OBBLabelsExporter { private static exportAsYOLO(): void { const zip = new JSZip(); + + // Add labels.txt file with label names + const labelNames: LabelName[] = LabelsSelector.getLabelNames(); + const labelsContent: string = labelNames.map((label: LabelName) => label.name).join('\n'); + try { + zip.file('labels.txt', labelsContent); + } catch (error) { + throw new Error(error as string); + } + + // Add annotation files for each image LabelsSelector.getImagesData() .forEach((imageData: ImageData) => { const fileContent: string = OBBLabelsExporter.wrapOBBLabelsIntoYOLO(imageData); diff --git a/src/logic/export/RectLabelsExporter.ts b/src/logic/export/RectLabelsExporter.ts index 151d4513f..21658086a 100644 --- a/src/logic/export/RectLabelsExporter.ts +++ b/src/logic/export/RectLabelsExporter.ts @@ -32,6 +32,18 @@ export class RectLabelsExporter { private static exportAsYOLO(): void { const zip = new JSZip(); + + // Add labels.txt file with label names + const labelNames: LabelName[] = LabelsSelector.getLabelNames(); + const labelsContent: string = labelNames.map((label: LabelName) => label.name).join('\n'); + try { + zip.file('labels.txt', labelsContent); + } catch (error) { + // TODO + throw new Error(error as string); + } + + // Add annotation files for each image LabelsSelector.getImagesData() .forEach((imageData: ImageData) => { const fileContent: string = RectLabelsExporter.wrapRectLabelsIntoYOLO(imageData); diff --git a/src/logic/import/yolo/YOLOImporter.ts b/src/logic/import/yolo/YOLOImporter.ts index e5e3da685..05aa2418b 100644 --- a/src/logic/import/yolo/YOLOImporter.ts +++ b/src/logic/import/yolo/YOLOImporter.ts @@ -78,12 +78,30 @@ export class YOLOImporter extends AnnotationImporter { public static applyAnnotations(imageData: ImageData, rawAnnotations: string, labelNames: LabelName[]): ImageData { const image: HTMLImageElement = ImageRepository.getById(imageData.id); - imageData.labelRects = YOLOUtils.parseYOLOAnnotationsFromString( - rawAnnotations, - labelNames, - {width: image.width, height: image.height}, - imageData.fileData.name - ); + + // Check if this is OBB format by looking at the first line + const firstLine = rawAnnotations.split(/[\r\n]/)[0]; + if (firstLine) { + const components = firstLine.trim().split(' '); + const isOBB = components.length === 9; + + if (isOBB) { + imageData.labelOBBs = YOLOUtils.parseYOLOOBBAnnotationsFromString( + rawAnnotations, + labelNames, + {width: image.width, height: image.height}, + imageData.fileData.name + ); + } else { + imageData.labelRects = YOLOUtils.parseYOLOAnnotationsFromString( + rawAnnotations, + labelNames, + {width: image.width, height: image.height}, + imageData.fileData.name + ); + } + } + return imageData; } diff --git a/src/logic/import/yolo/YOLOUtils.ts b/src/logic/import/yolo/YOLOUtils.ts index 7d540f37d..2a5c2f2df 100644 --- a/src/logic/import/yolo/YOLOUtils.ts +++ b/src/logic/import/yolo/YOLOUtils.ts @@ -1,4 +1,4 @@ -import {LabelName, LabelRect} from '../../../store/labels/types'; +import {LabelName, LabelRect, LabelOBB} from '../../../store/labels/types'; import {LabelUtil} from '../../../utils/LabelUtil'; import {AnnotationsParsingError, LabelNamesNotUniqueError} from './YOLOErrors'; import {ISize} from '../../../interfaces/ISize'; @@ -51,6 +51,20 @@ export class YOLOUtils { )); } + public static parseYOLOOBBAnnotationsFromString( + rawAnnotations: string, + labelNames: LabelName[], + imageSize: ISize, + imageName: string + ): LabelOBB[] { + return rawAnnotations + .split(/[\r\n]/) + .filter(Boolean) + .map((rawAnnotation: string) => YOLOUtils.parseYOLOOBBAnnotationFromString( + rawAnnotation, labelNames, imageSize, imageName + )); + } + public static parseYOLOAnnotationFromString( rawAnnotation: string, labelNames: LabelName[], @@ -76,6 +90,31 @@ export class YOLOUtils { return LabelUtil.createLabelRect(labelId, rect); } + public static parseYOLOOBBAnnotationFromString( + rawAnnotation: string, + labelNames: LabelName[], + imageSize: ISize, + imageName: string + ): LabelOBB { + const components = rawAnnotation.split(' '); + if (!YOLOUtils.validateYOLOOBBAnnotationComponents(components, labelNames.length)) { + throw new AnnotationsParsingError(imageName); + } + const labelIndex: number = parseInt(components[0]); + const labelId: string = labelNames[labelIndex].id; + + // Parse 4 corner coordinates (8 values) + const vertices = []; + for (let i = 0; i < 4; i++) { + vertices.push({ + x: parseFloat(components[1 + i * 2]) * imageSize.width, + y: parseFloat(components[2 + i * 2]) * imageSize.height + }); + } + + return LabelUtil.createLabelOBB(labelId, vertices); + } + public static validateYOLOAnnotationComponents(components: string[], labelNamesCount: number): boolean { const validateCoordinateValue = (rawValue: string): boolean => { const floatValue: number = Number(rawValue); @@ -95,4 +134,28 @@ export class YOLOUtils { validateCoordinateValue(components[4]) ].every(Boolean) } + + public static validateYOLOOBBAnnotationComponents(components: string[], labelNamesCount: number): boolean { + const validateCoordinateValue = (rawValue: string): boolean => { + const floatValue: number = Number(rawValue); + return !isNaN(floatValue) && 0.0 <= floatValue && floatValue <= 1.0; + } + const validateLabelIdx = (rawValue: string): boolean => { + const intValue: number = parseInt(rawValue); + return !isNaN(intValue) && 0 <= intValue && intValue < labelNamesCount; + } + + return [ + components.length === 9, + validateLabelIdx(components[0]), + validateCoordinateValue(components[1]), + validateCoordinateValue(components[2]), + validateCoordinateValue(components[3]), + validateCoordinateValue(components[4]), + validateCoordinateValue(components[5]), + validateCoordinateValue(components[6]), + validateCoordinateValue(components[7]), + validateCoordinateValue(components[8]) + ].every(Boolean) + } } From 48057d8a80136ae36c8d29a644fcae80ca542280 Mon Sep 17 00:00:00 2001 From: Nagy Daniel Date: Mon, 27 Oct 2025 12:13:50 +0100 Subject: [PATCH 6/7] OBB support: fix imported annotations --- src/ai/RoboflowAPIObjectDetector.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ai/RoboflowAPIObjectDetector.ts b/src/ai/RoboflowAPIObjectDetector.ts index 301efdeca..2378b392a 100644 --- a/src/ai/RoboflowAPIObjectDetector.ts +++ b/src/ai/RoboflowAPIObjectDetector.ts @@ -59,7 +59,7 @@ export class RoboflowAPIObjectDetector { FileUtil.loadImageBase64(imageData.fileData).then((data) => { axios({ method: 'POST', - url: '' + roboflowAPIDetails.model, + url: 'https://detect.roboflow.com/' + roboflowAPIDetails.model, params: { api_key: roboflowAPIDetails.key }, From f55a0e119c21a82e354582f7105c4410be8e325a Mon Sep 17 00:00:00 2001 From: Nagy Daniel Date: Mon, 27 Oct 2025 12:14:59 +0100 Subject: [PATCH 7/7] OBB support: fix imported annotations --- OBB_IMPLEMENTATION.md | 167 ------------------------------------------ 1 file changed, 167 deletions(-) delete mode 100644 OBB_IMPLEMENTATION.md diff --git a/OBB_IMPLEMENTATION.md b/OBB_IMPLEMENTATION.md deleted file mode 100644 index fb3b065ce..000000000 --- a/OBB_IMPLEMENTATION.md +++ /dev/null @@ -1,167 +0,0 @@ -# YOLO OBB (Oriented Bounding Box) Implementation - -## Overview -This document describes the implementation of YOLO Oriented Bounding Box (OBB) support in the make-sense image labeling tool. - -## What is YOLO OBB? -YOLO OBB (Oriented Bounding Box) is a format used by Ultralytics YOLO models for annotating rotated objects. Unlike regular axis-aligned bounding boxes, OBBs can be rotated to better fit objects at any angle, making them particularly useful for aerial/satellite imagery and other scenarios where objects are not always upright. - -### YOLO OBB Format Specification -According to the Ultralytics documentation, the YOLO OBB format is: -``` -class_index x1 y1 x2 y2 x3 y3 x4 y4 -``` - -Where: -- `class_index`: The class ID of the object -- `x1 y1`, `x2 y2`, `x3 y3`, `x4 y4`: Four corner points of the oriented bounding box -- All coordinates are normalized to [0, 1] range (divided by image width/height) - -## Implementation Details - -### 1. Core Type System -**Files Modified:** -- `src/data/enums/LabelType.ts` - Added `OBB = 'OBB'` to LabelType enum -- `src/store/labels/types.ts` - Added `LabelOBB` type and `labelOBBs` array to `ImageData` - -The `LabelOBB` type includes: -```typescript -export type LabelOBB = Annotation & { - vertices: IPoint[]; - isCreatedByAI: boolean; - status: LabelStatus; - suggestedLabel: string; -} -``` - -### 2. Label Toolkit Integration -**Files Modified:** -- `src/data/info/LabelToolkitData.ts` - Added OBB option to the label toolkit menu - -Users can now select "OBB" as a labeling mode alongside Rect, Point, Line, and Polygon. - -### 3. Rendering Engine -**Files Created:** -- `src/logic/render/OBBRenderEngine.ts` - Complete rendering engine for drawing and editing OBBs - -**Features:** -- Interactive drawing: Users draw OBBs by clicking 3 points: - 1. First point: Start of first edge - 2. Second point: End of first edge (defines one side of the box) - 3. Third point: Defines the width/height by projecting perpendicular to the first edge -- Vertex manipulation: Users can drag any of the 4 corners to reshape the OBB -- Visual feedback: Active and highlighted OBBs show anchors at corners -- Semi-transparent fill for better visibility - -**Files Modified:** -- `src/logic/actions/EditorActions.ts` - Integrated OBBRenderEngine into the editor - -### 4. Export Functionality -**Files Created:** -- `src/logic/export/OBBLabelsExporter.ts` - Exports OBB annotations in YOLO OBB format - -**Export Format:** -- Creates a .zip file containing .txt files (one per image) -- Each line in the .txt file represents one OBB annotation -- Format: `class_index x1 y1 x2 y2 x3 y3 x4 y4` -- All coordinates normalized to [0, 1] - -**Files Modified:** -- `src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx` - Added OBB export option -- `src/data/ExportFormatData.ts` - Added YOLO OBB format description - -### 5. Label Management -**Files Created:** -- `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.tsx` - List view for OBB labels -- `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.scss` - Styles for OBB list - -**Files Modified:** -- `src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx` - Added OBB tab and list -- `src/logic/actions/LabelActions.ts` - Added `deleteOBBLabelById()` and OBB support in visibility toggle -- `src/utils/LabelUtil.ts` - Added `createLabelOBB()` method -- `src/utils/ImageDataUtil.ts` - Initialize `labelOBBs: []` in ImageData creation - -## How to Use - -### Drawing OBBs: -1. Load your images into the tool -2. Create or select label classes -3. Select "OBB" from the label toolkit on the right sidebar -4. Click to place the first corner of your box -5. Click to place the second corner (defines one edge) -6. Click to define the width of the box (the tool auto-completes the rectangle) - -### Editing OBBs: -1. Click on an existing OBB to select it -2. Drag any of the 4 corner anchors to reshape the box -3. The OBB remains a quadrilateral but can be freely adjusted - -### Exporting: -1. Click "Export Labels" (or use the export action) -2. Select "OBB" as the label type -3. Choose "YOLO" as the export format -4. Download the .zip file containing your annotations - -### File Structure: -``` -annotations.zip -├── image1.txt -├── image2.txt -└── image3.txt -``` - -Each .txt file contains lines like: -``` -0 0.512 0.345 0.678 0.389 0.634 0.556 0.468 0.512 -1 0.234 0.123 0.345 0.234 0.298 0.401 0.187 0.290 -``` - -## Training with Ultralytics YOLO - -Once you have your OBB annotations, you can use them to train YOLO models: - -```python -from ultralytics import YOLO - -# Load a model -model = YOLO('yolo11n-obb.yaml') - -# Train the model -results = model.train(data='your_dataset.yaml', epochs=100, imgsz=640) -``` - -Your dataset.yaml should specify: -```yaml -path: /path/to/dataset -train: images/train -val: images/val - -names: - 0: class_name_1 - 1: class_name_2 -``` - -## Files Changed Summary - -### New Files Created (7): -1. `src/logic/render/OBBRenderEngine.ts` -2. `src/logic/export/OBBLabelsExporter.ts` -3. `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.tsx` -4. `src/views/EditorView/SideNavigationBar/OBBLabelsList/OBBLabelsList.scss` - -### Files Modified (11): -1. `src/data/enums/LabelType.ts` -2. `src/store/labels/types.ts` -3. `src/data/info/LabelToolkitData.ts` -4. `src/utils/LabelUtil.ts` -5. `src/logic/actions/EditorActions.ts` -6. `src/views/PopupView/ExportLabelsPopup/ExportLabelPopup.tsx` -7. `src/data/ExportFormatData.ts` -8. `src/utils/ImageDataUtil.ts` -9. `src/views/EditorView/SideNavigationBar/LabelsToolkit/LabelsToolkit.tsx` -10. `src/logic/actions/LabelActions.ts` - -## References -- [Ultralytics OBB Documentation](https://docs.ultralytics.com/datasets/obb/) -- YOLO OBB format uses 4 corner points normalized to [0, 1] -- Internally, YOLO processes in xywhr format (center x, center y, width, height, rotation)