Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

Text2Vision-PT

面向 Encoder-Free OCR 多模态大模型的在线视觉预训练数据引擎

用有限的文本语料,在线产生近乎无限的视觉观测,并根据训练任务动态生成不同长度的视觉 token 序列。

版本

v0.2.0 (2026-07-14)

  • 动态 canvas (bucket-based sizing, 3841024 × 64512)
  • 多任务支持: Full OCR (A) + Optical Continuation (B) + Text Replay (D)
  • RenderUnit 不可变模式: target_text 渲染前确定,绝不修改
  • NeedsSplit/TooWide 异常处理
  • 与 HainaOCR-NativePixel (modelv3) 训练框架对接

项目结构

text2vision_pt/
├── configs/              # 配置文件 (data, render, tasks, train)
├── preprocessing/        # 预处理: gzip读取, HTML清洗, LaTeX公式解析, 文档切分
├── rendering/            # 渲染: Chromium常驻, KaTeX, 动态canvas, 样式采样
├── tasks/                # 任务系统: Full OCR, Optical Continuation, Span Reconstruction, Text Replay
├── dataset/              # Dataset + Collator + Bucket Sampler
├── model/                # 动态2D位置编码, 视觉token工具
├── qc/                   # 质量控制: benchmark, preview, validate
├── tests/                # 单元测试
└── doc/                  # 设计文档

四种训练任务

任务 权重 描述
A: Full OCR 60% image → text 图片到全文识别
B: Optical Continuation 25% prefix + image → suffix 文字+图片→续写
C: Span Reconstruction — context + image → span 上下文+图片→重建
D: Text Replay 15% text → text 纯文本续写,防止语言退化

数据流

原始文本 → 预处理 → JSONL shards
    → TaskSampler (在线分配任务)
    → HtmlOCRRenderer (Chromium + KaTeX 在线渲染)
    → dynamic canvas (内容自适应尺寸)
    → uint8 Tensor → GPU bf16

安装

pip install playwright pillow numpy torch beautifulsoup4 transformers
playwright install chromium

对接 HainaOCR-NativePixel

from dataset.render_dataset import RenderDataset
from dataset.render_collator import RenderCollator
from rendering.html_ocr_renderer import HtmlOCRRenderer, RenderConfig, BrowserConfig

renderer = HtmlOCRRenderer(
    RenderConfig(output_mode="uint8"),
    BrowserConfig(executable_path="/path/to/chromium", katex_dist="/path/to/katex/dist"),
)
dataset = RenderDataset(manifest_files, renderer=renderer, base_seed=42)
collator = RenderCollator(tokenizer=tokenizer)
loader = DataLoader(dataset, batch_size=4, collate_fn=collator)

License

Internal research use.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages