feat: support offsetless BYO tokenizers - #142

Bugbot Analysis Progress (4m 32s elapsed)

✅ Gathered PR context (3s)
✅ Completed bug detection (4m 27s)
✅ Posted analysis results (2s)

Final Result: Bugbot completed review - no issues found! ✅

Request ID: serverGenReqId_5f71c8b3-fc73-41a2-ae14-cea2300113a4