yao-ocr

OCR Tools

Two tools for optical character recognition, supporting both VLM-OCR (vision language models) and traditional OCR APIs (Baidu, Google, Azure, PaddleOCR).

ocr_recognize

Extract text from images or PDF files using OCR.

Basic usage (plain text output):

tai tool ocr_recognize --source /path/to/image.png

With URL:

tai tool ocr_recognize --source https://example.com/document.jpg

Table extraction as Markdown:

tai tool ocr_recognize --source /path/to/table.png --type table --output_format markdown

Invoice structured extraction:

tai tool ocr_recognize --source /path/to/invoice.pdf --type invoice --output_format json

With specific provider:

tai tool ocr_recognize --source /path/to/doc.png --provider baidu

VLM-OCR with custom prompt:

tai tool ocr_recognize --source /path/to/doc.png --provider llm:qwen-ocr --prompt "只提取表格中的金额列"

PDF page range:

tai tool ocr_recognize --source /path/to/report.pdf --pages "1-5" --output_format markdown

Parameter	Type	Required	Description
source	string	yes	Image or PDF file path/URL to recognize
provider	string	no	LLM connector ID (llm:xxx) or OCR settings key (baidu/paddleocr/google/azure). Auto-selects if omitted
type	string	no	Recognition type (default: general). See type table below
output_format	string	no	text (default), json (with coordinates/fields), or markdown (structured)
mode	string	no	accurate (default, best quality) or standard (faster)
language	string	no	Language hint (ISO 639-1, e.g. en, zh, ja). Auto-detected if omitted
prompt	string	no	Custom instruction for VLM-OCR only, appended to system prompt. Ignored by traditional OCR
pages	string	no	PDF page range, e.g. 1-5 or 1,3,7. All pages if omitted
extra	JSON	no	Provider-specific parameters as a JSON object

Recognition types

Type	Description	Best output_format
general	General text (default)	text
table	Table extraction	markdown
handwriting	Handwritten text	text
document	Document layout parsing	markdown
invoice	Invoice (VAT)	json
receipt	Receipt / ticket	json
id_card	ID card	json
bank_card	Bank card	json
license	Business license	json
vehicle_license	Vehicle license	json
passport	Passport	json
license_plate	License plate	json

If the chosen provider does not support the requested type, it automatically degrades to general and annotates the response metadata with degraded_from. VLM-OCR supports all types via prompt adaptation.

ocr_providers

List available OCR providers and their supported recognition types.

tai tool ocr_providers

Returns a list of providers including VLM-OCR models (from LLM connectors with ocr capability) and traditional API providers (from OCR settings). Each entry includes id, name, type (vlm or traditional), and supported_types.

PDF support

Provider	PDF	Notes
Baidu	yes	pdf_file parameter
Azure	yes	Document Intelligence native support
PaddleOCR	yes	pdf + fileType=0
Google	no	Sync API does not support PDF
VLM (llm:)	no	Vision models accept images only

For providers that do not support PDF, use Baidu, Azure, or PaddleOCR instead.

Multi-page PDF response

Multi-page PDFs are automatically split page-by-page. Instead of printing all text, the tool returns a JSON summary with file paths for each page result:

{
  "source": "report.pdf",
  "total_pages": 10,
  "pages": 3,
  "results": [
    {"page": 1, "file": ".tool-tmp/ocr-a1b2c3d4/page-1.txt", "preview": "Invoice No: INV-001..."},
    {"page": 2, "file": ".tool-tmp/ocr-a1b2c3d4/page-2.txt", "preview": "Invoice No: INV-002..."},
    {"page": 3, "file": ".tool-tmp/ocr-a1b2c3d4/page-3.txt", "preview": "Invoice No: INV-003..."}
  ]
}

To read full content of a specific page, use cat:

cat .tool-tmp/ocr-a1b2c3d4/page-2.txt

Single-page PDFs and images return inline text as usual (no file indirection).

Guidelines

• Use output_format=text (default) when you just need the text content — simplest for LLM processing
• Use output_format=json for structured types (invoice, id_card, etc.) to get key-value fields
• Use output_format=markdown for documents and tables to preserve layout
• The prompt parameter only works with VLM-OCR providers; traditional OCR ignores it
• For structured document types (invoice, receipt, id_card, etc.), prefer json output to get fields with key-value pairs
• Use ocr_providers first to check which providers are available and what types they support
• Multi-page PDFs return a JSON summary with temporary file paths; use cat <file> to read specific pages
• Google Vision and VLM providers do not support PDF input directly; use Baidu, Azure, or PaddleOCR for PDF files