mynode/packages/transcribe/README.md

# Installation and Setup

## Configure Docker for Transcribe

The transcribe server embeds the llama.cpp binary directly in the Docker image. The AI models must be downloaded separately and mounted as a volume.

### 1. Create data directory and download models

```shell
mkdir -p ./data/models
chmod 755 ./data
wget -O ./data/models/Model-7.6B-Q4_K_M.gguf https://huggingface.co/openbmb/MiniCPM-o-2_6-gguf/resolve/main/Model-7.6B-Q4_K_M.gguf
wget -O ./data/models/mmproj-model-f16.gguf https://huggingface.co/openbmb/MiniCPM-o-2_6-gguf/resolve/main/mmproj-model-f16.gguf
```

### 2. Configure environment

1. Copy `.env-transcribe-sample` to your Docker configuration directory.
2. Rename it to `.env-transcribe`.
3. Set `API_KEY` to a secure value.

### 3. Run the server

```shell
docker run --rm --env-file .env-transcribe -p 4567:4567 \
	-v ./data:/data \
	joplin/transcribe:amd64-latest
```

The container automatically creates the following inside `/data`:
- `images/` - uploaded images
- `models/` - AI models (you provide these)
- `queue.sqlite3` - job queue database

## Using Docker Compose

The minimal configuration is provided in `.env-sample` and `docker-compose.server.yml`.

1. Run `cp .env-sample .env`
2. Update any options you need in `.env`
3. Start the server:

   ```shell
   docker compose -f docker-compose.server.yml --profile full up --detached
   ```

For advanced configuration, refer to `.env-sample-transcribe`.

## Security

The transcribe container runs with these security measures:

- **Non-root user**: The application runs as the `transcribe` user, not root
- **Read-only filesystem**: The container filesystem is read-only (only `/app/packages/transcribe/images` and `/tmp` are writable)
- **Resource limits**: Memory and CPU limits prevent runaway processes
- **No Docker socket**: Unlike previous versions, no Docker socket mount is required

# Development Setup

## Testing

Integration tests requiring the full model **do not run by default (including on CI)**. Be cautious when modifying the model or prompts.
The disabled test is located at: `workers/JobProcessor.test.ts`.

Run all tests with:

```shell
yarn test-all
```

## Starting the Server

From `packages/transcribe`, run:

```shell
yarn start
```

### Environment variables

Required:
- `API_KEY`: Authentication key for API requests
- `DATA_DIR`: Base directory for all data (images, models, database)
- `HTR_CLI_BINARY_PATH`: Path to the llama-mtmd-cli binary

Optional:
- `QUEUE_DRIVER`: `sqlite` (default in Docker) or `pg` for PostgreSQL

The following paths are automatically derived from `DATA_DIR`:
- `$DATA_DIR/images` - uploaded images
- `$DATA_DIR/models` - AI models
- `$DATA_DIR/queue.sqlite3` - SQLite database (when using sqlite driver)

### GPU support

The server uses CPU by default. For NVIDIA GPU, run a CUDA-capable image with `--gpus all` and set `HTR_CLI_GPU_LAYERS=9999`.

```shell
docker run --rm --gpus all --env-file .env-transcribe -p 4567:4567 \
	-e HTR_CLI_GPU_LAYERS=9999 \
	-v ./data:/data \
	joplin/transcribe:gpu-latest
```

This needs NVIDIA Container Toolkit on the host. Use `0` or leave it unset for CPU.

### Native Windows without Docker

On Windows, CUDA GPU acceleration can run in Docker using the NVIDIA instructions above, or natively on the host. For a native run, use a Windows x64 CUDA-enabled build of llama-mtmd-cli.exe, then set HTR_CLI_GPU_LAYERS to the number of layers you want offloaded to the GPU.

### Native macOS/Metal

On Apple Silicon, GPU acceleration runs natively on the host (Docker can't expose Metal to containers). Use a macOS ARM64 build of llama-mtmd-cli with Metal support, then set HTR_CLI_GPU_LAYERS to the number of layers you want offloaded to the GPU.

### Native GPU configuration

You can download the binary from the official llama.cpp releases page: <https://github.com/ggml-org/llama.cpp/releases>.

Native GPU settings:

```env
HTR_CLI_BINARY_PATH=/path/to/llama-mtmd-cli
HTR_CLI_GPU_LAYERS=9999
```

With this setup, the server passes `-ngl 9999` to `llama-mtmd-cli`. GPU support comes from the selected binary.

# API Endpoints

All requests must include the `Authorization` header with the value set to your `API_KEY`.

## POST `/transcribe`

Creates a transcription job. The uploaded image is resized, stored on disk, and assigned to a job record in the database.

**Request Body:**

* **Content-Type**: `multipart/form-data`
* **Field**: `file` (required) – the image file to process

**Response:**

```json
{
	"jobId": "bcd2e633-eb10-44cb-a280-bf723238c12e"
}
```

**Example (cURL):**

```shell
curl --request POST \
	--url http://localhost:4567/transcribe \
	--header 'Authorization: api-key' \
	--header 'Content-Type: multipart/form-data' \
	--form file=@/home/js/Pictures/2025-07-24_17-42_1.png
```

## GET `/transcribe/{jobId}`

Fetches the result of a transcription job created with `POST /transcribe`.

**Request:**

* Requires a valid `jobId`.

**Example Responses:**

```json
{
	"id": "57ebd2e2-b496-40ab-9008-5f861bcb7858",
	"state": "created"
}
```

```json
{
	"id": "07f09553-f5e9-467e-b98d-406778e61969",
	"state": "active"
}
```

```json
{
	"id": "57ebd2e2-b496-40ab-9008-5f861bcb7858",
	"completedOn": "2025-06-11T18:20:22.000Z",
	"output": {
		"result": "markdown\r\n# Main title\r\n\r\nSome text here. This should take more than one line.\r\n\r\n## Sub title\r\n\r\n- One kind\r\n  - of list\r\n    - sub-item\r\n\r\n## Conclusion\r\n\r\nLet's finish here."
	},
	"state": "completed"
}
```

**Example (cURL):**

```shell
curl --request GET \
	--url http://localhost:4567/transcribe/57ebd2e2-b496-40ab-9008-5f861bcb7858 \
	--header 'Authorization: api-key'
```