# UDF Semantics

This page explains how `src/geneva/transformer.py` defines UDFs and runs them.

## Responsibilities
- Provide the `@udf` decorator and `UDF` wrapper class.
- Infer or validate output data types.
- Map input columns to function parameters, including dotted struct paths.
- Support scalar, array, and RecordBatch execution modes.
- Attach resource hints and error handling configuration.
- Generate checkpoint identifiers for repeatable backfills.

## UDF Modes
The UDF wrapper supports three execution styles.

- Scalar UDFs return Python scalars and are invoked per row.
- Array UDFs accept `pa.Array` inputs and return a `pa.Array`.
- RecordBatch UDFs accept a `pa.RecordBatch` and return a `pa.Array`.

Mode detection is based on annotations.

- If a single parameter is annotated as `pa.RecordBatch`, the UDF is RecordBatch mode.
- If the return annotation is `pa.Array` or a `pa.DataType` instance and inputs are `pa.Array`, the UDF is array mode.
- Otherwise the UDF is scalar mode.

## Input Column Mapping
- If `input_columns` is not provided, parameter names define input columns.
- RecordBatch UDFs must not set `input_columns`.
- Dotted paths like `info.left` map to struct fields inside a column.

## Type Inference And Validation
- Scalar UDFs can infer `data_type` from the return annotation.
- Array and RecordBatch UDFs must provide an explicit `data_type`.
- `validate_against_schema` checks column existence and type compatibility.

## List And ndarray Handling
When the input column is list-like, the wrapper can avoid extra copies.

- Parameters annotated as `list[...]` receive Python lists.
- Parameters annotated as `numpy.ndarray` receive numpy arrays, zero-copy when possible.
- List and ndarray annotations require Arrow `list`, `large_list`, or `fixed_size_list` columns.

## Checkpointing And Versioning
- Default `version` is an md5 hash of the pickled function.
- `checkpoint_key` can be overridden to control stability across revisions.
- `checkpoint_prefix` encodes dataset uri, target column, filter, and source hash.

## Execution Path
- `UDF.__call__` dispatches based on the UDF mode.
- Scalar UDFs iterate rows from a RecordBatch and build a result array.
- `BACKFILL_SELECTED` is honored to skip unselected rows.
- RecordBatch UDFs can accept list-of-dict batches during blob handling, and `BlobFile` values are materialized before invocation.

## Resource Hints
The `UDF` object carries scheduling hints for Ray.

- `num_cpus` and `num_gpus` describe required compute.
- `cuda` is deprecated and normalizes to `num_gpus`.
- `checkpoint_size`, `min_checkpoint_size`, and `max_checkpoint_size` tune batch sizing.

## Error Handling
- `@udf` accepts `on_error` or `error_handling`, not both.
- `on_error` is resolved into a full `ErrorHandlingConfig`.

## Example
```python
import pyarrow as pa
import geneva

@geneva.udf(data_type=pa.int32())
def double(x: int) -> int:
    return x * 2
```
