# About BPM [about-bpm]
> Use this chapter to understand what BPM is, its main capabilities, and how it helps organizations automate business processes and manage information efficiently.

BPM is a low-code platform that enables organizations to automate business processes, manage information efficiently, and facilitate decision-making. The platform centralizes and simplifies daily operations by providing tools for creating and executing business processes, generating queries and reports, and integrating with external systems.

## Main Capabilities

**Business Process Automation**: BPM integrates with BPMN and Flowable, allowing you to design, model, and execute complex workflows that automate repetitive and critical business tasks. Processes can include user tasks, script tasks, forms, and automated actions, minimizing errors and reducing dependency on manual work.

**Data Management**: The platform provides powerful tools for querying and displaying data:
- **Queries**: Retrieve and display data from SQL databases, Collections (NoSQL), or JSON/REST APIs in interactive tables
- **Reports**: Generate documents in PDF, Excel, HTML, and DOCX formats for analysis and presentation
- **Collections**: Define and manage custom data structures (similar to tables) for transactional data

**Services and Forms**: Create independent forms and services that execute endpoints, trigger file downloads, display queries, or initiate processes. These services can be accessed directly by end users based on their roles and permissions.

**Artificial Intelligence**: Integrate AI assistants that can interact with data and services through conversational interfaces (e.g., Telegram). These assistants are backed by Vector Stores for storing and querying large volumes of data, enabling quick and relevant responses.

**Custom Pages**: Create microsites and dashboards using HTML, Python/JavaScript scripts, or Velocity templates, providing dynamic content and real-time data visualization.

## Benefits

- **Low-Code Development**: Create processes and applications without extensive custom development
- **Centralized Management**: All processes, data, and services in one platform
- **Real-Time Information**: Access to data and reports in real-time based on user roles
- **Integration**: Connect with external systems, databases, and APIs
- **Scalability**: Adapt to changing business needs quickly and efficiently

This guide provides comprehensive technical documentation for all artifact types in the BPM platform. Each chapter covers a specific artifact type with complete descriptions, YAML/JSON structures, configuration options, and practical examples.

---

# Queries [queries]
> Use this chapter when you need to retrieve and display data in tables, configure filters, add actions to rows, or query SQL databases, Collections, or JSON/REST APIs.

Queries define how data is retrieved, displayed, and interacted with. They are defined in YAML.

**Related Topics:** Queries can use [Templates](#templates) for custom rendering, [Post Processors](#post-processors) for data transformation, [Value Providers](#value-providers) for filter dropdowns, and [Kanban](#kanban) for visual board views.

## Data Source Types

Queries support three **mutually exclusive** data source types. The type is determined by the `dataSource` field and `dataSourceOptions` configuration:

1. **SQL Database**: Query data from a SQL database connection
2. **Collections**: Query data from a [Collection](#collections) (NoSQL document store)
3. **JSON/REST Sources**: Query data from external APIs or scripts that return JSON (via URL, curl, or script execution)

**⚠️ Important:** These data source types are **mutually exclusive**. You cannot combine them in a single query.

## YAML Structure

```yaml
name: "PROJECT/QUERY_ID"        # Unique identifier (optional, usually derived from filename)
caption: "Display Title"        # Visible title
icon: "ADDRESS_BOOK"            # Icon name (Vaadin icons or custom)
dataSource: "NORTHWIND"         # Data source: connection name (SQL), "run:SCRIPT" (JSON), "curl ..." (JSON), or URL (JSON)
view: "SELECT * FROM table"     # SQL query (for SQL), script/URL (for JSON sources), or omitted (for Collections)
viewBindings:                   # Parameters for the view (use "?" as placeholder)
  - param1
stringFormat: "%{id} - %{caption}" # Format string for Value Providers (uses %{fieldName} placeholders)
template: |                     # Polymer/Vaadin template for custom column rendering (see Templates chapter)
  <div>[[item.field1]] - [[item.field2]]</div>
defaultFilterExpression: "column1 LIKE '%' || ? || '%' OR column2 LIKE '%' || ? || '%'" # SQL expression for user_input filter
columns:                        # Column definitions
  - name: "field_name"
    caption: "Header"
    fieldType: "string|integer|decimal|date|datetime|boolean"
    format: "format_string"     # e.g., "#,##0.00" or 0 (for integers), compatible with Java's NumberFormat or SimpleDateFormat
    width: "100px"              # or "flex" (default), or numeric (1, 2, etc.) for relative sizing
    align: "CENTER|START|END"
keys:                           # Primary key fields
  - id
sortables:                      # Sortable fields
  - name
targetUsers: []                 # Specific users allowed (empty = all)
targetGroups:                   # Groups allowed to access
  - "ADMIN"
menus:                          # Menu items where query appears
  - "Clientes"
filterGroup:                    # Filter configuration
  callToAction: "Filter"
  name: "Filtros"
  caption: "Filtros"
  description: "Optional description text"  # Description shown in filter form
  header: "Optional header HTML"            # HTML header shown above filters
  summary: "Optional summary HTML"         # HTML summary shown below filters
  formDisposition: "field1, field2:2, field3"  # Field layout: comma-separated, :N for colspan
  defaultFilter:                 # Default filter values (JSON object)
    status: "active"             # Static default value
    start_date: ";new Date().getTime() - 30*24*60*60*1000"  # JavaScript expression (starts with ;)
    # Values starting with ; are evaluated as JavaScript with access to 'context'
  uniqueItems: false             # Whether filter items must be unique (for subgroups)
  filters:
    - name: "param1"              # Filter name (must match viewBindings or column name)
      caption: "Label"             # Display label for the filter field
      fieldType: "string"          # Data type: string, integer, number, date, datetime, boolean, or null
      required: true               # Whether the filter is required
      possibleValues: ["A", "B"]  # Static list of allowed values (alternative to valueProvider)
      valueProvider: "PROJECT/VP_ID" # Value Provider query for dynamic dropdown (alternative to possibleValues)
      expression: "column = ::value::" # SQL expression with ::value:: placeholder (see Filter Expressions section)
      stringFormat: "%{id} - %{caption}" # Format string for value provider display
      validationScript: "run:PROJECT/SCRIPT_ID" # Script to validate filter value before applying
      dependents: ["param2"]       # Other filter names that depend on this filter (triggers refresh)
details:                        # Queries opened as drill-down (detail views)
  - "PROJECT/DETAIL_QUERY_ID"
reports:                        # Reports available from query
  - "PROJECT/REPORT_ID"
services:                       # Services available from query
  - "PROJECT/SERVICE_ID"
scripts:                        # Script actions available from query
  - "PROJECT/SCRIPT_ID"
postProcessors:                 # Scripts executed for each row retrieved (see Post Processors chapter)
  - "PROJECT/POST_PROCESSOR_SCRIPT"
context:                        # Initial context values (JSON object)
  param1: "value1"
  param2: 123
initContextScript: |            # JavaScript script to initialize context
  context.param1 = "computed_value";
  context.timestamp = new Date().getTime();
selectionScript: |              # Script executed when selection changes
  #!python3
  import redflagbpm
  bpm = redflagbpm.BPMService()
  bpm.reply(f"Selected {len(bpm.context.selection)} items")
dataSourceOptions:              # Advanced data source options
  collection: "PROJECT/COLLECTION_ID" # For Collection-based queries
  onOpen: "SET SESSION group_concat_max_len = 10240" # SQL executed when opening connection
  onClose: "CLOSE ALL"          # SQL executed when closing connection
  pageSize: 100                 # Custom page size (overrides options.pageSize)
  timeout: 30000                # Query timeout in milliseconds
  filter: ".items[]"            # JQ expression to filter JSON results (for REST/script sources)
  where: "status = 'active'"    # Additional WHERE clause
  orderBy: "created_date DESC"  # Additional ORDER BY clause
  canDelete: true               # Enable delete operations (CRUD)
  canCreate: true               # Enable create operations (CRUD)
  canUpdate: true               # Enable update operations (CRUD)
options:                        # Advanced options
  crud: true                    # Enable default CRUD actions
  collection: "PROJECT/COLLECTION_ID" # For CRUD queries on Collections
  pageSize: 50                  # Rows per page
  autorefresh: 60               # Seconds
  exportable: true              # Enable export functionality
  themeVariants:                # Visual styles for the grid (array of GridVariant values)
    - LUMO_COMPACT              # Makes grid more dense by reducing header/row heights and column spacing
    - LUMO_ROW_STRIPES          # Alternating row colors for better readability
    - LUMO_NO_BORDER            # Removes outer border of the grid
    - LUMO_NO_ROW_BORDERS      # Removes horizontal borders between rows
    - LUMO_COLUMN_BORDERS       # Adds vertical borders between columns
    - LUMO_WRAP_CELL_CONTENT    # Wraps cell content, allowing rows to expand height as needed
  htmlColumns:                  # Columns containing HTML
    - description
  frozenColumns:                # Columns to freeze
    - id
  classColumn: "status_class"   # Column name whose value is used as CSS class for each row (see Row Styling section)
debug: false                    # Enable debug mode (prints SQL to console)
waitForFilter: true            # Wait for user to apply filter before executing query
exportable: true               # Enable export functionality (can also be at root level)
autorefresh: 60                # Auto-refresh interval in seconds (can also be at root level)
```

## Columns and Hidden Fields

**Displayed Columns:**

Only columns defined in the `columns` array are displayed in the query grid. Each column in the array must have:
- `name`: Field name (must match a column in the SQL SELECT, Collection field, or JSON property)
- `caption`: Header text displayed in the grid
- `fieldType`: Data type for proper formatting

**Hidden Columns:**

To have columns that are not displayed but are still available in the data:

**For SQL Queries:**
- Include the field in your SQL `SELECT` statement
- **Do NOT** add it to the `columns` array

**For Collection Queries:**
- The field exists in the Collection document
- **Do NOT** add it to the `columns` array

**For JSON/REST Queries:**
- The field exists in the JSON response
- **Do NOT** add it to the `columns` array
- The field will be available in:
  - `bpm.context.selection` (for action scripts)
  - `selectionScript` context
  - `postProcessors` row data
  - `keys` array (can be used as primary key even if hidden)

**Example:**

```yaml
view: |
  SELECT 
    order_id,           # Visible column
    customer_id,        # Visible column
    internal_notes,     # Hidden column (not in columns array)
    status_code,        # Hidden column (not in columns array)
    created_by          # Hidden column (not in columns array)
  FROM orders
columns:
  - name: order_id
    caption: "Order ID"
    fieldType: integer
  - name: customer_id
    caption: "Customer"
    fieldType: string
  # internal_notes, status_code, and created_by are NOT in columns array
  # but they are available in scripts via bpm.context.selection
keys:
  - order_id           # Can use visible or hidden fields
sortables:
  - order_id
```

**Using Hidden Fields in Scripts:**

Hidden fields are fully accessible in action scripts and other contexts:

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Access hidden fields from selection
if bpm.context.selection:
    for item in bpm.context.selection:
        order_id = item.get('order_id')        # Visible field
        internal_notes = item.get('internal_notes')  # Hidden field - still accessible!
        status_code = item.get('status_code')  # Hidden field - still accessible!
        created_by = item.get('created_by')   # Hidden field - still accessible!
        
        # Use hidden fields in logic
        if status_code == 'INTERNAL':
            # Process internal orders
            pass
```

**Important Notes:**

- Fields in `keys` can be hidden (not in `columns`) - they're still used as primary keys
- Fields in `sortables` should typically be visible, but hidden fields can also be sorted
- Hidden fields are included in exported data (Excel, etc.) if the export includes all fields
- Hidden fields are available in `postProcessors` for each row
- Hidden fields can be used in `filterGroup` filters if referenced by name

## Filter Groups

The `filterGroup` object configures filters for the query. Filters allow users to narrow down the data displayed in the query grid.

**Filter Group Configuration:**

```yaml
filterGroup:
  callToAction: "Filter"         # Button label for applying filters
  name: "Filtros"                # Internal name for the filter group
  caption: "Filtros"             # Display title for the filter form
  description: "Optional description text"  # Description shown in filter form
  header: "Optional header HTML"            # HTML header shown above filters
  summary: "Optional summary HTML"         # HTML summary shown below filters
  formDisposition: "field1, field2:2, field3"  # Field layout: comma-separated, :N for colspan
  defaultFilter:                 # Default filter values (JSON object)
    status: "active"             # Static default value
    start_date: ";new Date().getTime() - 30*24*60*60*1000"  # JavaScript expression (starts with ;)
    # Values starting with ; are evaluated as JavaScript with access to 'context'
  uniqueItems: false             # Whether filter items must be unique (for subgroups)
  filters:
    - name: "param1"              # Filter name (must match viewBindings or column name)
      caption: "Label"             # Display label for the filter field
      fieldType: "string"          # Data type: string, integer, number, date, datetime, boolean, or null
      required: true               # Whether the filter is required
      possibleValues: ["A", "B"]  # Static list of allowed values (alternative to valueProvider)
      valueProvider: "PROJECT/VP_ID" # Value Provider query for dynamic dropdown (alternative to possibleValues)
      expression: "column = ::value::" # SQL expression with ::value:: placeholder (see Filter Expressions section)
      stringFormat: "%{id} - %{caption}" # Format string for value provider display
      validationScript: "run:PROJECT/SCRIPT_ID" # Script to validate filter value before applying
      dependents: ["param2"]       # Other filter names that depend on this filter (triggers refresh)
```

**Filter Field Properties:**

**`stringFormat`:**

When using a `valueProvider` for filter dropdowns, `stringFormat` customizes how values are displayed. Uses the same format as Value Providers:

```yaml
filters:
  - name: customer_id
    caption: "Customer"
    fieldType: string
    valueProvider: "PROJECT/VP_CUSTOMERS"
    stringFormat: "%{id} - %{caption}"  # Displays: "123 - Company Name"
```

**`validationScript`:**

Script executed to validate the filter value before applying it. Can reference a script using `run:PROJECT/SCRIPT_ID` or inline code:

```yaml
filters:
  - name: order_date
    caption: "Order Date"
    fieldType: date
    validationScript: "run:PROJECT/VALIDATE_DATE"  # External script
    # or inline:
    # validationScript: |
    #   #!python3
    #   import redflagbpm
    #   bpm = redflagbpm.BPMService()
    #   if bpm.context.input.get('order_date') < some_minimum_date:
    #     raise '[[[Date must be in the future]]]'
```

**`dependents`:**

List of other filter names that depend on this filter. When this filter changes, dependent filters are refreshed:

```yaml
filters:
  - name: country
    caption: "Country"
    fieldType: string
    valueProvider: "PROJECT/VP_COUNTRIES"
  - name: city
    caption: "City"
    fieldType: string
    valueProvider: "PROJECT/VP_CITIES"
    dependents: ["country"]  # City dropdown refreshes when country changes
```

**`required`:**

Whether the filter must have a value before the query can execute:

```yaml
filters:
  - name: start_date
    caption: "Start Date"
    fieldType: date
    required: true  # Query won't execute until this filter has a value
```

**`possibleValues`:**

Static list of allowed values for the filter dropdown:

```yaml
filters:
  - name: status
    caption: "Status"
    fieldType: string
    possibleValues: ["Pending", "Processing", "Shipped", "Delivered"]
```

**`valueProvider`:**

Reference to a Value Provider query that dynamically populates the filter dropdown:

```yaml
filters:
  - name: customer_id
    caption: "Customer"
    fieldType: string
    valueProvider: "PROJECT/VP_CUSTOMERS"
    stringFormat: "%{id} - %{caption}"
```

**Default Filter Expression:**

The `defaultFilterExpression` defines a SQL expression used when filtering by `user_input` (the general search box). If not specified, the system searches across all columns.

```yaml
defaultFilterExpression: "name LIKE '%' || ? || '%' OR description LIKE '%' || ? || '%'"
```

When a user types in the search box, the `?` placeholders are replaced with the search value, and the expression is applied to filter results.

**Note:** `defaultFilterExpression` uses `?` placeholders directly (not `::value::`), as it's processed differently from filter group expressions.

## Actions (`details`, `reports`, `services`, `scripts`)

Queries can trigger other artifacts based on selection. These are referenced by ID in separate arrays:

- **`details`**: Array of query IDs opened as drill-down views when a row is selected.
- **`reports`**: Array of report IDs that can be generated from the selection.
- **`services`**: Array of service IDs that can be opened with the selection.
- **`scripts`**: Array of script IDs that execute actions on the selection.

**Action Properties (in the target artifact `.config`):**

Scripts used as actions in queries can be configured with the following properties:

```yaml
properties:
  caption: "Action Name"        # Display label for the action button
  icon: "EDIT"                  # Icon name (Vaadin icons or custom)
  type: "call"                  # Action type: "create", "update", "delete", "query", "display", "open", "call" (default)
  selectionScope: 1             # Selection requirement: 0 (none), 1 (single), -1 (any), -2 (multiple), -3 (group), -4 (filter)
  displayHints:                 # Array of display hints for UI behavior
    - "default"                 # Mark as default action (double-click)
    - "main"                    # Show as main button in toolbar
    - "confirm"                 # Require confirmation before execution
    - "small"                   # Display in small size (for type: "display")
    - "medium"                  # Display in medium size (for type: "display")
    - "large"                   # Display in large size (for type: "display")
    - "x-large"                 # Display in extra-large size (for type: "display")
    - "refresh-on-close"        # Refresh query when display closes
  reusable: false               # Keep action open after execution (default: false)
  asynchronous: false           # Execute asynchronously (default: false)
  groupName: "group1"           # Group name for organizing actions
  shortcuts:                    # Keyboard shortcuts
    ok: "Enter"
    cancel: "Escape"
  name: "SCRIPT_NAME"          # Optional script name (legacy)
  inputSchema:                  # Optional form schema
    type: object
    properties: {...}
```

**Action Types (`type`):**

- **`call`** (default): Execute a secondary action/script. Most common for custom scripts.
- **`create`**: Treated as a "New" button. Opens form for creating new records.
- **`update`**: Treated as an "Edit" button. Opens form for editing selected record.
- **`delete`**: Treated as a "Delete" button. Automatically shows confirmation dialog.
- **`query`**: Treated as a "Query" button. Opens a query view.
- **`display`**: Display detail in screen. Opens a display view (can use displayHints for size).
- **`open`**: Open in a new window. Opens action in a new window/tab.

**Display Hints (`displayHints`):**

Display hints control the UI behavior and appearance of actions:

- **`default`**: Marks the action as the default action. This action is executed on double-click (unless overridden by `defaultAction` in query options).
- **`main`**: Shows the action as a main button in the toolbar (prominent placement).
- **`confirm`**: Requires user confirmation before executing the action. A confirmation dialog is shown. Actions with `type: "delete"` automatically show confirmation.
- **`small`**: For `type: "display"` actions, displays in small size.
- **`medium`**: For `type: "display"` actions, displays in medium size (default for display).
- **`large`**: For `type: "display"` actions, displays in large size.
- **`x-large`**: For `type: "display"` actions, displays in extra-large size.
- **`refresh-on-close`**: For `type: "display"` actions, refreshes the parent query when the display view is closed.

**Selection Scope (`selectionScope`):**

Controls how many rows must be selected for the action to be available:

- **`0`**: No selection required (action always available)
- **`1`**: Single row must be selected (default)
- **`-1`**: Any number of rows (0 or more)
- **`-2`**: Multiple rows (2 or more)
- **`-3`**: Group selection (continuous group)
- **`-4`**: Filter selection (sends filter instead of selected rows)

**Examples:**

```yaml
# Simple action with confirmation
properties:
  caption: "Delete Order"
  icon: "TRASH"
  type: "delete"
  selectionScope: 1

# Default action (double-click)
properties:
  caption: "View Details"
  icon: "EYE"
  type: "display"
  displayHints:
    - "default"
    - "main"
    - "large"

# Action that refreshes on close
properties:
  caption: "Edit Order"
  icon: "EDIT"
  type: "update"
  displayHints:
    - "refresh-on-close"
  selectionScope: 1

# Multiple selection action
properties:
  caption: "Bulk Update"
  icon: "CHECK_CIRCLE"
  type: "call"
  selectionScope: -2  # Requires multiple selections
```

## Selection Script

The `selectionScript` is executed automatically whenever the user's selection changes in the query grid. It receives:

- `bpm.context.selection`: Array of selected row objects (always an array, regardless of selectionScope)
- `bpm.context.inputData`: Current filter values

The script can display feedback in the query footer (HTML or plain text).

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()
selected_count = len(bpm.context.selection)
if selected_count > 0:
    bpm.reply(f"<strong>{selected_count} items selected</strong>")
else:
    bpm.reply("No items selected")
```

## Context and Initialization

**Context Object:**

The `context` field provides initial values that are merged into the query's execution context. These values are available to:
- `viewBindings` parameters
- Filter expressions
- Post processors
- Selection scripts

```yaml
context:
  department: "Sales"
  year: 2024
  active: true
```

**Init Context Script:**

The `initContextScript` is a JavaScript snippet executed to compute or initialize context values dynamically. It has access to:
- `context`: The context object (can be modified)
- `initContext`: Alias for `context`

```javascript
// Compute values based on current date
context.currentYear = new Date().getFullYear();
context.startDate = new Date(context.currentYear, 0, 1).getTime();
context.endDate = new Date(context.currentYear, 11, 31).getTime();

// Set default filter values
context.status = "active";
```

The script runs before the query executes, allowing dynamic context initialization.

## Data Source Configuration

### 1. SQL Database Queries

**Configuration:**
- `dataSource`: Database connection name (e.g., `"NORTHWIND"`, `"POSTGRES"`)
- `view`: SQL SELECT query
- `viewBindings`: Optional parameters for the SQL query (use `?` as placeholder)

**Example:**
```yaml
dataSource: "NORTHWIND"
view: |
  SELECT 
    order_id,
    customer_id,
    order_date,
    total_amount
  FROM orders
  WHERE status = ?
viewBindings:
  - status
```

**SQL-Specific Options:**
```yaml
dataSourceOptions:
  onOpen: "SET SESSION group_concat_max_len = 10240"  # SQL executed when connection opens
  onClose: "COMMIT"                                   # SQL executed when connection closes
  where: "deleted = false"                            # Additional WHERE conditions
  orderBy: "created_date DESC"                        # Additional ORDER BY clause
  pageSize: 100                                       # Override default page size
  timeout: 30000                                      # Query timeout in milliseconds
  canDelete: true                                     # Enable delete operations (CRUD)
  canCreate: true                                     # Enable create operations (CRUD)
  canUpdate: true                                     # Enable update operations (CRUD)
```

**Filter Expressions:** SQL queries support filter expressions with `::value::` placeholders (see [Filter Expressions](#filter-expressions) section).

### 2. Collection Queries

**Configuration:**
- `dataSourceOptions.collection`: Collection ID (e.g., `"PROJECT/COLLECTION_ID"`)
- `view`: **Omitted** (not used for Collections)
- `options.collection`: Alternative location for collection ID (for CRUD queries)

**Example:**
```yaml
dataSourceOptions:
  collection: "PROJECT/ORDERS_COLLECTION"
# view is not needed for Collections
```

**Collection-Specific Options:**
```yaml
dataSourceOptions:
  collection: "PROJECT/COLLECTION_ID"  # Required: Collection to query
  pageSize: 100                        # Override default page size
  timeout: 30000                       # Query timeout in milliseconds
```

**CRUD Queries on Collections:**
```yaml
options:
  crud: true                           # Enable CRUD operations
  collection: "PROJECT/COLLECTION_ID" # Collection for CRUD operations
```

**Filter Expressions:** Collection queries use JSONB operators. Filter expressions work with PostgreSQL JSONB syntax.

**Note:** Collections are stored in PostgreSQL as JSONB documents. The query system uses JSONB operators to filter and query the documents.

### 3. JSON/REST Source Queries

These queries retrieve data from external sources that return JSON. There are three ways to specify JSON sources:

#### A. Script Execution (`run:SCRIPT_ID`)

**Configuration:**
- `dataSource`: `"run:PROJECT/SCRIPT_ID"`
- `view`: **Omitted** (script is executed directly)

**Example:**
```yaml
dataSource: "run:PROJECT/FETCH_ORDERS_SCRIPT"
# Script must return JSON array or object
```

#### B. REST API URL

**Configuration:**
- `dataSource`: Valid HTTP/HTTPS URL
- `view`: **Omitted** (URL is fetched directly)

**Example:**
```yaml
dataSource: "https://api.example.com/v1/orders"
```

#### C. CURL Command

**Configuration:**
- `dataSource`: CURL command starting with `"curl "`
- `view`: **Omitted** (curl command is executed)

**Example:**
```yaml
dataSource: "curl -X GET https://api.example.com/v1/orders -H 'Authorization: Bearer ${token}'"
```

**JSON Source Options:**
```yaml
dataSourceOptions:
  filter: ".items[] | select(.status == 'active')"  # JQ expression to filter JSON results
  pageSize: 100                                      # Override default page size
  timeout: 30000                                     # Request timeout in milliseconds
```

**JSON Filtering with JQ:**
The `filter` option uses JQ expressions to transform or filter the JSON response:

```yaml
dataSourceOptions:
  # Extract items from nested structure
  filter: ".items[]"
  
  # Filter by condition
  filter: ".items[] | select(.status == 'active')"
  
  # Transform data
  filter: ".items[] | {id: .order_id, name: .customer_name, total: .amount}"
```

**Filter Expressions:** JSON/REST queries **do not support SQL filter expressions**. Filters work on the JSON data structure after retrieval.

**Important Notes:**
- The script/URL/curl must return valid JSON (array or object)
- For arrays, each element becomes a row
- For objects, the object becomes a single row
- Use `dataSourceOptions.filter` with JQ expressions to transform the JSON structure
- Filter expressions in `filterGroup` work on the JSON fields after retrieval

## Data Source Type Detection

The system determines the data source type in the following order:

1. **Collection**: If `dataSourceOptions.collection` or `options.collection` is present, or `options.crud` is `true`
2. **Script**: If `dataSource` starts with `"run:"`
3. **REST**: If `dataSource` is a valid URL (starts with `http://` or `https://`)
4. **CURL**: If `dataSource` starts with `"curl "`
5. **SQL**: Default (if none of the above match)

**⚠️ Important:** Only one data source type can be used per query. Mixing configurations will result in unexpected behavior.

**Quick Reference:**

| Data Source Type | `dataSource` Value | `view` Field | `dataSourceOptions` |
|-----------------|-------------------|--------------|---------------------|
| **SQL** | Database connection name | SQL SELECT query | SQL-specific options |
| **Collection** | Any (ignored) | Omitted | `collection: "PROJECT/ID"` |
| **Script (JSON)** | `"run:PROJECT/SCRIPT_ID"` | Omitted | JSON-specific options |
| **REST (JSON)** | `"https://..."` | Omitted | JSON-specific options |
| **CURL (JSON)** | `"curl ..."` | Omitted | JSON-specific options |

## Filter Expressions

**⚠️ Note:** Filter expressions with `::value::` placeholders are **only supported for SQL and Collection queries**. JSON/REST source queries do not support SQL-style filter expressions.

The `expression` field in filter definitions allows you to customize how filter values are applied to queries. This provides fine-grained control over filtering logic.

**For SQL Queries:**
Filter expressions use SQL syntax and are inserted into the WHERE clause of the SQL query.

**For Collection Queries:**
Filter expressions use PostgreSQL JSONB operators to query JSONB documents.

**For JSON/REST Queries:**
Filter expressions are **not supported**. Use `dataSourceOptions.filter` with JQ expressions to filter JSON data, or rely on client-side filtering in `filterGroup`.

**Basic Expression Syntax:**

The `expression` field accepts SQL conditions that will be inserted into the WHERE clause. Use the `::value::` placeholder to indicate where the filter value should be inserted.

**With `::value::` Placeholder:**

When `expression` contains `::value::`, the placeholder is replaced with SQL parameter placeholders `(?)`, and the filter value is bound as a parameter:

```yaml
filters:
  - name: customer_name
    caption: "Customer Name"
    fieldType: string
    expression: "LOWER(company_name) LIKE LOWER('%' || ::value:: || '%')"
```

**Multiple `::value::` Placeholders:**

If the expression contains multiple `::value::` placeholders, all are replaced with the same filter value:

```yaml
filters:
  - name: search_term
    caption: "Search"
    fieldType: string
    expression: "(company_name LIKE '%' || ::value:: || '%' OR contact_name LIKE '%' || ::value:: || '%')"
```

This generates: `(company_name LIKE '%' || ? || '%' OR contact_name LIKE '%' || ? || '%')` with the same value bound to both parameters.

**Without `::value::` Placeholder:**

If `expression` doesn't contain `::value::`, the filter value is passed as a single parameter:

```yaml
filters:
  - name: status
    caption: "Status"
    fieldType: string
    expression: "status = ?"
```

**Conditional Expressions:**

You can use SQL functions and operators in expressions:

```yaml
filters:
  - name: start_date
    caption: "Start Date"
    fieldType: date
    expression: "order_date >= ::value::"
  
  - name: price_range
    caption: "Minimum Price"
    fieldType: number
    expression: "unit_price >= ::value:: AND unit_price <= ::value:: * 2"
```

**Fixed Conditions (No Field Type):**

If `fieldType` is `null` or omitted, the expression is executed directly without any value binding. This is useful for fixed conditions:

```yaml
filters:
  - name: active_only
    caption: "Show Active Only"
    fieldType: null  # or omit fieldType
    expression: "status = 'active' AND deleted = false"
```

**Expression Examples:**

```yaml
filterGroup:
  filters:
    # Simple equality
    - name: country
      caption: "Country"
      fieldType: string
      expression: "country = ::value::"
    
    # Case-insensitive search
    - name: company_search
      caption: "Company Name"
      fieldType: string
      expression: "LOWER(company_name) LIKE LOWER('%' || ::value:: || '%')"
    
    # Date range
    - name: order_date
      caption: "Order Date"
      fieldType: date
      expression: "order_date BETWEEN ::value:: AND ::value:: + INTERVAL '1 day'"
    
    # Numeric comparison
    - name: min_price
      caption: "Minimum Price"
      fieldType: number
      expression: "unit_price >= ::value::"
    
    # Multiple conditions
    - name: search
      caption: "Search"
      fieldType: string
      expression: "(name LIKE '%' || ::value:: || '%' OR description LIKE '%' || ::value:: || '%')"
    
    # Fixed condition (no fieldType)
    - name: show_active
      caption: "Active Only"
      fieldType: null
      expression: "status = 'active' AND archived = false"
```

**How Expressions Work:**

1. **When `fieldType` is set and value is provided:**
   - If `expression` contains `::value::`: All occurrences are replaced with `(?)` and the value is bound to each parameter
   - If `expression` doesn't contain `::value::`: The value is bound as a single parameter to the entire expression
   - The expression is only applied if the value is not null and not blank

2. **When `fieldType` is null:**
   - The expression is executed directly without any value binding
   - Useful for fixed WHERE conditions that don't depend on user input

**Default Behavior:**

If `expression` is not specified, the system uses default filtering behavior:
- For filters with `fieldType`: `column_name = ?` (exact match)
- The filter name must match a column name or be in `viewBindings`

## Advanced Options

**Debug Mode:**

```yaml
debug: true  # Prints SQL queries to browser console for debugging
```

**Wait for Filter:**

```yaml
waitForFilter: true  # Query waits for user to apply filters before executing
                      # Useful for expensive queries that shouldn't run on page load
```

**Export and Auto-refresh:**

These can be specified at the root level or in `options`:

```yaml
exportable: true      # Enable Excel export
autorefresh: 60       # Auto-refresh every 60 seconds
```

**Theme Variants:**

The `themeVariants` option in `options` allows you to customize the visual appearance of the grid. Valid values are Vaadin GridVariant constants:

```yaml
options:
  themeVariants:
    - LUMO_COMPACT              # Makes grid more dense by reducing header/row heights and column spacing
    - LUMO_ROW_STRIPES          # Alternating row colors for better readability
    - LUMO_NO_BORDER            # Removes outer border of the grid
    - LUMO_NO_ROW_BORDERS      # Removes horizontal borders between rows
    - LUMO_COLUMN_BORDERS       # Adds vertical borders between columns
    - LUMO_WRAP_CELL_CONTENT    # Wraps cell content, allowing rows to expand height as needed
```

**Available Theme Variants:**

- **`LUMO_COMPACT`**: Makes the grid more dense by reducing header and row heights, as well as column spacing. Useful for displaying more information on screen without scrolling. Also helps improve scanning and comparison between rows.

- **`LUMO_ROW_STRIPES`**: Produces alternating row colors to highlight rows and improve readability.

- **`LUMO_NO_BORDER`**: Removes the outer border of the grid.

- **`LUMO_NO_ROW_BORDERS`**: Removes horizontal borders between rows.

- **`LUMO_COLUMN_BORDERS`**: Adds vertical borders between columns.

- **`LUMO_WRAP_CELL_CONTENT`**: Displays all cell content even when it requires increasing row height. Useful for cells with long text that should be fully visible.

**Example:**

```yaml
options:
  themeVariants:
    - LUMO_COMPACT
    - LUMO_ROW_STRIPES
    - LUMO_COLUMN_BORDERS
```

Multiple variants can be combined for different visual effects. The variants are applied to the Vaadin Grid component using `GridVariant.valueOf()`.

**Row Styling with `classColumn`:**

The `classColumn` option allows you to apply CSS classes to rows based on a column value. The value from the specified column is used as the CSS class name for each row.

```yaml
options:
  classColumn: "status_class"  # Column name whose value will be used as CSS class
```

**How it works:**

- The value from the `classColumn` is read for each row
- This value is applied as a CSS class to all cells in that row
- The CSS class must be defined in your theme's `vaadin-grid.css` file
- If the column value is `null` or empty, no class is applied

**Example:**

```yaml
columns:
  - name: status
    caption: "Status"
    fieldType: string
  - name: status_class
    caption: "Class"
    fieldType: string
    # This column can be hidden or used for styling only
view: |
  SELECT 
    status,
    CASE 
      WHEN status = 'error' THEN 'error-row'
      WHEN status = 'warning' THEN 'warning-row'
      WHEN status = 'success' THEN 'success-row'
      ELSE 'pending-row'
    END AS status_class
  FROM orders
options:
  classColumn: "status_class"
```

**Available CSS Classes:**

The following CSS classes are predefined in the BPM theme (`vaadin-grid.css`). You can use these class names in your `classColumn`:

**Status-based classes:**
- `error-row` - Light red background (#f8d7da) with dark red text (#721c24) - for errors or critical issues
- `warning-row` - Light yellow background (#fff3cd) with dark yellow text (#856404) - for warnings
- `success-row` - Light green background (#d4edda) with dark green text (#155724) - for success or completed
- `in-progress-row` - Light blue background (#cce5ff) with dark blue text (#004085) - for in-progress items
- `pending-row` - Very light gray background (#fefefe) with gray text (#6c757d) - for pending or waiting items
- `highlight-row` - Light gray-blue background (#e2e3e5) with dark gray-blue text (#383d41) - for highlighted or important items
- `disabled-row` - Light gray background (#f8f9fa) with gray text (#6c757d) - for disabled or inactive items

**Business logic classes:**
- `due-soon-row` - Light yellow background - for items with upcoming deadlines
- `conflict-row` - Light red background - for items with conflicts
- `archived-row` - Light gray-blue background - for archived items
- `pending-comments-row` - Light blue background - for items with pending comments
- `dependent-row` - Light green background - for items with dependencies
- `critical-row` - Light red background - for items in critical state
- `new-row` - Light green background - for newly created items
- `recently-modified-row` - Light blue background - for recently modified items
- `exception-row` - Light yellow background - for items with exceptions
- `paused-row` - Light gray background - for paused items

**Color-based classes:**
- `light-red` / `medium-red` - Red tones
- `light-orange` / `medium-orange` - Orange tones
- `light-yellow` / `medium-yellow` - Yellow tones
- `light-green` / `medium-green` - Green tones
- `light-teal` / `medium-teal` - Teal/turquoise tones
- `light-blue` / `medium-blue` - Blue tones
- `light-purple` / `medium-purple` - Purple tones
- `light-grey` / `medium-grey` - Gray tones

**Complete Example:**

```yaml
name: "HLW/QRY_ORDERS"
caption: "Orders"
dataSource: "NORTHWIND"
view: |
  SELECT 
    order_id,
    status,
    CASE 
      WHEN status = 'shipped' THEN 'success-row'
      WHEN status = 'processing' THEN 'in-progress-row'
      WHEN status = 'cancelled' THEN 'error-row'
      WHEN status = 'pending' THEN 'pending-row'
      ELSE 'highlight-row'
    END AS status_class
  FROM orders
columns:
  - name: order_id
    caption: "Order ID"
    fieldType: integer
  - name: status
    caption: "Status"
    fieldType: string
  - name: status_class
    caption: "Class"
    fieldType: string
    # This column can be excluded from display if desired
keys:
  - order_id
options:
  classColumn: "status_class"
```

**Note:** You can also define custom CSS classes in your theme's `vaadin-grid.css` file. The class name must match exactly (case-sensitive) with the value in your `classColumn`.

**HTML Columns:**

Columns containing HTML can be specified in `options.htmlColumns`:

```yaml
options:
  htmlColumns:
    - description
```

**Frozen Columns:**

Columns to freeze (keep visible when scrolling horizontally) can be specified in `options.frozenColumns`:

```yaml
options:
  frozenColumns:
    - id
```

## Complete Query Example

Here's a comprehensive example combining multiple features:

```yaml
name: "HLW/QRY_ORDERS"
caption: "Order Management"
icon: "SHOPPING_CART"
dataSource: "NORTHWIND"
view: |
  SELECT 
    o.order_id,
    o.order_date,
    c.company_name,
    o.total_amount,
    o.status,
    '<div class="order-card">
       <strong>[[item.company_name]]</strong><br>
       <span>Total: $[[item.total_amount]]</span>
     </div>' AS card_html
  FROM orders o
  JOIN customers c ON o.customer_id = c.customer_id
  WHERE o.order_date >= ?::date
viewBindings:
  - start_date
stringFormat: "%{order_id} - %{company_name}"
defaultFilterExpression: "company_name LIKE '%' || ? || '%' OR order_id::text LIKE '%' || ? || '%'"
columns:
  - name: order_id
    caption: "Order ID"
    fieldType: integer
    width: 1
  - name: order_date
    caption: "Date"
    fieldType: date
    format: "yyyy-MM-dd"
  - name: company_name
    caption: "Customer"
    fieldType: string
    width: 2
  - name: total_amount
    caption: "Total"
    fieldType: decimal
    format: "#,##0.00"
  - name: status
    caption: "Status"
    fieldType: string
  - name: card_html
    caption: "Card"
    fieldType: string
keys:
  - order_id
sortables:
  - order_date
  - company_name
  - total_amount
targetGroups:
  - "SALES"
  - "ADMIN"
menus:
  - "Orders"
filterGroup:
  callToAction: "Filter Orders"
  name: "OrderFilters"
  caption: "Order Filters"
  filters:
    - name: start_date
      caption: "Start Date"
      fieldType: date
      required: true
    - name: status
      caption: "Status"
      fieldType: string
      possibleValues: ["Pending", "Processing", "Shipped", "Delivered"]
details:
  - "HLW/QRY_ORDER_DETAILS"
reports:
  - "HLW/RPT_ORDER_SUMMARY"
services:
  - "HLW/SRV_CREATE_ORDER"
scripts:
  - "HLW/SCRIPT_CANCEL_ORDER"
  - "HLW/SCRIPT_EXPORT_ORDER"
context:
  default_status: "Pending"
initContextScript: |
  // Set default start date to beginning of current month
  var now = new Date();
  context.start_date = new Date(now.getFullYear(), now.getMonth(), 1).getTime();
  // Set status from context if not in filter
  if (!context.status) {
    context.status = context.default_status;
  }
selectionScript: |
  #!python3
  import redflagbpm
  bpm = redflagbpm.BPMService()
  count = len(bpm.context.selection)
  total = sum(float(item.get('total_amount', 0)) for item in bpm.context.selection)
  bpm.reply(f"<strong>{count} orders selected</strong> - Total: ${total:,.2f}")
dataSourceOptions:
  pageSize: 50
  timeout: 30000
  where: "o.deleted = false"
  orderBy: "o.order_date DESC"
options:
  crud: false
  pageSize: 50
  exportable: true
  themeVariants:
    - LUMO_COMPACT
    - LUMO_ROW_STRIPES
debug: false
waitForFilter: false
exportable: true
autorefresh: 300
```

---

# Templates [templates]
> Use this chapter when you need to customize how query rows are displayed with custom HTML layouts, render dynamic content with innerHTML, or generate templates from post processors.

The `template` field allows custom HTML rendering for columns using Polymer/Vaadin template syntax. When a template is defined, it replaces the standard column display with a single column containing custom HTML.

**Related Topics:** Templates are used in [Queries](#queries), [Value Providers](#value-providers), and can be generated by [Post Processors](#post-processors).

## Template Overview

Templates provide a way to customize how query rows are displayed in the grid. Instead of showing individual columns side by side, templates allow you to create custom HTML layouts that can combine multiple fields, add styling, and create interactive elements.

**Key Features:**

- Custom HTML rendering for query rows
- Polymer/Vaadin template syntax
- Replaces standard column display when defined
- Single column with custom layout
- Access to all row fields in template

## Template Syntax

Templates use Polymer/Vaadin template syntax with data binding:

```yaml
template: |
  <div class="custom-row">
    <span class="name">[[item.name]]</span>
    <span class="status" data-status$="[[item.status]]">[[item.status]]</span>
  </div>
```

**Data Binding:**

- `[[item.fieldName]]`: Binds to row field values
- `data-status$="[[item.status]]"`: Binds to HTML attributes
- All row fields are available via `[[item.fieldName]]`

**HTML Structure:**

Templates can include:
- Custom HTML structure
- CSS classes for styling
- HTML attributes with data binding
- Conditional rendering (using Polymer template features)
- Interactive elements

## Template Field in Queries

The `template` field is defined in the QueryInfo YAML structure:

```yaml
template: |
  <div class="custom-row">
    <span class="name">[[item.name]]</span>
    <span class="status">[[item.status]]</span>
  </div>
```

**When Template is Defined:**

- Replaces individual columns with a single column
- All columns defined in the `columns` array are still available for data binding
- The template receives all row fields via `[[item.fieldName]]`
- Template renderer uses `TemplateRenderer.of()` to create the custom column

**Accessing Row Fields:**

All fields from the query result are available in the template:

```yaml
template: |
  <div>
    <strong>[[item.order_id]]</strong> - [[item.company_name]]
    <br>
    <small>Status: [[item.status]]</small>
  </div>
```

Even hidden columns (not in the `columns` array) are accessible in templates.

## Templates in Value Providers

Templates can also be used in Value Providers to customize how dropdown items are displayed:

```yaml
caption: "Lista de Clientes"
view: |
  SELECT 
    customer_id AS id,
    company_name AS caption,
    contact_name,
    phone
  FROM customers
columns:
  - name: id
    caption: "ID Cliente"
    fieldType: string
  - name: caption
    caption: "Nombre de la Empresa"
    fieldType: string
template: |
  <div>
    <strong>[[item.caption]]</strong>
    <br>
    <small>[[item.contact_name]] - [[item.phone]]</small>
  </div>
keys:
  - id
dataSource: NORTHWIND
```

When a Value Provider has a template, it's used to render items in combo boxes and dropdown lists, providing richer visual representation than simple text.

## Template from Post Processors

Post processors can generate templates dynamically by setting the `row.template` field. This allows for dynamic template generation based on row data.

**Markdown Post Processors:**

When a post processor script has `lang: "Markdown"`, the rendered HTML is automatically set to `row.template`:

```python
# Post processor script with lang: "Markdown"
# The markdown content is rendered to HTML and set as row.template
```

**YAML/JSON Post Processors:**

When a post processor script has `lang: "YAML"` or `lang: "JSON"`, the content is set directly to `row.template`:

```yaml
# Post processor script with lang: "YAML" or "JSON"
# The content is set as row.template
```

**Velocity (VTL) Post Processors:**

When a post processor script has `lang: "Velocity"`, the rendered output is set to `row.template`:

```vtl
# Post processor script with lang: "Velocity"
# The VTL template output is set as row.template
<div class="custom">
  <strong>$!{row.name}</strong>
</div>
```

**How It Works:**

1. Post processor script executes for each row
2. Script output (Markdown rendered HTML, YAML/JSON content, or VTL rendered output) is captured
3. The output is set to `row.template` field
4. The query grid uses `row.template` to render the row if no query-level template is defined

## Template Examples

**Basic Template with Field Binding:**

```yaml
template: |
  <div>
    <strong>[[item.name]]</strong>
    <br>
    <span>[[item.description]]</span>
  </div>
```

**Template with Conditional Styling:**

```yaml
template: |
  <div class$="[[item.status]]">
    <span class="name">[[item.name]]</span>
    <span class="status" data-status$="[[item.status]]">[[item.status]]</span>
  </div>
```

**Template with Multiple Fields:**

```yaml
template: |
  <div class="order-row">
    <div class="order-header">
      <strong>Order #[[item.order_id]]</strong>
      <span class="date">[[item.order_date]]</span>
    </div>
    <div class="order-details">
      <span>Customer: [[item.company_name]]</span>
      <span>Total: $[[item.total_amount]]</span>
    </div>
    <div class="order-status">
      Status: <span class="status-badge">[[item.status]]</span>
    </div>
  </div>
```

**Template in Value Provider:**

```yaml
caption: "Clientes"
view: |
  SELECT 
    customer_id AS id,
    company_name AS caption,
    contact_name,
    phone
  FROM customers
template: |
  <div>
    <strong>[[item.caption]]</strong>
    <br>
    <small>Contact: [[item.contact_name]]</small>
    <br>
    <small>Phone: [[item.phone]]</small>
  </div>
columns:
  - name: id
    fieldType: string
  - name: caption
    fieldType: string
keys:
  - id
```

**Template Generated by Post Processor:**

Post processor script (`lang: "Velocity"`):

```vtl
<div class="custom-card">
  <h3>$!{row.company_name}</h3>
  <p>Order ID: $!{row.order_id}</p>
  <p>Total: $!{row.total_amount}</p>
  <span class="status-$!{row.status}">$!{row.status}</span>
</div>
```

The VTL output is automatically set to `row.template` and used for rendering.

**Rendering Arbitrary HTML with innerHTML:**

When you need to render HTML that contains dynamic content or complex structures, you can use Polymer's `inner-h-t-m-l` binding to set HTML content directly:

```yaml
template: |
  <div inner-h-t-m-l="[[item.html_content]]"></div>
```

**Important:** The field name must use hyphens (`inner-h-t-m-l`) instead of camelCase (`innerHTML`) in Polymer templates.

**Example Query with innerHTML (HTML from SQL):**

```yaml
name: "PROJECT/QRY_ORDERS"
caption: "Orders"
dataSource: "NORTHWIND"
view: |
  SELECT 
    order_id,
    company_name,
    order_date,
    total_amount,
    status,
    '<div class="order-card">
      <h3>Order #' || order_id || '</h3>
      <p>' || company_name || '</p>
      <span class="status-' || status || '">' || status || '</span>
    </div>' AS html_content
  FROM orders
columns:
  - name: order_id
    caption: "Order ID"
    fieldType: integer
  - name: html_content
    caption: "Content"
    fieldType: string
template: |
  <div class="order-card">
    <div inner-h-t-m-l="[[item.html_content]]"></div>
  </div>
keys:
  - order_id
```

**Note:** In this example, the HTML is generated directly in the SQL query. For more complex HTML generation, use a post processor (see examples below).

**Using Post Processor to Generate HTML:**

You can use a post processor script to calculate and set the HTML content dynamically:

**Post Processor Script (`lang: "Python"`):**

```python
#!python3
import redflagbpm
from datetime import datetime

bpm = redflagbpm.BPMService()

# Calculate HTML content based on row data
order_id = row.get('order_id')
company_name = row.get('company_name', '')
order_date = row.get('order_date')
total_amount = row.get('total_amount', 0)
status = row.get('status', '')

# Format date if available
if order_date:
    if isinstance(order_date, (int, float)):
        # Unix timestamp
        date_str = datetime.fromtimestamp(order_date / 1000).strftime('%Y-%m-%d')
    else:
        date_str = str(order_date)
else:
    date_str = 'N/A'

# Determine status color
status_colors = {
    'pending': '#ffc107',
    'processing': '#17a2b8',
    'shipped': '#28a745',
    'delivered': '#6c757d',
    'cancelled': '#dc3545'
}
status_color = status_colors.get(status.lower(), '#6c757d')

# Generate HTML
html_content = f"""
<div style="padding: 10px; border-left: 4px solid {status_color};">
  <div style="display: flex; justify-content: space-between; align-items: center;">
    <div>
      <h3 style="margin: 0; color: #333;">Order #{order_id}</h3>
      <p style="margin: 5px 0; color: #666;">{company_name}</p>
    </div>
    <div style="text-align: right;">
      <span style="background-color: {status_color}; color: white; padding: 4px 8px; border-radius: 4px; font-size: 12px;">
        {status.upper()}
      </span>
    </div>
  </div>
  <div style="margin-top: 10px; display: flex; gap: 20px; font-size: 14px;">
    <span><strong>Date:</strong> {date_str}</span>
    <span><strong>Total:</strong> ${total_amount:,.2f}</span>
  </div>
</div>
"""

# Set the HTML content in the row
row['html_content'] = html_content
```

**Query Configuration:**

```yaml
name: "PROJECT/QRY_ORDERS"
caption: "Orders"
dataSource: "NORTHWIND"
view: |
  SELECT 
    order_id,
    company_name,
    order_date,
    total_amount,
    status
  FROM orders
columns:
  - name: order_id
    caption: "Order ID"
    fieldType: integer
  # html_content is added by post processor, don't include in columns
postProcessors:
  - "PROJECT/POST_PROCESS_ORDER_HTML"
template: |
  <div inner-h-t-m-l="[[item.html_content]]"></div>
keys:
  - order_id
```

**Complete Example: Dynamic HTML with Post Processor**

This example shows a complete setup where a post processor generates complex HTML with calculations:

**Post Processor Script (`PROJECT/POST_PROCESS_ORDER_HTML`):**

```python
#!python3
import redflagbpm
from datetime import datetime

bpm = redflagbpm.BPMService()

# Get row data
order_id = row.get('order_id')
company_name = row.get('company_name', '')
order_date = row.get('order_date')
total_amount = float(row.get('total_amount', 0))
status = row.get('status', '').lower()

# Calculate status badge
status_config = {
    'pending': {'color': '#ffc107', 'icon': '⏳'},
    'processing': {'color': '#17a2b8', 'icon': '⚙️'},
    'shipped': {'color': '#28a745', 'icon': '🚚'},
    'delivered': {'color': '#6c757d', 'icon': '✓'},
    'cancelled': {'color': '#dc3545', 'icon': '✗'}
}
status_info = status_config.get(status, {'color': '#6c757d', 'icon': '?'})

# Format amount with currency
amount_formatted = f"${total_amount:,.2f}"

# Format date
if order_date:
    if isinstance(order_date, (int, float)):
        date_obj = datetime.fromtimestamp(order_date / 1000)
        date_formatted = date_obj.strftime('%b %d, %Y')
    else:
        date_formatted = str(order_date)
else:
    date_formatted = 'N/A'

# Generate HTML with inline styles
html_content = f"""
<div style="
  padding: 15px;
  margin: 5px 0;
  border-radius: 8px;
  background: linear-gradient(135deg, #f5f7fa 0%, #c3cfe2 100%);
  box-shadow: 0 2px 4px rgba(0,0,0,0.1);
">
  <div style="display: flex; justify-content: space-between; align-items: start;">
    <div>
      <h3 style="margin: 0 0 8px 0; color: #2c3e50; font-size: 18px;">
        Order #{order_id}
      </h3>
      <p style="margin: 0; color: #34495e; font-size: 14px;">
        {company_name}
      </p>
    </div>
    <div style="
      background-color: {status_info['color']};
      color: white;
      padding: 6px 12px;
      border-radius: 20px;
      font-size: 12px;
      font-weight: bold;
      display: inline-flex;
      align-items: center;
      gap: 5px;
    ">
      <span>{status_info['icon']}</span>
      <span>{status.upper()}</span>
    </div>
  </div>
  <div style="margin-top: 12px; padding-top: 12px; border-top: 1px solid rgba(0,0,0,0.1);">
    <div style="display: flex; gap: 20px; font-size: 13px; color: #555;">
      <span><strong>Date:</strong> {date_formatted}</span>
      <span><strong>Amount:</strong> {amount_formatted}</span>
    </div>
  </div>
</div>
"""

# Set HTML content
row['html_content'] = html_content
```

**Query Configuration:**

```yaml
name: "PROJECT/QRY_ORDERS"
caption: "Orders"
icon: "SHOPPING_CART"
dataSource: "NORTHWIND"
view: |
  SELECT 
    order_id,
    company_name,
    order_date,
    total_amount,
    status
  FROM orders
  WHERE status IS NOT NULL
columns:
  - name: order_id
    caption: "Order ID"
    fieldType: integer
  # html_content is generated by post processor
postProcessors:
  - "PROJECT/POST_PROCESS_ORDER_HTML"
template: |
  <div inner-h-t-m-l="[[item.html_content]]"></div>
keys:
  - order_id
options:
  pageSize: 20
  exportable: true
```

**Security Note:**

When using `inner-h-t-m-l` with dynamic HTML content:
- **Sanitization**: The system automatically sanitizes HTML content to prevent XSS attacks
- **Trusted Content**: Only use HTML from trusted sources (your own post processors)
- **Avoid User Input**: Never directly insert user input into HTML without sanitization

## Best Practices

**When to Use Templates vs Columns:**

- **Use Templates when:**
  - You need custom layouts combining multiple fields
  - You want rich HTML formatting
  - You need conditional styling based on data
  - You want interactive elements

- **Use Columns when:**
  - You need standard grid display
  - You want sortable columns
  - You need column-based filtering
  - You want simple, tabular data display

**Performance Considerations:**

- Templates are rendered on the client side
- Complex templates with many bindings may impact performance
- Consider using post processors to pre-generate HTML if templates are complex
- Hidden columns are still available in templates without performance penalty

**Accessibility and Responsive Design:**

- Ensure templates are accessible (proper HTML structure, ARIA labels)
- Test templates on different screen sizes
- Use semantic HTML elements
- Consider mobile responsiveness when designing templates

**Template Maintenance:**

- Keep templates simple and readable
- Document complex template logic
- Use CSS classes for styling rather than inline styles when possible
- Test templates with different data sets

---

# Post Processors [post-processors]
> Use this chapter when you need to transform query row data, add computed fields, format values, or generate HTML templates dynamically for each row before display.

Post processors are scripts executed for each row retrieved by the query. They can modify row data, add computed fields, or transform values before display.

**Related Topics:** Post processors are configured in [Queries](#queries) and can generate [Templates](#templates) dynamically.

## Post Processor Overview

Post processors provide a way to process and transform data after it's retrieved from the data source but before it's displayed in the grid. They run for each row individually, allowing you to:

- Modify row data before display
- Add computed fields based on existing data
- Transform values for better presentation
- Enrich data with external lookups
- Generate HTML templates dynamically

**Execution Flow:**

1. Query retrieves data from data source
2. For each row, post processors execute in order
3. Each post processor receives the row object (can modify it)
4. Modified row data is displayed in the grid

## Post Processor Configuration

Post processors are configured in the QueryInfo YAML using the `postProcessors` array:

```yaml
postProcessors:                 # Scripts executed for each row retrieved
  - "PROJECT/POST_PROCESSOR_SCRIPT"
  - "PROJECT/ANOTHER_PROCESSOR"
```

**Configuration Details:**

- **Array Format**: List of script IDs in `PROJECT/SCRIPT_ID` format
- **Execution Order**: Processors execute sequentially in the order listed
- **Multiple Processors**: Each processor receives the row modified by previous processors
- **Script Types**: Can be Python, JavaScript, Markdown, YAML, JSON, or Velocity

## Post Processor Script Context

Post processor scripts receive the current row as a context variable:

**Python Scripts:**

```python
#!python3
# row is a dictionary containing all row fields
row['new_field'] = 'computed_value'
row['existing_field'] = row['existing_field'].upper()
```

**JavaScript Scripts:**

```javascript
// row is a JavaScript object
row.newField = 'computed_value';
row.existingField = row.existingField.toUpperCase();
```

**Context Variables:**

- **`row`**: The current row object (JsonObject/dict, can be modified)
- **All row fields**: Available directly as `row.fieldName` or `row['fieldName']`
- **Hidden fields**: Also accessible (fields not in `columns` array)

**Modifying Row Data:**

- **Add new fields**: `row['new_field'] = value`
- **Modify existing fields**: `row['existing_field'] = new_value`
- **Access all fields**: All fields from SQL SELECT are available, including hidden columns

## Post Processor Script Languages

Post processors support multiple scripting languages, each with different capabilities:

**Python:**

Full access to `redflagbpm` library and standard Python features:

```python
#!python3
import redflagbpm
from datetime import datetime

bpm = redflagbpm.BPMService()
# Access row data
row['computed'] = row['field1'] + row['field2']
```

**JavaScript:**

Standard JavaScript execution:

```javascript
row.computed = row.field1 + row.field2;
row.formatted = new Date(row.timestamp).toLocaleDateString();
```

**Markdown:**

Markdown content is rendered to HTML and set to `row.template`:

```markdown
# Post processor with lang: "Markdown"
**Order:** [[order_id]]
**Status:** [[status]]
```

The rendered HTML becomes the template for the row.

**YAML/JSON:**

Content is set directly to `row.template`:

```yaml
# Post processor with lang: "YAML" or "JSON"
<div class="custom">
  <strong>{{order_id}}</strong>
</div>
```

**Velocity (VTL):**

Velocity template is rendered and output set to `row.template`:

```vtl
<div class="order-card">
  <h3>$!{row.company_name}</h3>
  <p>Order ID: $!{row.order_id}</p>
  <p>Total: $!{row.total_amount}</p>
</div>
```

## Template Generation from Post Processors

Post processors can generate templates dynamically by setting the `row.template` field. This is especially useful for languages that produce HTML output.

**Markdown Post Processors:**

When `lang: "Markdown"`, the markdown is parsed and rendered to HTML, then set to `row.template`:

```markdown
# Order Details

**Order ID:** [[order_id]]
**Customer:** [[company_name]]
**Total:** $[[total_amount]]
```

**YAML/JSON Post Processors:**

When `lang: "YAML"` or `lang: "JSON"`, the script content is written directly to `row.template`:

```yaml
<div class="order-summary">
  <strong>Order #{{order_id}}</strong>
  <p>Customer: {{company_name}}</p>
</div>
```

**Velocity Post Processors:**

When `lang: "Velocity"`, the VTL template is rendered and the output is set to `row.template`:

```vtl
<div class="custom-card">
  <h3>$!{row.company_name}</h3>
  <p>Order: $!{row.order_id}</p>
  <span class="status-$!{row.status}">$!{row.status}</span>
</div>
```

**Using Generated Templates:**

- If a query has a `template` field defined, it takes precedence
- If no query template exists, `row.template` from post processors is used
- Templates generated by post processors allow per-row customization

## Common Use Cases

**Computing Derived Values:**

```python
#!python3
# Calculate total from items
row['total'] = row.get('quantity', 0) * row.get('unit_price', 0)

# Calculate percentage
if row.get('total') and row.get('target'):
    row['percentage'] = (row['total'] / row['target']) * 100
```

**Formatting Data for Display:**

```python
#!python3
from datetime import datetime

# Format date
if 'order_date' in row:
    timestamp = row['order_date']
    if isinstance(timestamp, (int, float)):
        dt = datetime.fromtimestamp(timestamp / 1000)
        row['order_date_formatted'] = dt.strftime('%B %d, %Y')

# Format currency
if 'total_amount' in row:
    row['total_formatted'] = f"${row['total_amount']:,.2f}"
```

**Adding HTML Templates:**

```python
#!python3
# Add status badge HTML
status = row.get('status', 'Unknown')
status_colors = {
    'Pending': 'orange',
    'Processing': 'blue',
    'Shipped': 'purple',
    'Delivered': 'green'
}
color = status_colors.get(status, 'gray')
row['status_badge'] = f'<span style="background: {color}; color: white; padding: 2px 8px; border-radius: 4px;">{status}</span>'
```

**Enriching Data with External Lookups:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Lookup additional data from a collection
customer_id = row.get('customer_id')
if customer_id:
    customer = bpm.documentService.readById('customers', customer_id)
    if customer:
        row['customer_email'] = customer.get('email')
        row['customer_phone'] = customer.get('phone')
```

**Data Validation and Transformation:**

```python
#!python3
# Validate and transform data
if row.get('price') and row['price'] < 0:
    row['price'] = 0
    row['price_warning'] = 'Price was negative, set to 0'

# Normalize text
if 'name' in row:
    row['name_normalized'] = row['name'].strip().title()
```

**Conditional Field Addition:**

```python
#!python3
# Add fields conditionally
if row.get('status') == 'urgent':
    row['priority_indicator'] = 'HIGH'
    row['requires_attention'] = True
else:
    row['requires_attention'] = False
```

## Post Processor Execution

**Execution Order:**

Multiple post processors execute sequentially:

1. First post processor receives original row data
2. Second post processor receives row modified by first processor
3. Third post processor receives row modified by first and second processors
4. And so on...

**Error Handling:**

- If a post processor encounters an error, the error is logged
- The row continues processing with remaining post processors
- Error information can be added to the row (e.g., `row['_error'] = 'Error message'`)

**Performance Considerations:**

- Post processors run for every row, so keep them efficient
- Avoid expensive operations (database queries, API calls) if possible
- Consider caching external lookups
- Use SQL transformations when possible instead of post processors

## Post Processor Examples

**Adding Computed Fields:**

```python
#!python3
# Add computed field
row['full_name'] = f"{row.get('first_name', '')} {row.get('last_name', '')}"

# Calculate age from birth date
if 'birth_date' in row:
    from datetime import datetime
    birth_date = datetime.fromtimestamp(row['birth_date'] / 1000)
    age = (datetime.now() - birth_date).days // 365
    row['age'] = age
```

**Formatting Dates and Numbers:**

```python
#!python3
from datetime import datetime

# Format date for display
if 'order_date' in row:
    timestamp = row['order_date']
    if isinstance(timestamp, (int, float)):
        dt = datetime.fromtimestamp(timestamp / 1000)
        row['order_date_formatted'] = dt.strftime('%B %d, %Y')

# Format number with currency
if 'total_amount' in row:
    row['total_formatted'] = f"${row['total_amount']:,.2f}"
```

**Adding Status Badges and HTML:**

```python
#!python3
# Add status badge HTML
status = row.get('status', 'Unknown')
status_colors = {
    'Pending': 'orange',
    'Processing': 'blue',
    'Shipped': 'purple',
    'Delivered': 'green'
}
color = status_colors.get(status, 'gray')
row['status_badge'] = f'<span style="background: {color}; color: white; padding: 2px 8px; border-radius: 4px;">{status}</span>'
```

**Complete Post Processor Example:**

Example post processor script (`HLW/POST_PROCESS_ORDER`):

```python
#!python3
import redflagbpm
from datetime import datetime

bpm = redflagbpm.BPMService()

# Format date for display
if 'order_date' in row:
    timestamp = row['order_date']
    if isinstance(timestamp, (int, float)):
        dt = datetime.fromtimestamp(timestamp / 1000)
        row['order_date_formatted'] = dt.strftime('%B %d, %Y')

# Calculate days since order
if 'order_date' in row:
    order_date = datetime.fromtimestamp(row['order_date'] / 1000)
    days_ago = (datetime.now() - order_date).days
    row['days_ago'] = days_ago
    row['is_recent'] = days_ago <= 7

# Add status badge HTML
status = row.get('status', 'Unknown')
status_colors = {
    'Pending': 'orange',
    'Processing': 'blue',
    'Shipped': 'purple',
    'Delivered': 'green'
}
color = status_colors.get(status, 'gray')
row['status_badge'] = f'<span style="background: {color}; color: white; padding: 2px 8px; border-radius: 4px;">{status}</span>'

# Add computed total with currency
if 'total_amount' in row:
    row['total_formatted'] = f"${row['total_amount']:,.2f}"
```

**Markdown Post Processor Generating Template:**

Post processor script with `lang: "Markdown"`:

```markdown
# Order Summary

**Order ID:** [[order_id]]
**Customer:** [[company_name]]
**Date:** [[order_date]]
**Total:** $[[total_amount]]
**Status:** [[status]]
```

The rendered HTML is automatically set to `row.template`.

**Velocity Post Processor Generating Template:**

Post processor script with `lang: "Velocity"`:

```vtl
<div class="order-card">
  <div class="order-header">
    <h3>Order #$!{row.order_id}</h3>
    <span class="date">$!{row.order_date}</span>
  </div>
  <div class="order-details">
    <p><strong>Customer:</strong> $!{row.company_name}</p>
    <p><strong>Total:</strong> $!{row.total_amount}</p>
    <p><strong>Status:</strong> <span class="status-$!{row.status}">$!{row.status}</span></p>
  </div>
</div>
```

The rendered output is set to `row.template`.

## Best Practices

**Efficient Row Processing:**

- Keep post processor logic simple and fast
- Avoid nested loops or complex calculations
- Cache frequently accessed data
- Use SQL transformations when possible (more efficient)

**Error Handling Strategies:**

- Always check if fields exist before accessing: `row.get('field', default)`
- Handle type conversions carefully
- Log errors but don't fail the entire row processing
- Add error indicators to rows when needed

**Performance Optimization:**

- Minimize external API calls or database queries
- Use batch lookups when enriching data
- Consider moving complex logic to SQL queries
- Profile post processors if performance is an issue

**When to Use Post Processors vs SQL Transformations:**

- **Use Post Processors when:**
  - You need external data lookups
  - You need complex formatting logic
  - You need to generate HTML templates
  - You need conditional field addition based on complex logic

- **Use SQL Transformations when:**
  - Simple calculations and formatting
  - Data is already in the database
  - Performance is critical
  - You want to leverage database functions

---

# Value Providers [value-providers]
> Use this chapter when you need to populate dropdown lists in forms or filters with dynamic data from queries. Requires queries that return id and caption columns.

Value Providers are special queries used to populate dropdown lists in forms and filters. They must return two columns: `id` (the value to be stored) and `caption` (the display text).

**Related Topics:** Value Providers are a special type of [Query](#queries) and can use [Templates](#templates) for custom rendering. They are used in [Forms](#forms) and query filter groups.

## Value Provider Structure

Value Providers are queries with a specific structure. They must return at least two columns:

- **`id`**: The value to be stored (typically aliased from a primary key or identifier field)
- **`caption`**: The display text shown in the dropdown

**Basic YAML Structure:**

```yaml
caption: "Lista de Clientes"
view: |
  SELECT 
    customer_id AS id,
    company_name AS caption
  FROM 
    customers
columns:
  - name: id
    caption: "ID Cliente"
    fieldType: string
  - name: caption
    caption: "Nombre de la Empresa"
    fieldType: string
keys:
  - id
dataSource: NORTHWIND
```

**Column Definitions:**

- Both `id` and `caption` columns must be defined in the `columns` array
- The `id` column is typically used as the primary key
- Additional columns can be included for use in `stringFormat` or templates

## Using Value Providers

Value Providers are referenced using the `valueProvider` hint in forms and filters.

**In Forms:**

Value Providers are used in form field hints:

```yaml
properties:
  cliente:
    type: string
    title: "Cliente"
    hints:
      valueProvider: PROJECT/VP_CLIENTES
      callToAction: "Seleccionar Cliente"
```

**In Filters:**

Value Providers can be used in query filter groups:

```yaml
filterGroup:
  filters:
    - name: customer_id
      caption: "Customer"
      fieldType: string
      valueProvider: "PROJECT/VP_CUSTOMERS"
      stringFormat: "%{id} - %{caption}"
```

**Referencing Value Providers:**

- Format: `PROJECT/VP_ID` (e.g., `HLW/VP_CLIENTES`)
- The Value Provider query ID must match an existing query
- Value Providers can be used across different artifacts (forms, filters, etc.)

## String Format Customization

Value Providers can use `stringFormat` to customize how the dropdown displays values. The format uses placeholders to combine multiple fields.

**Format Placeholders:**

- **`%{fieldName}`**: Replaced with the value of the specified field
- **`%1$s`, `%2$s`, etc.**: Positional placeholders (1-indexed, refers to column order)

**Examples:**

```yaml
stringFormat: "%{id} - %{caption}"  # Displays: "123 - Company Name"
# or
stringFormat: "%1$s - %2$s"        # Positional: first column - second column
```

**Display Text Formatting:**

The formatted string is used as the display text in dropdowns. This allows you to show more information than just the caption:

```yaml
# Value Provider with additional fields
view: |
  SELECT 
    customer_id AS id,
    company_name AS caption,
    contact_name,
    phone
  FROM customers
stringFormat: "%{caption} - %{contact_name} (%{phone})"
# Displays: "Company Name - Contact Name (123-456-7890)"
```

**Using Positional Placeholders:**

Positional placeholders refer to columns in the order they appear in the SELECT:

```yaml
stringFormat: "%1$s - %2$s - %3$s"  # First, second, third columns
```

## Value Provider Examples

**Basic Value Provider Query:**

```yaml
caption: "Lista de Clientes"
view: |
  SELECT 
    customer_id AS id,
    company_name AS caption
  FROM 
    customers
columns:
  - name: id
    caption: "ID Cliente"
    fieldType: string
  - name: caption
    caption: "Nombre de la Empresa"
    fieldType: string
keys:
  - id
dataSource: NORTHWIND
```

**Value Provider with String Format:**

```yaml
caption: "Lista de Clientes"
view: |
  SELECT 
    customer_id AS id,
    company_name AS caption,
    contact_name,
    phone
  FROM customers
stringFormat: "%{id} - %{caption} (%{contact_name})"
columns:
  - name: id
    fieldType: string
  - name: caption
    fieldType: string
  - name: contact_name
    fieldType: string
  - name: phone
    fieldType: string
keys:
  - id
dataSource: NORTHWIND
```

**Using Value Provider in Form Field:**

```yaml
properties:
  cliente:
    type: string
    title: "Cliente"
    hints:
      valueProvider: PROJECT/VP_CLIENTES
      callToAction: "Seleccionar Cliente"
```

**Using Value Provider in Filter:**

```yaml
filterGroup:
  filters:
    - name: customer_id
      caption: "Customer"
      fieldType: string
      valueProvider: "PROJECT/VP_CUSTOMERS"
      stringFormat: "%{id} - %{caption}"
```

**Value Provider with Template:**

Value Providers can also use templates for custom rendering in dropdowns (see [Templates](#templates) chapter for details).

---

# Kanban [kanban]
> Use this chapter when you need to display query data as a Kanban board with cards that can be dragged between columns. Configure columns, card fields, and move actions.

Queries can be displayed as Kanban boards, providing a visual card-based interface for managing items across different stages or categories.

**Related Topics:** Kanban is a visualization option for [Queries](#queries). Cards can use HTML content generated in SQL queries or from [Post Processors](#post-processors).

## Kanban Configuration

Enable Kanban view by setting `options.kanban: true` in your query configuration:

```yaml
options:
  kanban: true
  kanbanColumns:                # Column definitions for Kanban
    Argentina: Argentina
    USA: USA
    Germany: Germany
  titleField: company_name      # Field to display as card title
  cardIdField: customer_id      # Field used as card identifier
  columnIdField: country        # Field used to group cards into columns
  bodyField: body               # Field containing card body HTML
  cardMovedScript: "run:PROJECT/SCRIPT_ID" # Script executed when card is moved
```

**Required Configuration:**

- **`kanban: true`**: Enables Kanban view
- **`kanbanColumns`**: Map of column IDs to display names
- **`titleField`**: Field to display as card title
- **`cardIdField`**: Field used as unique card identifier
- **`columnIdField`**: Field used to group cards into columns
- **`bodyField`**: Field containing HTML for card body

## Kanban Column Definitions

The `kanbanColumns` option defines the columns (swimlanes) in the Kanban board:

```yaml
options:
  kanbanColumns:
    Pending: "Pending Orders"
    Processing: "Processing"
    Shipped: "Shipped"
    Delivered: "Delivered"
```

**Column Mapping:**

- **Key**: The value from `columnIdField` that groups cards
- **Value**: The display name shown as the column header

Cards are automatically grouped into columns based on the `columnIdField` value matching the keys in `kanbanColumns`.

## Card Fields

Kanban cards require specific field mappings:

**`titleField`:**

Field displayed as the card title (typically the main identifier or name):

```yaml
titleField: company_name
```

**`cardIdField`:**

Field used as the unique card identifier. This should be a primary key or unique identifier:

```yaml
cardIdField: customer_id
```

**`columnIdField`:**

Field used to group cards into columns. The value of this field must match a key in `kanbanColumns`:

```yaml
columnIdField: country
```

**`bodyField`:**

Field containing HTML content for the card body. This HTML is rendered inside each card:

```yaml
bodyField: body
```

## Card Body HTML

The `bodyField` typically contains HTML generated in the SQL query. This allows you to create rich card content:

**Generating HTML in SQL:**

```sql
SELECT 
  customer_id,
  company_name,
  country,
  '<div class="card">' ||
    '<p>' || coalesce(contact_name, '') || '</p>' ||
    '<address>' || coalesce(address, '') || '</address>' ||
  '</div>' AS body
FROM customers
```

**Card Body Structure:**

The HTML in `bodyField` can include:
- Text content
- Formatted data (dates, numbers, etc.)
- Styling with CSS classes
- Multiple data fields combined
- Icons or images

**Including Data in Card Body:**

You can combine multiple fields in the card body:

```sql
SELECT 
  order_id,
  company_name,
  status,
  '<div class="order-card">
     <h4>Order #' || order_id || '</h4>
     <p>Customer: ' || company_name || '</p>
     <p>Total: $' || total_amount || '</p>
     <span class="status">' || status || '</span>
   </div>' AS body
FROM orders
```

## Card Moved Script

The `cardMovedScript` is executed when a card is moved between columns (drag and drop):

```yaml
options:
  cardMovedScript: "run:PROJECT/SCRIPT_ID"
```

**Script Context:**

The script receives:
- Card data (all fields from the row)
- New column value (from `columnIdField`)
- Previous column value

**Accessing Card Data:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Access card data from context
card_id = bpm.context.cardIdField  # e.g., customer_id
new_column = bpm.context.columnIdField  # New column value
previous_column = bpm.context.previousColumn  # Previous column value

# Update data based on card move
bpm.documentService.updateById(
    collection="customers",
    id=card_id,
    data={columnIdField: new_column}
)
```

**Updating Data on Card Move:**

Typically, you'll want to update the `columnIdField` value in your data source when a card is moved:

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Get card identifier and new column
card_id = bpm.context.get('customer_id')
new_status = bpm.context.get('status')  # Assuming columnIdField is 'status'

# Update the record
# This depends on your data source (SQL update, document update, etc.)
```

## Complete Kanban Example

**Kanban Query Configuration:**

```yaml
caption: "Consulta de Clientes"
icon: "ADDRESS_BOOK"
view: |
  SELECT customer_id, 
         company_name,
         country,
         '<div class="card">' ||
           '<p>' || coalesce(contact_name, '') || '</p>' ||
           '<address>' || coalesce(address, '') || '</address>' ||
         '</div>' AS body
  FROM customers
columns:
  - name: customer_id
    caption: "ID Cliente"
    fieldType: string
  - name: country
    caption: "País"
    fieldType: string
  - name: body
    caption: "Datos"
    fieldType: string
keys:
  - customer_id
dataSource: NORTHWIND
options:
  kanban: true
  kanbanColumns:
    Argentina: Argentina
    USA: USA
    Germany: Germany
  titleField: company_name
  cardIdField: customer_id
  columnIdField: country
  bodyField: body
  cardMovedScript: "run:PROJECT/SCRIPT_ID"
scripts:
  - PROJECT/SCRIPT_ID
```

**Kanban with Order Status:**

```yaml
caption: "Order Management"
view: |
  SELECT 
    o.order_id,
    c.company_name,
    o.status,
    '<div class="order-card">
       <h4>Order #' || o.order_id || '</h4>
       <p><strong>' || c.company_name || '</strong></p>
       <p>Date: ' || to_char(o.order_date, 'MM/DD/YYYY') || '</p>
       <p>Total: $' || o.total_amount || '</p>
     </div>' AS body
  FROM orders o
  JOIN customers c ON o.customer_id = c.customer_id
columns:
  - name: order_id
    fieldType: integer
  - name: company_name
    fieldType: string
  - name: status
    fieldType: string
  - name: body
    fieldType: string
keys:
  - order_id
options:
  kanban: true
  kanbanColumns:
    Pending: "Pending Orders"
    Processing: "Processing"
    Shipped: "Shipped"
    Delivered: "Delivered"
  titleField: company_name
  cardIdField: order_id
  columnIdField: status
  bodyField: body
  cardMovedScript: "run:HLW/UPDATE_ORDER_STATUS"
```

---

# Services [services]
> Use this chapter when you need to create executable actions that call external APIs, wrap internal logic with user interfaces, or define services with request/response handling and forms.

Services define executable actions, often wrapping external API calls or internal logic with a user interface. Services can be defined as YAML files or Python scripts.

**Related Topics:** Services can use [Forms](#forms) for input, execute [Scripts](#scripts), generate [Reports](#reports), and open [Queries](#queries) with response data.

## YAML Structure

```yaml
name: "PROJECT/SERVICE_ID"      # Unique identifier (optional, usually derived from filename)
caption: "Service Title"
icon: "ANALYSIS"                # Icon name
reusable: true                  # Keep open after execution? (false closes window after execution)
requestUrl: "run:PROJECT/SCRIPT_ID" # Endpoint: URL, "run:SCRIPT", "curl COMMAND", or document path
requestOptions:                 # Request configuration options
  timeout: 30                   # Maximum wait time in seconds (default: 300 for curl)
  upload: false                 # true for POST/Multipart file uploads
requestScript: |                # JavaScript script executed before request
  // Modify context before sending request
  context.param1 = "computed_value";
  context.timestamp = new Date().getTime();
responseType: "MESSAGE"         # QUERY, ASSET, DISPLAY, SERVICE, TERMINAL, MESSAGE, REPORT
responseFilter: ".result"       # JQ expression to filter/extract from JSON response
responseScript: |               # JavaScript script executed after receiving response
  // Transform response data
  response.data = response.result.map(item => item.value);
responseTarget: "TARGET_ID"     # Artifact ID to handle response (required for QUERY/REPORT/SERVICE types)
targetUsers: []                 # Specific users allowed (empty = all users)
targetGroups: ["ADMIN"]         # Access control groups
menus:                          # Menu items where service appears
  - "Services"
  - "Tools"
inputSchema:                    # Form definition (JSON Schema)
  type: object
  title: "Form Title"
  description: |                # HTML description shown above form
    <p>This is a <strong>description</strong> with HTML support.</p>
  required:
    - field1
  properties:
    field1:
      type: string
      title: "Label"
```

## Python Script Services

Services can also be implemented as Python scripts. The script should generate the response (HTML, JSON, etc.) and set appropriate headers:

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

print("""
<!DOCTYPE html>
<html>
<head><title>Result</title></head>
<body><h1>Result</h1></body>
</html>
""")

_responseHeaders = bpm.context.json._responseHeaders
_responseHeaders["status"] = "200"
_responseHeaders["Content-Type"] = "text/html"
_responseHeaders["Content-Disposition"] = "inline; filename=result.html"
```

## Request URL Types

The `requestUrl` can be:

- **`run:PROJECT/SCRIPT_ID`**: Execute a script endpoint
- **`https://api.example.com/endpoint`**: REST API URL (supports VTL template variables)
- **`curl -X POST https://api.example.com ...`**: cURL command (full command line)
- **`/path/to/document`**: Document path for document-based services

**VTL Template Variables in URLs:**

```yaml
requestUrl: "https://api.example.com/users/${userId}/orders?status=${status}"
```

Variables from `inputSchema` form fields are automatically substituted.

## Request Script

The `requestScript` is a JavaScript snippet executed before making the request. It can:

- Modify form input values
- Compute derived values
- Add authentication tokens
- Prepare data transformations

**Request Script Context:**

- `context`: The form input object (can be modified)
- All form fields are available as `context.fieldName`

**Example:**

```javascript
// Add computed timestamp
context.timestamp = new Date().getTime();

// Transform data structure
context.apiPayload = {
  userId: context.user_id,
  action: context.action_type,
  metadata: {
    source: "bpm",
    version: "1.0"
  }
};

// Add authentication
context.apiKey = "Bearer " + context.token;
```

## Request Options

**Timeout:**

```yaml
requestOptions:
  timeout: 60  # Seconds (default: 300 for curl commands)
```

**File Upload:**

```yaml
requestOptions:
  upload: true  # Enables POST/Multipart for file uploads
```

When `upload: true` and form has upload fields, files are automatically attached to the POST request.

## Response Types

- **`MESSAGE`**: Display a status message to the user (default)
- **`ASSET`**: Download a file (PDF, Excel, etc.). The endpoint should return file content with appropriate headers
- **`DISPLAY`**: Display asset inline in browser (e.g., PDF viewer)
- **`QUERY`**: Open or refresh a query with response data. Requires `responseTarget` pointing to a Query ID
- **`SERVICE`**: Open another service with response data. Requires `responseTarget` pointing to a Service ID
- **`TERMINAL`**: Display terminal-style output (monospace, scrollable)
- **`REPORT`**: Generate a report with response data. Requires `responseTarget` pointing to a Report ID

**Response Type Details:**

**QUERY Response:**

- Response must be JSON array
- Data is loaded into the target query
- Query opens with the provided data

**REPORT Response:**

- Response can be JSON array or object
- Data is passed to report as parameters
- Report is generated and displayed

**SERVICE Response:**

- Response is JSON object
- Data is passed to target service as input
- Target service opens with the data

**ASSET Response:**

- Endpoint should return file content
- Headers should include `Content-Type` and optionally `Content-Disposition`
- File is downloaded or displayed based on headers

## Response Filter

The `responseFilter` uses JQ (JSON Query) expressions to extract or transform JSON responses:

```yaml
responseFilter: ".result.items[]"  # Extract items array from result object
responseFilter: ".data | select(.status == 'active')"  # Filter active items
responseFilter: "{id: .id, name: .name}"  # Transform structure
```

**JQ Examples:**

- `.result`: Extract `result` field
- `.items[]`: Extract all items from array
- `.data | select(.active == true)`: Filter by condition
- `{id: .id, name: .name}`: Create new object structure

## Response Script

The `responseScript` is a JavaScript snippet executed after receiving the response (and after `responseFilter` if present). It can:

- Transform response data
- Add computed fields
- Validate response
- Convert data formats

**Response Script Context:**

- `response`: The filtered response (JSON object or array, depending on `responseType`)
- For `QUERY` type: receives JsonArray
- For `SERVICE`/`REPORT` types: receives JsonObject
- For `MESSAGE`/`TERMINAL` types: receives String

**Example Response Scripts:**

**For QUERY response:**

```javascript
// Add computed field to each item
response.forEach(item => {
  item.fullName = item.firstName + " " + item.lastName;
  item.isActive = item.status === "active";
});
```

**For SERVICE response:**

```javascript
// Transform response structure
response = {
  formData: {
    customerId: response.customer.id,
    orderTotal: response.order.total,
    items: response.order.items
  }
};
```

**For MESSAGE/TERMINAL response:**

```javascript
// Transform string response
response = "<div class='success'>" + response + "</div>";
```

## Response Target

The `responseTarget` specifies which artifact handles the response:

- **For `QUERY`**: Query ID that will display the response data
- **For `REPORT`**: Report ID that will generate report with response data
- **For `SERVICE`**: Service ID that will open with response data as input

**Example:**

```yaml
responseType: QUERY
responseTarget: "PROJECT/QRY_RESULTS"  # Query that displays the results
```

The target artifact receives the filtered and script-processed response data.

## Menus

Services can appear in application menus:

```yaml
menus:
  - "Main Menu"
  - "Tools"
  - "Administration"
```

Menu items are automatically created based on the `menus` array. Users with appropriate permissions will see the service in these menu locations.

## Complete Service Examples

**Service with Query Response:**

```yaml
name: "HLW/SRV_SEARCH_ORDERS"
caption: "Search Orders"
icon: "SEARCH"
reusable: true
requestUrl: "run:HLW/SEARCH_ORDERS_SCRIPT"
requestOptions:
  timeout: 60
requestScript: |
  // Prepare search parameters
  context.searchQuery = context.customer_name + " " + context.order_id;
  context.apiKey = "Bearer " + context.auth_token;
responseType: QUERY
responseFilter: ".data.orders[]"
responseTarget: "HLW/QRY_ORDER_RESULTS"
targetGroups:
  - "SALES"
menus:
  - "Orders"
inputSchema:
  type: object
  title: "Search Orders"
  properties:
    customer_name:
      type: string
      title: "Customer Name"
    order_id:
      type: string
      title: "Order ID"
    auth_token:
      type: string
      title: "API Token"
      hints:
        subtype: password
  required:
    - customer_name
```

**Service with Asset Response (File Download):**

```yaml
name: "HLW/SRV_GENERATE_REPORT"
caption: "Generate PDF Report"
icon: "FILE"
reusable: false
requestUrl: "run:HLW/GENERATE_PDF_SCRIPT"
requestOptions:
  timeout: 120
requestScript: |
  // Prepare report parameters
  context.reportDate = new Date().toISOString();
  context.includeCharts = true;
responseType: ASSET
targetGroups:
  - "ADMIN"
inputSchema:
  type: object
  title: "Report Options"
  properties:
    startDate:
      type: integer
      title: "Start Date"
      hints:
        subtype: date
    endDate:
      type: integer
      title: "End Date"
      hints:
        subtype: date
    format:
      type: string
      title: "Format"
      enum: ["PDF", "Excel"]
      default: "PDF"
  required:
    - startDate
    - endDate
```

**Service with REST API and Response Transformation:**

```yaml
name: "HLW/SRV_API_INTEGRATION"
caption: "External API Integration"
icon: "CLOUD"
reusable: true
requestUrl: "https://api.example.com/v1/process?userId=${user_id}&action=${action}"
requestOptions:
  timeout: 30
requestScript: |
  // Add authentication header
  context.headers = {
    "Authorization": "Bearer " + context.api_token,
    "Content-Type": "application/json"
  };
responseType: MESSAGE
responseFilter: ".result.message"
responseScript: |
  // Transform response message
  response = "<div class='success'><strong>Success:</strong> " + response + "</div>";
targetGroups:
  - "ADMIN"
inputSchema:
  type: object
  title: "API Request"
  properties:
    user_id:
      type: string
      title: "User ID"
    action:
      type: string
      title: "Action"
      enum: ["create", "update", "delete"]
    api_token:
      type: string
      title: "API Token"
      hints:
        subtype: password
  required:
    - user_id
    - action
    - api_token
```

**Service with cURL Command:**

```yaml
name: "HLW/SRV_CURL_SERVICE"
caption: "cURL Service"
icon: "TERMINAL"
reusable: true
requestUrl: "curl -X POST https://api.example.com/endpoint -H 'Authorization: Bearer ${token}' -d 'data=${data}'"
requestOptions:
  timeout: 60
responseType: TERMINAL
responseScript: |
  // Format terminal output
  response = "<pre>" + response + "</pre>";
targetGroups:
  - "ADMIN"
inputSchema:
  type: object
  properties:
    token:
      type: string
      title: "API Token"
    data:
      type: string
      title: "Data"
  required:
    - token
    - data
```

**Service with Report Response:**

```yaml
name: "HLW/SRV_GENERATE_SALES_REPORT"
caption: "Generate Sales Report"
icon: "CHART"
reusable: false
requestUrl: "run:HLW/FETCH_SALES_DATA"
requestOptions:
  timeout: 90
requestScript: |
  // Prepare report parameters
  context.reportPeriod = context.start_date + " to " + context.end_date;
responseType: REPORT
responseFilter: ".salesData"
responseTarget: "HLW/RPT_SALES_SUMMARY"
targetGroups:
  - "SALES"
  - "MANAGEMENT"
menus:
  - "Reports"
inputSchema:
  type: object
  title: "Sales Report Parameters"
  properties:
    start_date:
      type: integer
      title: "Start Date"
      hints:
        subtype: date
    end_date:
      type: integer
      title: "End Date"
      hints:
        subtype: date
    includeDetails:
      type: boolean
      title: "Include Details"
      default: true
  required:
    - start_date
    - end_date
```

## Service Response Script Examples

**Transforming Query Response:**

```javascript
// Add computed fields to each order
response.forEach(order => {
  order.totalFormatted = "$" + order.total.toFixed(2);
  order.statusColor = order.status === "completed" ? "green" : "orange";
  order.daysSinceOrder = Math.floor((Date.now() - order.orderDate) / (1000 * 60 * 60 * 24));
});
```

**Transforming Service Response:**

```javascript
// Restructure response for target service
response = {
  formData: {
    customerId: response.customer.id,
    customerName: response.customer.name,
    orderDetails: {
      items: response.order.items,
      total: response.order.total,
      discount: response.order.discount || 0
    }
  },
  metadata: {
    source: "external_api",
    timestamp: Date.now()
  }
};
```

**Transforming Message Response:**

```javascript
// Format message with HTML
var message = response.message || "Operation completed";
var status = response.status || "success";
var html = "<div class='alert alert-" + status + "'>" + message + "</div>";
response = html;
```

---

# Collections [collections]
> Use this chapter when you need to define NoSQL document structures, store JSON data, or create CRUD interfaces for document management. Requires JSON Schema and a Query with crud enabled.

Collections define data structures managed by the system (NoSQL-like documents). They require a corresponding Query with `crud: true` and `options.collection` pointing to the Collection ID for management. They follow the same JSON Schema structure as Forms but with additional hints for UI rendering.

**Related Topics:** Collections are managed through [Queries](#queries) with CRUD enabled. They use the same JSON Schema structure as [Forms](#forms).

## YAML Structure

```yaml
type: object
title: "Collection Title"
description: "Collection description"
hints:
  idExpression: ".cliente"      # JQ expression for unique ID (e.g., ".code", ".cliente")
properties:
  cliente:
    type: string
    title: "Cliente"
    description: "ID del Cliente"
    hints:
      valueProvider: PROJECT/VP_CLIENTES  # Use Value Provider for dropdown
  code:
    type: string
    title: "Code"
  name:
    type: string
    title: "Name"
required:
  - code
  - name
```

**Key Fields:**

- **`idExpression`**: JQ expression that extracts the unique identifier from the document (e.g., `.cliente`, `.code`). This value is stored in the `id` column.
- **`properties`**: Field definitions following JSON Schema format
- **`hints`**: UI hints for form rendering (same as Forms)

## Storage Structure

Collections are stored in the `document.document` table with the following structure:

**Table: `document.document`**

| Column | Type | Description |
|--------|------|-------------|
| `oid` | UUID | Primary key (auto-generated) |
| `id` | VARCHAR | Unique identifier within the collection (from `idExpression`) |
| `collection` | VARCHAR | Collection ID (e.g., "PROJECT/COLLECTION_ID") |
| `body` | JSONB | Complete document as JSON |

**Example Document:**

```json
{
  "$id": "customer123",
  "$oid": "550e8400-e29b-41d4-a716-446655440000",
  "cliente": "customer123",
  "cafe_favorito": "ESPRESSO",
  "edulcorante": true
}
```

The `$id` field is stored in the `id` column, and the entire JSON object (including `$id` and `$oid`) is stored in the `body` JSONB column.

## Querying Collections with SQL

You can query Collections directly using SQL by accessing the `document.document` table. The document data is stored in the `body` JSONB column, and you can extract fields using PostgreSQL JSONB operators.

**Basic Query Structure:**

```sql
SELECT 
  oid::varchar AS "$oid",
  id AS "$id",
  body#>>'{fieldName}' AS field_name
FROM document.document
WHERE collection = 'PROJECT/COLLECTION_ID'
```

**Extracting Fields from JSONB:**

Use the `body#>>'{fieldName}'` operator to extract field values as text:

```sql
SELECT 
  oid::varchar AS "$oid",
  id AS "$id",
  body#>>'{cliente}' AS cliente,
  body#>>'{cafe_favorito}' AS cafe_favorito,
  body#>>'{edulcorante}' AS edulcorante
FROM document.document
WHERE collection = 'PROJECT/CAFE_FAVORITO'
```

**Type Conversions:**

For proper type handling in queries, cast extracted values based on their field types:

**Dates:**

```sql
-- For date fields (stored as milliseconds timestamp)
(TIMESTAMP with time zone 'epoch' + 
  ((body#>>'{order_date}')::numeric) * INTERVAL '1 millisecond')::DATE AS order_date

-- For datetime/timestamp fields
(TIMESTAMP with time zone 'epoch' + 
  ((body#>>'{created_at}')::numeric) * INTERVAL '1 millisecond') AS created_at
```

**Numbers:**

```sql
-- For numeric fields (integer, decimal, float, etc.)
(body#>>'{price}')::numeric AS price
(body#>>'{quantity}')::integer AS quantity
```

**Booleans:**

```sql
-- For boolean fields
(body#>>'{is_active}') = 'true' AS is_active
```

**Complete Query Example:**

```sql
SELECT 
  oid::varchar AS "$oid",
  id AS "$id",
  body#>>'{cliente}' AS cliente,
  body#>>'{cafe_favorito}' AS cafe_favorito,
  (body#>>'{edulcorante}') = 'true' AS edulcorante,
  (TIMESTAMP with time zone 'epoch' + 
    ((body#>>'{created_date}')::numeric) * INTERVAL '1 millisecond')::DATE AS created_date
FROM document.document
WHERE collection = 'PROJECT/CAFE_FAVORITO'
  AND body#>>'{cafe_favorito}' = 'ESPRESSO'
ORDER BY body#>>'{cliente}'
```

**Filtering and Sorting:**

You can filter and sort using JSONB path expressions:

```sql
-- Filter by nested field
WHERE body#>>'{status}' = 'active'

-- Filter by nested object field
WHERE body#>>'{customer,name}' = 'John Doe'

-- Filter by array element
WHERE body#>>'{tags,0}' = 'important'

-- Sort by extracted field
ORDER BY body#>>'{name}' ASC
```

## Using Collections in Queries

When creating a Query for a Collection, you can use `dataSourceOptions.collection` to automatically generate SQL:

```yaml
caption: "Gestión de Collection"
icon: "COFFEE"
dataSourceOptions:
  collection: "PROJECT/COLLECTION_ID"  # Automatically generates SQL for Collection
columns:
  - name: cliente
    caption: "Cliente"
    fieldType: string
  - name: cafe_favorito
    caption: "Café Favorito"
    fieldType: string
keys:
  - "$oid"  # Use $oid as primary key
options:
  collection: "PROJECT/COLLECTION_ID"  # Reference to Collection
  crud: true                           # Enable CRUD operations
```

The system automatically generates the appropriate SQL with type conversions based on the column `fieldType` definitions.

## CRUD Query for Collections

To enable CRUD operations on a Collection, create a Query that references it:

```yaml
caption: "Gestión de Collection"
icon: "COFFEE"
columns:
  - name: cliente
    caption: "Cliente"
    fieldType: string
keys:
  - "$oid"  # Primary key (or use $id for the idExpression field)
options:
  collection: "PROJECT/COLLECTION_ID"  # Reference to Collection
  crud: true                           # Enable CRUD operations
scripts:
  - "PROJECT/SCRIPT_ID"                # Optional custom scripts
```

The Collection's schema defines the form used for create/edit operations, and the Query defines how the data is displayed in the grid.

**Important Notes:**

- The `$oid` field is always available and can be used as a primary key
- The `id` field (from `idExpression`) is the unique identifier within the collection
- When using `crud: true`, the system automatically handles create, read, update, and delete operations
- Updates use PostgreSQL's `jsonb_set()` function to modify specific fields in the JSONB document
- The `body` column contains the complete document structure
- The Collection schema is used to generate forms for create/edit operations
- Value Providers can be used in Collection field hints for dropdowns

---

# Forms [forms]
> Use this chapter when you need to create user input forms, define field types and validation rules, configure dynamic behavior with dependents, or add custom UI hints for rendering.

Forms define user input interfaces using JSON Schema. They support various field types, validation, and advanced features for dynamic behavior.

**Related Topics:** Forms can use [Value Providers](#value-providers) for dropdown fields and [Scripts](#scripts) for validation and feedback actions.

## Key Hints (`hints`)

Forms use `hints` to configure field behavior and UI rendering:

- **`subtype`**:
    - `date`, `datetime`, `time`: Date pickers.
    - `upload`: File uploader (`resourcePath`, `acceptedFileTypes`, `multiple`, `dropLabel`).
    - `hyperlink`: Clickable link.
    - `expression`: Read-only calculated value.
    - `code`: Syntax highlighting (e.g., `python`, `javascript`, `sql`).
    - `number`: Numeric input field.
    - `multi-line-text`: Multi-line text area.
- **`valueProvider`**: Source for dropdowns.
    - `people`, `groups`, `projects`.
    - `QUERY_ID`: Use a query's results (Value Provider).
- **`callToAction`**: Button label for form submission (at form level) or field action.
- **`fieldOrder`**: List of field names to define render order.
- **`defaults`**: Script to set initial values.
- **`display`**: `"grid"` to render arrays as tables.
- **`selectOnly`**: `true` to allow selection only in grids.
- **`titlePlural`**: Plural title for array fields (e.g., "Operaciones").
- **`rowStart` / `rowEnd`**: `true` to force layout breaks.
- **`colspan`**: Number of columns to span.
- **`readOnly`**: `true` to disable editing.
- **`dependents`**: List of field names that depend on this field (for conditional display/validation).
- **`dropLabel`**: Custom label for file upload drop zone (e.g., "Suelte su archivo aquí").
- **`validation`**: Script to validate input.
    ```yaml
    validation: |
      #!python3
      import redflagbpm
      bpm = redflagbpm.BPMService()
      if bpm.context.input['val'] < 0:
        bpm.fail("Invalid")
    ```
- **`feedback`**: Scripts triggered by buttons or events. Array of feedback actions:
    ```yaml
    feedback:
      - "Button Label": ""
        try: "code to execute"
        except: "error handling"
        if condition: "raise '[[[Message]]]'"
        else: "raise '[[[Other message]]]'"
    ```

## Default Values

Fields can have default values:
- **`default: "today"`**: For date fields, sets to current date.
- **`default: "---"`**: For string fields, sets a placeholder value.
- **`default: 0`**: For numeric fields.

## Advanced Features

**Dynamic Defaults (Python):**

```yaml
hints:
  defaults: |
    #!python3
    import time
    import redflagbpm
    bpm = redflagbpm.BPMService()
    # Set default date to now
    bpm.context.input["date"] = round(time.time() * 1000)
```

**Validation Script (Python):**

```yaml
hints:
  validation: |
    #!python3
    import redflagbpm
    bpm = redflagbpm.BPMService()
    price = bpm.context.input.get('price')
    if price is None or price <= 0:
       raise 'Price must be greater than 0'
```

**Layout Control:**

```yaml
properties:
  field1:
    type: string
    hints:
      colspan: 2       # Span 2 columns
      rowStart: true   # Force start of new row
      rowEnd: true     # Force end of row after this field
```

**File Upload:**

```yaml
properties:
  files:
    type: string
    title: "Attachments"
    hints:
      subtype: upload
      resourcePath: "uploads/temp" # Target directory
      multiple: true
      acceptedFileTypes: "image/*,application/pdf"
```

**Computed Expressions (VTL):**

```yaml
properties:
  summary:
    type: string
    hints:
      subtype: expression
      value: |
        #!vtl
        <b>User: $!{user_name}</b>
```

## Complete Examples

**Form with Dependent Fields:**

```yaml
properties:
  comitente:
    type: string
    title: "Comitente"
    default: "---"
    hints:
      dependents:
        - expr
      rowStart: true
      rowEnd: true
  expr:
    title: "..."
    type: string
    hints:
      subtype: expression
      value: |
        #!vtl
        <b>Hola $!{comitente}</b>
      rowStart: true
      rowEnd: true
```

**Form with Arrays as List:**

```yaml
properties:
  operaciones_list:
    type: array
    items:
      title: Operación
      type: object
      properties:
        operacion:
          title: Operación
          type: string
          enum: ["COMPRA", "VENTA"]
      hints:
        fieldOrder: ["operacion", "precio"]
        titlePlural: Operaciones
```

**Form with Arrays as Grid:**

```yaml
properties:
  operaciones_grid:
    type: array
    items:
      hints:
        display: grid
        fieldOrder: ["operacion", "precio", "cantidad"]
        titlePlural: Operaciones
      title: Operación
      type: object
      properties:
        operacion:
          title: Operación
          type: string
          enum: ["COMPRA", "VENTA"]
```

---

# Pages [pages]
> Use this chapter when you need to create custom HTML pages, generate dynamic content with Python/JavaScript/Shell scripts, or use Velocity templates for page rendering.

Pages are custom UI resources that can be static HTML files or dynamic scripts generating HTML content.

## Page Types

- **Static**: HTML or TXT files.
- **Dynamic**: Python/JS scripts generating HTML by printing to stdout. The system captures the output and renders it as HTML.
- **Velocity**: `.vhtml` templates with `$variables`.

## Generating HTML with Dynamic Languages

When using dynamic languages (Python/Jython, JavaScript, or Shell) to generate HTML, the script must **print the HTML content to stdout**. The system captures this output and renders it as HTML in the browser.

**Python/Jython Example:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()
param_value = bpm.context.param

# Print HTML to stdout - this is what gets rendered
print("""
<!DOCTYPE html>
<html>
<head><title>Dynamic Page</title></head>
<body>
    <h1>Hello, {}</h1>
    <p>This HTML was generated by Python.</p>
</body>
</html>
""".format(param_value))
```

**JavaScript Example:**

```javascript
// Print HTML to stdout - this is what gets rendered
print(`
<!DOCTYPE html>
<html>
<head><title>Dynamic Page</title></head>
<body>
    <h1>Hello from JavaScript</h1>
    <p>This HTML was generated by JavaScript.</p>
</body>
</html>
`);
```

**Shell Script Example:**

```bash
#!/bin/bash
# Print HTML to stdout - this is what gets rendered
cat <<EOF
<!DOCTYPE html>
<html>
<head><title>Dynamic Page</title></head>
<body>
    <h1>Hello from Shell</h1>
    <p>This HTML was generated by a shell script.</p>
</body>
</html>
EOF
```

**Important:** The system captures everything printed to stdout and renders it as HTML. Any output to stderr or other streams will not be included in the rendered page.

## Accessing Parameters

**URL Parameters:**

Access pages via URL: `.../Page?__page__=ID&param=value`

**Python:**

```python
#!python3
import redflagbpm
bpm = redflagbpm.BPMService()
param_value = bpm.context.param
```

**Velocity:**

```vtl
$param
```

## Configuration

The `.config` file for Pages can include:

```yaml
properties:
  title: "Page Title"  # Title displayed in browser/tab
```

**Language Types:**

- `lang: "Velocity"` for `.vhtml` files (Velocity templates)
- `lang: "Python"` for Python scripts
- `lang: "JavaScript"` for JavaScript files

---

# Reports [reports]
> Use this chapter when you need to generate PDF, Excel, or HTML documents from templates and data. Supports JasperReports templates and Velocity templates with data from queries or scripts.

Reports generate documents (PDF, Excel, HTML, etc.) from templates and data.

**Related Topics:** Reports can be generated from [Services](#services) and [Scripts](#scripts) using the `reportService` API. They can use Velocity templates or JasperReports format.

## Report Types

- **Jasper Reports**: `.jrxml` files (JasperReports format).
    - Connection: `com.jaspersoft.studio.data.defaultdataadapter` must match BPM connection name.
    - Parameters: "Is For Prompting" creates form fields for user input.
    - **Resources**: Additional files (images, etc.) referenced in the report can be specified in the `.config`:
      ```yaml
      properties:
        resources:
          - "cherry.jpg"
          - "coffee_stain.png"
        targetGroups:
          - "ADMIN"
      ```

- **Velocity Templates**: `.vhtml` templates + Python script.
    - Python script prepares data (`content`) and calls `bpm.reportService.renderReport()` or `bpm.reportService.renderReports()`.

**Configuration:**

- `lang: "Binary"` for binary report files (e.g., `.jrxml`)
- `properties.resources`: Array of resource file names used by the report
- `properties.targetGroups`: Groups allowed to access the report

## Using `renderReport` and `renderReports`

The `redflagbpm` library provides `reportService` with two methods for generating reports:

### `renderReport(options, reportData)`

Generates a single report and returns the file path or URL.

**Parameters:**

- **`options`** (dict): Report generation options
  - **`format`** (str): Output format - `"PDF"`, `"HTML"`, `"XLSX"`, `"ODT"`, `"DOCX"`, `"JPG"` (default: `"PDF"`)
  - **`options`** (dict): Format-specific options
    - **`resource`** (str): Output file path (e.g., `"/tmp/report.pdf"`)
    - **`autoprint`** (bool): For PDF, automatically open print dialog
    - **`silentPrint`** (bool): For PDF, silently print without dialog
    - **`password`** (str): For PDF, set password protection
    - **`pageIndex`** (int): For JPG, specific page to render
    - **`reportIndex`** (int): For JPG, specific report index
    - **`zoom`** (float): For JPG, zoom factor

- **`reportData`** (dict): Report data and configuration
  - **`reportName`** (str): Report ID or path (e.g., `"PROJECT/REPORT_ID"`)
  - **`parameters`** (dict): Parameters to pass to the report template
  - **`content`** (str, optional): Content data (for JSON_STRING contentType)
  - **`contentType`** (str, optional): Content type - `"JSON_URL"`, `"JSON_RESOURCE"`, `"JSON_STRING"`, `"SQL"`, `"NONE"` (default: `"NONE"`)
  - **`request`** (dict, optional): Additional request data

**Example:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Configure report options
options = {
    "format": "PDF",
    "options": {
        "resource": "/tmp/customer_report.pdf"
    }
}

# Prepare report data
reportData = {
    "reportName": "HLW/Clientes",
    "parameters": {
        "customer_id": "12345",
        "start_date": "2024-01-01",
        "end_date": "2024-12-31"
    }
}

# Generate report
result = bpm.reportService.renderReport(options=options, reportData=reportData)
print(f"Report generated: {result}")
```

**Example with Velocity Template:**

```python
#!python3
import redflagbpm
from datetime import datetime

bpm = redflagbpm.BPMService()

# Prepare data for Velocity template
content = {
    "customers": [
        {"name": "John Doe", "email": "john@example.com", "total": 1500.00},
        {"name": "Jane Smith", "email": "jane@example.com", "total": 2300.50}
    ],
    "report_date": datetime.now().strftime("%Y-%m-%d")
}

# Configure report options
options = {
    "format": "PDF",
    "options": {
        "resource": "/tmp/customers.pdf",
        "autoprint": False
    }
}

# Prepare report data with JSON content
reportData = {
    "reportName": "HLW/CUSTOMER_LIST",
    "parameters": {
        "title": "Customer Report",
        "company": "My Company"
    },
    "content": str(content).replace("'", '"'),  # Convert to JSON string
    "contentType": "JSON_STRING"
}

# Generate report
result = bpm.reportService.renderReport(options=options, reportData=reportData)
print(f"Report generated: {result}")
```

### `renderReports(options, reports)`

Generates multiple reports and returns a combined file path or URL.

**Parameters:**

- **`options`** (dict): Same as `renderReport` - applies to all reports
- **`reports`** (list): List of report data dictionaries, each with the same structure as `reportData` in `renderReport`

**Example:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Configure common options for all reports
options = {
    "format": "PDF",
    "options": {
        "resource": "/tmp/combined_reports.pdf"
    }
}

# Prepare multiple reports
reports = [
    {
        "reportName": "HLW/SALES_REPORT",
        "parameters": {
            "month": "January",
            "year": 2024
        }
    },
    {
        "reportName": "HLW/EXPENSES_REPORT",
        "parameters": {
            "month": "January",
            "year": 2024
        }
    }
]

# Generate combined report
result = bpm.reportService.renderReports(options=options, reports=reports)
print(f"Combined report generated: {result}")
```

## Report Options Details

**Format Options:**

- **PDF**: Default format, supports `autoprint`, `silentPrint`, `password`
- **HTML**: Web-friendly format
- **XLSX**: Excel format
- **ODT**: OpenDocument Text format
- **DOCX**: Microsoft Word format
- **JPG**: Image format, supports `pageIndex`, `reportIndex`, `zoom`

**Content Types:**

- **`NONE`** (default): Report uses its own data source (JasperReports connection or Velocity template variables)
- **`JSON_STRING`**: Content is provided as a JSON string in the `content` field
- **`JSON_URL`**: Content is loaded from a URL specified in the `content` field
- **`JSON_RESOURCE`**: Content is loaded from a resource path specified in the `content` field
- **`SQL`**: Content is generated from SQL query (advanced usage)

**Example with Different Formats:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# PDF with password protection
options_pdf = {
    "format": "PDF",
    "options": {
        "resource": "/tmp/secure_report.pdf",
        "password": "secret123"
    }
}

# Excel format
options_xlsx = {
    "format": "XLSX",
    "options": {
        "resource": "/tmp/data.xlsx"
    }
}

# HTML format
options_html = {
    "format": "HTML",
    "options": {
        "resource": "/tmp/report.html"
    }
}

reportData = {
    "reportName": "HLW/MY_REPORT",
    "parameters": {
        "param1": "value1"
    }
}

# Generate in different formats
pdf_result = bpm.reportService.renderReport(options=options_pdf, reportData=reportData)
xlsx_result = bpm.reportService.renderReport(options=options_xlsx, reportData=reportData)
html_result = bpm.reportService.renderReport(options=options_html, reportData=reportData)
```

## Report Data Sources

Reports can get data from different sources:

**1. JasperReports Connection:**

```python
# Report uses connection defined in JasperReports
reportData = {
    "reportName": "HLW/Clientes",
    "parameters": {
        "customer_id": "12345"  # Parameters for report query
    }
}
```

**2. JSON Content:**

```python
# Provide data directly as JSON
reportData = {
    "reportName": "HLW/CUSTOMER_LIST",
    "parameters": {
        "title": "Customer List"
    },
    "content": '{"customers": [{"name": "John", "total": 100}]}',
    "contentType": "JSON_STRING"
}
```

**3. External URL:**

```python
# Load data from URL
reportData = {
    "reportName": "HLW/EXTERNAL_DATA",
    "parameters": {},
    "content": "https://api.example.com/data.json",
    "contentType": "JSON_URL"
}
```

**4. Resource File:**

```python
# Load data from resource
reportData = {
    "reportName": "HLW/RESOURCE_DATA",
    "parameters": {},
    "content": "PROJECT/data.json",
    "contentType": "JSON_RESOURCE"
}
```

## Complete Example: Velocity Template Report

```python
#!python3
import redflagbpm
from datetime import datetime, timedelta

bpm = redflagbpm.BPMService()

# Fetch data from query or database
customers = bpm.documentService.readList(
    collection="customers",
    criteria="status = ?",
    parameters={"1": "active"}
)

# Prepare data for Velocity template
report_data = {
    "customers": customers,
    "report_date": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
    "total_customers": len(customers),
    "summary": {
        "active": len([c for c in customers if c.get("status") == "active"]),
        "inactive": len([c for c in customers if c.get("status") == "inactive"])
    }
}

# Convert to JSON string for Velocity template
import json
content_json = json.dumps(report_data, default=str)

# Configure report
options = {
    "format": "PDF",
    "options": {
        "resource": f"/tmp/customer_report_{datetime.now().strftime('%Y%m%d')}.pdf"
    }
}

reportData = {
    "reportName": "HLW/CUSTOMER_REPORT",
    "parameters": {
        "title": "Customer Activity Report",
        "company_name": "My Company"
    },
    "content": content_json,
    "contentType": "JSON_STRING"
}

# Generate report
result = bpm.reportService.renderReport(options=options, reportData=reportData)
bpm.reply(f"Report generated successfully: {result}")
```

---

# Scripts [scripts]
> Use this chapter when you need to write automation scripts, access BPM services via redflagbpm library, create action scripts for queries, or implement validation and feedback logic.

Scripts are the core logic engine. They use the `redflagbpm` python library.

**Related Topics:** Scripts can be used as actions in [Queries](#queries), validation/feedback in [Forms](#forms), and can be exposed as [Endpoints](#endpoints). See also [Post Processors](#post-processors) for row-level processing.

## Script Types

- **Script Task**: Part of a BPMN process.
- **Endpoint**: Callable via HTTP (`/api/run/ID`).
    - **Public endpoint**: No authentication (`type: "Script"`).
    - **User endpoint**: Requires login (`type: "User endpoint"`).
    - **Web Service endpoint**: Token authentication.
- **Action**: Triggered from Queries.
- **Feedback/Validation**: Triggered from Forms.

## Scripts with Forms

Scripts can have an associated form defined in the `.config` file's `properties.inputSchema`:

```yaml
properties:
  caption: "Action Name"
  selectionScope: 1
  inputSchema:
    type: object
    title: "Form Title"
    hints:
      fieldOrder: ["field1", "field2"]
      callToAction: "Submit"
      feedback:
        - "Check Value": |
            #!python3
            import redflagbpm
            bpm = redflagbpm.BPMService()
            try:
              edulcorante = bpm.context.input.get('edulcorante', False)
            except:
              edulcorante = False
            if edulcorante:
              raise '[[[Sí, tiene edulcorante]]]'
            else:
              raise '[[[No, no tiene edulcorante]]]'
        - "Validate": |
            #!python3
            import redflagbpm
            bpm = redflagbpm.BPMService()
            field1 = bpm.context.input.get('field1', '')
            if not field1 or len(field1) < 3:
              raise '[[[El campo debe tener al menos 3 caracteres]]]'
            # Modify input values
            bpm.context.input['field1'] = field1.upper()
            # Return success message
            raise '[[[Validación exitosa]]]'
    properties:
      field1:
        type: string
        title: "Field Label"
      edulcorante:
        type: boolean
        title: "Tiene Edulcorante"
    required:
      - field1
```

## Feedback Script Structure

Feedback actions are defined as an array of objects, where each object represents a button with its associated script. The key is the button label, and the value is the Python3 script code.

**Feedback Script Context:**

Feedback scripts must import `redflagbpm` and create a `BPMService` instance to access form values:

```python
import redflagbpm
bpm = redflagbpm.BPMService()
```

Feedback scripts have access to:
- **`bpm.context.input`**: Dictionary containing all form field values (e.g., `bpm.context.input.get('field1')`)
- **`bpm.context.input['fieldName']`**: Direct access to form fields (can be modified to update form values)
- **`bpm.context.selection`**: Selected items from query (if applicable)
- **`bpm.context`**: Full context object with additional properties

**Feedback Script Behavior:**

- **Import `redflagbpm`**: Always start with `import redflagbpm` and `bpm = redflagbpm.BPMService()`
- **Access form values**: Use `bpm.context.input.get('fieldName', default)` or `bpm.context.input['fieldName']`
- **`try/except`**: Handle exceptions gracefully when accessing form fields
- **`if/else`**: Conditional logic based on form values
- **`raise '[[[Message]]]'`**: Display message to user (extracts text between `[[[` and `]]]`)
- **Modify form values**: Changes to `bpm.context.input['fieldName']` update form field values dynamically

**Example Feedback Script:**

```python
#!python3
import redflagbpm
bpm = redflagbpm.BPMService()

# Access form field values using bpm.context.input
try:
    cafe_favorito = bpm.context.input.get('cafe_favorito')
    edulcorante = bpm.context.input.get('edulcorante', False)
except:
    cafe_favorito = None
    edulcorante = False

# Conditional logic
if cafe_favorito == "ESPRESSO":
    # Modify form field
    bpm.context.input['edulcorante'] = True
    raise '[[[Espresso seleccionado - Edulcorante habilitado]]]'
elif cafe_favorito:
    # Check another condition
    if edulcorante:
        raise '[[[Sí, tiene edulcorante]]]'
    else:
        raise '[[[No, no tiene edulcorante]]]'
else:
    raise '[[[Por favor seleccione un café]]]'
```

**Accessing Form Values:**

Use `bpm.context.input` dictionary to read and modify form field values:

```python
#!python3
import redflagbpm
bpm = redflagbpm.BPMService()

# Read form field with default value
field1 = bpm.context.input.get('field1', '')

# Read form field directly (raises KeyError if missing)
field2 = bpm.context.input['field2']

# Modify form field value
bpm.context.input['field1'] = 'new value'

# Check if field exists
if 'field1' in bpm.context.input:
    value = bpm.context.input['field1']
```

**Feedback Message Format:**

Messages displayed to users use the format `[[[Message text]]]`. The text between the triple brackets is extracted and shown to the user. This allows displaying user-friendly messages even when exceptions occur.

## redflagbpm Library

**Basic Structure:**

```python
import redflagbpm
bpm = redflagbpm.BPMService()
# Logic...
bpm.reply(result) # or bpm.fail("Error")
```

**Key Services (`bpm.service`):**

- `sendMail(msg)`: Send email.
- `execute(script, context)`: Run another script.
- `notifyUser(user, title, desc, target)`: Send notification.
- `env()`: Get environment variables.
- `tokenize(user, payload)`: Generate magic links.

**Logging:**

- `bpm.logger.info(category, message)`: Log informational messages.
  ```python
  bpm.logger.info("bpm.code/PROJECT", f"Processing: {data}")
  ```

**Document Service (`bpm.documentService`):**

- `create(collection, object)`
- `readById(collection, id)`
- `updateById(collection, id, data)`
- `deleteById(collection, id)`

**Runtime Service (`bpm.runtimeService`):**

- `startProcessInstanceByKey(key, businessKey, vars)`
- `setVariable(execId, name, value)`
- `getVariable(execId, name)`

**Execution Service (`bpm.execution` - Process Context Only):**

- `getVariable(name)`
- `setVariable(name, value)`
- `getBusinessKey()`

## Action Scripts - Accessing Selection

Action scripts (scripts triggered from Queries) receive selection data differently depending on the `selectionScope` value:

**When `selectionScope != 1` (multiple or any selection):**

- Selected rows are available as an array in `bpm.context.selection`
- Access via `bpm.context.selection[0].propertyName` for the first item
- Iterate through `bpm.context.selection` for multiple items

**When `selectionScope = 1` (single selection):**

- The selected row's properties are also **merged directly into the context** for convenience.
- Access properties directly via `bpm.context.propertyName`
- `bpm.context.selection` array will have a single item.

**Example: Script with `selectionScope = 1`:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# When selectionScope = 1, properties are directly accessible. Also available via bpm.context.selection[0]
id_doc = bpm.context.id_doc          # Direct access
nombre = bpm.context.nombre          # Direct access
estado = bpm.context.estado          # Direct access

# Process the selected row
bpm.documentService.updateById(
    collection="mi_coleccion",
    id=id_doc,
    data={"estado": "actualizado"}
)

bpm.reply({"type": "MESSAGE", "statusMessage": f"Estado actualizado para {nombre}"})
```

**Example: Script with `selectionScope = -2` (multiple selection):**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# When selectionScope != 1, use the selection array
selected_count = len(bpm.context.selection)
for item in bpm.context.selection:
    id_doc = item.get('id_doc')
    nombre = item.get('nombre')
    # Process each selected item
    bpm.documentService.updateById(
        collection="mi_coleccion",
        id=id_doc,
        data={"estado": "procesado"}
    )

bpm.reply({"type": "MESSAGE", "statusMessage": f"Procesados {selected_count} registros"})
```

**Configuration for single selection action:**

```yaml
properties:
  caption: "Actualizar Estado"
  selectionScope: 1              # Single selection - properties merged directly
  type: "CALL"                   # Execute directly
```

**Important:** When creating action scripts with `selectionScope = 1`, properties are available both directly from `bpm.context.propertyName` (recommended) and via `bpm.context.selection[0].propertyName`. The direct access is preferred for convenience, but both methods work.

## Action Script Responses

Scripts triggered from Queries should return a specific structure to control the UI. The response can be:
- A JSON object with a `type` field (structured response)
- A simple string or any other value (converted to string message)

**Response Structure:**

```python
bpm.reply({
    "type": "RESPONSE_TYPE",           # Required for structured responses
    "statusMessage": "Message text",    # Optional: message to display
    "statusMessageContentType": "text/html",  # Optional: "text/plain" (default) or "text/html"
    "content": {...},                   # Optional: JSON object with additional data
    "service": "SERVICE_ID",            # Optional: Service ID to open
    "query": "QUERY_ID",                # Optional: Query ID to execute
    "report": {...},                    # Optional: Report data object
    "asset": "/path/to/file",           # Optional: File path to download
    "assetContentType": "application/pdf"  # Optional: MIME type for asset
})
```

**Response Types:**

1. **`MESSAGE`**: Display a status message to the user
   ```python
   bpm.reply({"type": "MESSAGE", "statusMessage": "Operation completed"})
   ```

2. **`TERMINAL_UPDATE`**: Refresh the query (reload data)
   ```python
   bpm.reply({"type": "TERMINAL_UPDATE"})
   ```

3. **`TERMINAL_DELETE`**: Delete selected row(s) and refresh query
   ```python
   bpm.reply({"type": "TERMINAL_DELETE"})
   ```

4. **`TERMINAL_CREATE`**: Create new row and refresh query
   ```python
   bpm.reply({"type": "TERMINAL_CREATE"})
   ```

5. **`TERMINAL`**: Display terminal-style output (monospace, scrollable)
   ```python
   bpm.reply({"type": "TERMINAL", "statusMessage": "Output text"})
   ```

6. **`ERROR`**: Display error message
   ```python
   bpm.reply({"type": "ERROR", "statusMessage": "An error occurred"})
   ```

7. **`OPEN`**: Open a service in the same window
   ```python
   bpm.reply({
       "type": "OPEN",
       "service": "PROJECT/SERVICE_ID",
       "content": {"param1": "value1"}  # Optional: parameters for service
   })
   ```

8. **`OPEN_UPDATE`**: Open a service and refresh query
   ```python
   bpm.reply({
       "type": "OPEN_UPDATE",
       "service": "PROJECT/SERVICE_ID",
       "content": {"param1": "value1"}
   })
   ```

9. **`OPEN_NEW`**: Open URL in new tab/window
   ```python
   bpm.reply({
       "type": "OPEN_NEW",
       "statusMessage": "https://example.com"
   })
   ```

10. **`OPEN_NEW_UPDATE`**: Open URL in new tab and refresh query
    ```python
    bpm.reply({
        "type": "OPEN_NEW_UPDATE",
        "statusMessage": "https://example.com?q=value"
    })
    ```

11. **`QUERY`**: Execute a query with parameters
    ```python
    bpm.reply({
        "type": "QUERY",
        "query": "PROJECT/QUERY_ID",
        "content": {"filter1": "value1"},  # Optional: query parameters
        "statusMessage": "Query executed"   # Optional: message
    })
    ```

12. **`REPORT`**: Generate a report
    ```python
    bpm.reply({
        "type": "REPORT",
        "report": {
            "reportName": "PROJECT/REPORT_ID",
            "parameters": {"param1": "value1"}
        }
    })
    ```

13. **`ASSET`**: Download a file
    ```python
    bpm.reply({
        "type": "ASSET",
        "asset": "/path/to/file.pdf",
        "assetContentType": "application/pdf"  # Optional: auto-detected if omitted
    })
    ```

**HTML Messages:**

To display HTML content in messages, set `statusMessageContentType`:

```python
bpm.reply({
    "type": "MESSAGE",
    "statusMessage": "<h1>Success!</h1><p>Operation completed.</p>",
    "statusMessageContentType": "text/html"  # Default is "text/plain"
})
```

**Simple String Response:**

If you return a simple string or value without a `type` field, it's converted to a message:

```python
bpm.reply("Simple message")  # Equivalent to {"type": "TERMINAL", "statusMessage": "Simple message"}
```

**Content Parameter:**

The `content` field is a JSON object that can be used to pass parameters to services or queries:

```python
bpm.reply({
    "type": "OPEN",
    "service": "PROJECT/MY_SERVICE",
    "content": {
        "userId": "123",
        "action": "edit",
        "data": {"field1": "value1"}
    }
})
```

**Configuration Language:**

- `lang: "Python"` - Uses Jython (Java-based Python interpreter)
- `lang: "Shell"` - Processes the shebang (`#!`) line to determine the interpreter. Use this if you need to use system Python3 (with shebang `#!/usr/bin/env python3`) instead of Jython.

## Complete Examples

**Sending Email with Attachments:**

```python
#!python3
import redflagbpm
bpm = redflagbpm.BPMService()

# Create a temporary file
temp_file = "/tmp/report.txt"
with open(temp_file, 'w') as f:
    f.write("Report Content")

# Send email
msg = {
    "from": "System<no-reply@domain.com>",
    "to": ["user@domain.com"],
    "subject": "Report",
    "message": "<p>Attached is the report.</p>",
    "attachments": {"Report.txt": temp_file}
}
bpm.service.sendMail(msg)
```

---

# Endpoints [endpoints]
> Use this chapter when you need to create custom HTTP endpoints, expose APIs with different security levels (public, user, web service), or handle HTTP requests and responses with access to headers.

Endpoints allow you to create custom HTTP endpoints that receive and process HTTP requests. These endpoints can be designed with different security levels and allow access to request/response headers and interaction with the BPM context.

## Endpoint Types Overview

1. **Public endpoint**:  
   - Public access, no authentication required (`type: "Script"` in config).
2. **User endpoint**:  
   - Requires authenticated user and uses logged-in user credentials (`type: "User endpoint"` in config).
3. **Web Service endpoint**:  
   - Uses tokens to authenticate requests, providing higher security level. Requires valid JWT token with "WS" role.

## Authentication and Authorization Tokens

### Obtaining an Authorization Token

To obtain a JWT token for authenticating requests to Web Service endpoints, use the `/api/authToken` endpoint:

**Request:**
```bash
GET /api/authToken?user=USERNAME&password=PASSWORD
```

**Example:**
```bash
curl -X GET "http://localhost:8080/api/authToken?user=myuser&password=mypassword"
```

**Response:**
Returns a JWT token as plain text:
```
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJteXVzZXIiLCJpc3MiOiJibXBwbGF0Zm9ybSJ9...
```

**Error Responses:**
- `401 Unauthorized`: Invalid username or password
- `404 Not Found`: User not found

### Signing Data with a Token

To sign data with an existing token, use the `/api/payloadToken` endpoint:

**Request:**
```bash
POST /api/payloadToken?token=YOUR_TOKEN
Content-Type: application/json

{
  "data": "value",
  "timestamp": 1234567890
}
```

**Example:**
```bash
curl -X POST "http://localhost:8080/api/payloadToken?token=YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{"data": "value", "timestamp": 1234567890}'
```

**Response:**
Returns a signed token containing the data:
```
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJkYXRhIjoidmFsdWUiLCJ0aW1lc3RhbXAiOjEyMzQ1Njc4OTB9...
```

### Using Tokens in Requests

Tokens can be provided in three ways:

1. **Query Parameter:**
   ```bash
   GET /api/run/PROJECT/SCRIPT?token=YOUR_TOKEN&param1=value1
   ```

2. **Authorization Header:**
   ```bash
   GET /api/run/PROJECT/SCRIPT?param1=value1
   Authorization: YOUR_TOKEN
   ```
   Or with Bearer prefix:
   ```bash
   Authorization: Bearer YOUR_TOKEN
   ```

3. **Session (for User endpoints):**
   - Automatically handled when user is logged into the web application

**Example with curl:**
```bash
# Using query parameter
curl -X GET "http://localhost:8080/api/run/PROJECT/SCRIPT?token=YOUR_TOKEN&param1=value1"

# Using Authorization header
curl -X GET "http://localhost:8080/api/run/PROJECT/SCRIPT?param1=value1" \
     -H "Authorization: YOUR_TOKEN"

# Using Bearer prefix
curl -X GET "http://localhost:8080/api/run/PROJECT/SCRIPT?param1=value1" \
     -H "Authorization: Bearer YOUR_TOKEN"
```

### Web Service Endpoint Requirements

Web Service endpoints require:

1. **Valid Token**: The token must be valid (not expired, properly signed)
2. **WS Role**: The user associated with the token must have the "WS" role

**Token Validation:**
- The system automatically validates tokens using `securityService().isValid(token)`
- Invalid tokens result in `401 Unauthorized` response

**Role Verification:**
- The system checks for "WS" role using `securityService().hasRole(token, "WS")`
- Users without "WS" role cannot access Web Service endpoints

**User Information:**
- User ID is extracted from token: `securityService().getPrincipal(token).getString("sub")`
- Available in endpoint scripts via `bpm.context.json._requestHeaders.x-user`

### Complete Authentication Flow Example

**Step 1: Obtain Token**
```bash
TOKEN=$(curl -s -X GET "http://localhost:8080/api/authToken?user=myuser&password=mypassword")
echo "Token: $TOKEN"
```

**Step 2: Use Token in Request**
```bash
curl -X GET "http://localhost:8080/api/run/PROJECT/MY_SCRIPT?param1=value1" \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json"
```

**Step 3: Access User Information in Script**
```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Get user from request headers
_requestHeaders = bpm.context.json._requestHeaders
user_id = _requestHeaders.get('x-user', 'unknown')
ws_auth = _requestHeaders.get('x-ws-auth', 'false')

print(f"User: {user_id}, WS Auth: {ws_auth}")
```

### Token Security Best Practices

1. **Store Tokens Securely**: Never commit tokens to version control or log them
2. **Use HTTPS**: Always use HTTPS in production to protect tokens in transit
3. **Token Expiration**: Tokens have expiration times - handle token refresh appropriately
4. **Scope Permissions**: Only grant "WS" role to users who need Web Service access
5. **Validate on Server**: Always validate tokens on the server side, never trust client-side validation

## Context Variables for Request and Response Headers

- **`bpm.context.json._requestHeaders`**: Contains request headers. You can access information sent by the client, such as `User-Agent` or any other header sent with the request.
  - **`x-user`**: User ID extracted from authentication token (available when token is provided)
  - **`x-ws-auth`**: Indicates if user has Web Service role ("true" or "false")
  - **`x-method`**: HTTP method used in the request (GET, POST, etc.)
- **`bpm.context.json._responseHeaders`**: Allows configuring response headers. Useful for defining content type (`Content-Type`), response status (`status`), encoding (`Content-Encoding`), among others.

## Making Requests to an Endpoint

To execute a script via HTTP request in the BPM, you can compose a URL with the following format:

```
/api/run/<SCRIPT_ID>?parametro1=valor&parametro2=valor...
```

**Component Description:**

- **`/api/run/`**: Base endpoint indicating we're executing a script in the BPM.
- **`<SCRIPT_ID>`**: Unique identifier of the script to execute. By convention, this ID should be composed as: `<PROJECT>/<SCRIPT_NAME>`. For example: `MI_PROYECTO/LISTAR_VENTAS`.
- **`?parametro1=valor&parametro2=valor...`**: Parameters passed to the script in the URL. These parameters are accessible within the script as part of the context (`bpm.context`).

**Example Request URL:**

```
/api/run/MI_PROYECTO/LISTAR_VENTAS?fecha_inicio=2024-09-01&fecha_fin=2024-09-30
```

**Using curl:**

```bash
curl -X GET "http://<HOST>/api/run/MI_PROYECTO/LISTAR_VENTAS?fecha_inicio=2024-09-01&fecha_fin=2024-09-30"
```

**With Authentication Headers:**

```bash
curl -X GET "http://localhost:8080/api/run/MI_PROYECTO/LISTAR_VENTAS?fecha_inicio=2024-09-01&fecha_fin=2024-09-30" \
     -H "Authorization: Bearer <TOKEN>" \
     -H "Content-Type: application/json"
```

## Accessing Parameters Within the Script

Within the script executed when making this request, you can access URL parameters using the `bpm.context` object:

```python
#!python3

import redflagbpm
bpm = redflagbpm.BPMService()

# Get parameters passed in URL
fecha_inicio = bpm.context.fecha_inicio
fecha_fin = bpm.context.fecha_fin

# Generate response based on parameters
print(f"Listando ventas desde {fecha_inicio} hasta {fecha_fin}")
```

## Receiving POST Request Body

When making a POST request, the request body is automatically parsed as JSON and its properties are converted into context variables. There is no separate `body` variable - you access the body properties directly from `bpm.context`, just like URL parameters.

**How it works:**

1. The POST body is read as a JSON string from the request input stream
2. The JSON is parsed
3. All properties from the JSON are merged into the script's context variables
4. Properties are accessible directly via `bpm.context.propertyName`

**Example POST Request:**

```bash
curl -X POST "http://localhost:8080/api/run/MI_PROYECTO/mi_script" \
     -H "Content-Type: application/json" \
     -d '{"nombre": "Juan", "edad": 30, "email": "juan@example.com"}'
```

**Accessing Body Properties in Script:**

```python
#!python3

import redflagbpm
import json

bpm = redflagbpm.BPMService()

# Access body properties directly from context
nombre = bpm.context.nombre        # "Juan"
edad = bpm.context.edad            # 30
email = bpm.context.email          # "juan@example.com"

# Process the data
respuesta = {
    "status": "success",
    "mensaje": f"Hola {nombre}, tienes {edad} años",
    "email": email
}

print(json.dumps(respuesta))
```

**Example with Nested Objects:**

If the POST body contains nested objects:

```json
{
  "usuario": {
    "nombre": "Juan",
    "datos": {
      "ciudad": "Madrid"
    }
  }
}
```

Access nested properties:

```python
#!python3

import redflagbpm

bpm = redflagbpm.BPMService()

# Access nested object
usuario = bpm.context.usuario  # Dict/object with nested data

if isinstance(usuario, dict):
    nombre = usuario.get('nombre')
    datos = usuario.get('datos', {})
    ciudad = datos.get('ciudad') if isinstance(datos, dict) else None
    
    print(f"Usuario: {nombre}, Ciudad: {ciudad}")
```

**Example with Validation:**

```python
#!python3

import redflagbpm
import json

bpm = redflagbpm.BPMService()

# Access body properties with error handling
try:
    nombre = bpm.context.nombre
    edad = bpm.context.edad
    email = bpm.context.email
except AttributeError:
    nombre = None
    edad = None
    email = None

# Validate required fields
if not nombre or not email:
    respuesta = {
        "status": "error",
        "message": "Faltan campos requeridos: nombre y email"
    }
    print(json.dumps(respuesta))
else:
    # Process valid data
    respuesta = {
        "status": "success",
        "data": {
            "nombre": nombre,
            "edad": edad,
            "email": email
        }
    }
    print(json.dumps(respuesta))

# Set response headers
_responseHeaders = bpm.context.json._responseHeaders
_responseHeaders["status"] = "200"
_responseHeaders["Content-Type"] = "application/json"
```

**Important Notes:**

1. **JSON Body**: The body must be valid JSON. The system automatically parses it and makes properties available in the context.
2. **Direct Access**: Body properties are accessible directly via `bpm.context.propertyName`, not through a separate `body` variable.
3. **Nested Objects**: Nested JSON objects are also available. If JSON has `{"usuario": {"nombre": "Juan"}}`, you can access it via `bpm.context.usuario` (which will be a dict in Python).
4. **Same as GET Parameters**: POST body properties work exactly like GET query parameters - both are available in `bpm.context`.
5. **Form Data**: If you send `application/x-www-form-urlencoded` data, it's also parsed and available in the context.

**GET vs POST Comparison:**

- **GET with URL parameters**: `GET /api/run/SCRIPT?nombre=Juan&edad=30`
  - Access: `bpm.context.nombre`, `bpm.context.edad`

- **POST with JSON body**: `POST /api/run/SCRIPT` with body `{"nombre": "Juan", "edad": 30}`
  - Access: `bpm.context.nombre`, `bpm.context.edad` (same way!)

## Complete Endpoint Example

```python
#!python3

import redflagbpm

# Connect to BPM
bpm = redflagbpm.BPMService()

# Get request headers
nombre = bpm.context.nombre  # Get name from context
_requestHeaders = bpm.context.json._requestHeaders  # Get request headers

# Create HTML content for response
html = f"""
<html>
<body>
Hola {nombre}<br/>
Se que usas {_requestHeaders['User-Agent']}
</body>
</html>
"""

# Option 1: Print output directly
print(html)

# Option 2: Save output to temporary file
ruta_salida = "/tmp/archivo_de_salida.txt"
with open(ruta_salida, 'w', encoding='utf-8') as file:
    file.write(html)

# Prepare response headers
_responseHeaders = bpm.context.json._responseHeaders
_responseHeaders["status"] = "200"  # Set status 200 (OK)
_responseHeaders["Content-Type"] = "text/html"  # Set output as HTML
_responseHeaders["Content-Encoding"] = "UTF-8"  # Set encoding

# Option 2 (continuation): Indicate response will be a saved resource
_responseHeaders["resource"] = ruta_salida  # Indicate generated file will be the response

# If 'resource' is omitted, output will be what is printed in the script
```

**Code Explanation:**

1. **Request Headers:**  
   - **`_requestHeaders`**: Get request headers using `bpm.context.json._requestHeaders`. For example, the client's `User-Agent` can be extracted to display in the response.
   - **`x-user`**: User ID from authentication token (if token provided)
   - **`x-ws-auth`**: Web Service authentication status ("true" or "false")
   - **`x-method`**: HTTP method used in the request
2. **HTML Content Generation:**  
   - Create an HTML response that greets the user using the name extracted from context (`bpm.context.nombre`) and shows the client's `User-Agent`.
3. **Output Production:**  
   - **Option 1**: Print content directly using `print(html)`.  
   - **Option 2**: Save content to a temporary file and configure the response header to indicate the file is the resource to be returned.
4. **Response Headers:**  
   - **`_responseHeaders`**: Configure response status (`200` for OK), content type (`text/html`), and encoding (`UTF-8`).  
   - If you want to return a file instead of printed output, use the `resource` key to indicate the file path to send as response.

**Example: Accessing User Information from Token:**

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Get request headers
_requestHeaders = bpm.context.json._requestHeaders

# Extract user information (available when token is provided)
user_id = _requestHeaders.get('x-user', 'anonymous')
ws_auth = _requestHeaders.get('x-ws-auth', 'false')
method = _requestHeaders.get('x-method', 'GET')

# Use user information in logic
if ws_auth == 'true':
    print(f"Web Service request from user: {user_id}")
else:
    print(f"Regular request from user: {user_id}")

print(f"HTTP Method: {method}")
```

---

# Agents [agents]
> Use this chapter when you need to create AI assistants, enable RAG with knowledge bases, configure tools and memory, or implement conversational interfaces with streaming responses.

Agents are AI assistants that can help users with various tasks. They can process documents, handle audio, integrate with external services, and use RAG (Retrieval-Augmented Generation) capabilities.

**Related Topics:** Agents use [Markdown Resources Format](#markdown-resources-format) for knowledge bases and can be used in [Prompts](#prompts) for conversational forms.

## YAML Structure

The recommended field order follows the `@JsonPropertyOrder` annotation:

```yaml
name: "Agent Name"              # Display name
id: "PROJECT/AGENT_ID"          # Unique identifier
active: true                    # Whether agent is active/enabled
provider: "openai"              # AI provider: "openai", "Gemini", etc.
model: "gpt-4o-mini"            # Model name (varies by provider)
instructions: |                 # System prompt/instructions
  Your instructions here...
  Define the agent's personality, behavior, and capabilities.
bootScript: "PROJECT/SCRIPT_ID" # Optional script executed at agent startup
tools: []                       # List of available tool IDs
streaming: true                 # Enable streaming responses
memoryEnabled: true             # Enable conversation memory across sessions
structuredOutput: false         # Enable structured output mode
contextResources: []            # List of context resource IDs
mdResources:                    # Markdown resources for knowledge base
  - "resource.md"
ragResources:                    # RAG (Retrieval-Augmented Generation) resources
  - "PROJECT/DOCUMENT_ID"
mcpServers:                     # MCP (Model Context Protocol) servers configuration
  server1:
    name: "Server Name"
    # Additional MCP server configuration
chatStore: "PROJECT/STORE_ID"   # Chat storage identifier
targetUsers: []                 # Specific users allowed (empty = all users)
targetGroups:                   # Groups allowed to access
  - "ADMIN"
alias: "ShortName"              # Optional alias/short name
properties:                     # Additional properties (JSON object)
  documentEnabled: true          # Can process documents?
  audioEnabled: true             # Can process audio?
  supportsAudio: true           # Supports audio input/output?
  telegramBotToken: "token"      # Telegram bot token (optional)
  telegramBotUsername: "bot"    # Telegram bot username (optional)
  telegramEnabledUsers:        # Telegram user IDs allowed
    - 123456789
  embeddingModelName: "text-embedding-3-large"  # Embedding model for RAG
  ragEmbeddingStoreType: inmemory  # RAG embedding store type: "inmemory", "database", etc.
```

## Field Descriptions

- **`name`**: Display name of the agent shown to users.
- **`id`**: Unique identifier in format `PROJECT/AGENT_ID`.
- **`active`**: Boolean flag to enable/disable the agent.
- **`provider`**: AI provider name (e.g., `"openai"`, `"Gemini"`).
- **`model`**: Specific model identifier (e.g., `"gpt-4o-mini"`, `"gemini-2.5-flash"`).
- **`instructions`**: System prompt defining agent behavior, personality, and capabilities.
- **`bootScript`**: Optional script ID executed when the agent starts a conversation.
- **`tools`**: Array of tool IDs the agent can use.
- **`streaming`**: Enable real-time streaming of responses.
- **`memoryEnabled`**: Enable conversation memory across sessions.
- **`structuredOutput`**: Enable structured output mode for consistent formatting.
- **`contextResources`**: List of context resource IDs to include in agent context.
- **`mdResources`**: Markdown files used as knowledge base resources. These files are automatically indexed and made searchable to the agent. See [Markdown Resources Format](#markdown-resources-format) below.
- **`ragResources`**: Resources for RAG (document IDs, collection IDs, etc.).
- **`mcpServers`**: Map of MCP (Model Context Protocol) server configurations.
- **`chatStore`**: Identifier for chat storage/persistence.
- **`targetUsers`**: List of specific user IDs allowed (empty array = all users).
- **`targetGroups`**: List of group names allowed to access the agent.
- **`alias`**: Optional short name or alias for the agent.
- **`properties`**: JSON object with additional configuration (Telegram, RAG settings, etc.).

## Markdown Resources Format

The `mdResources` field references Markdown files that are automatically indexed and made available to the agent as a searchable knowledge base. The agent can retrieve specific sections from these documents using the `getIndexInfo` tool.

**File Structure:**

Each Markdown file should be organized into sections using level 1 headers (`# `). Each section follows this structure:

```markdown
# Section Title [section-id]
> Use this section when you need to understand the section format or when looking for specific information about section titles and structure.

Content of the section goes here.
This can include multiple paragraphs, code blocks, lists, etc.
```

More content can follow...

**Section Format:**

1. **Header**: Must start with `# ` (level 1 header)
   - Optional section ID: `# Title [section-id]` - if omitted, auto-generated as `section-N`
   - The ID is used to retrieve the specific section content

2. **Keywords** (optional): Lines starting with `>` after the header
   - Used for search and indexing
   - Should be a descriptive paragraph explaining when to use the section
   - Empty lines after keywords end the keyword collection
   - If no keywords are provided, the title is used as keywords

3. **Content**: Everything after the keywords (or header if no keywords)
   - Can include any Markdown content
   - Code blocks (``` or ~~~) are preserved but ignored for parsing
   - Multiple paragraphs, lists, tables, etc. are all supported

**Indexing:**

- Each section is indexed with the format: `{agent_id}/{mdResource}#{section-id}`
- Example: If agent is `HLW/BEPEMITO` and file is `manual.md` with section `[getting-started]`, the index is `HLW/BEPEMITO/manual.md#getting-started`
- The agent receives an index of all sections with their titles and keywords
- The agent can retrieve full content of any section using the section ID

**Example Markdown File:**

```markdown
# Getting Started [getting-started]
> Use this section when you need to set up the platform for the first time, understand installation requirements, or follow initial configuration steps.

This section explains how to get started with the platform.
Follow these steps to begin.

# Advanced Features [advanced]
> Use this section when you need to configure advanced settings, enable optional features, or understand complex configuration options beyond basic setup.

Advanced features include:

- Feature 1
- Feature 2

# Troubleshooting [troubleshooting]
> Use this section when you encounter errors, need help resolving problems, or want to understand common issues and their solutions.

Common issues and solutions...
```

**Agent Tool:**

When `mdResources` are configured, the agent automatically receives:
- `getIndexInfo`: Tool to retrieve the full content of a section by its ID
- An index listing all available sections with their IDs, titles, and keywords

The agent uses this information to provide accurate, context-aware responses based on the documentation.

## Complete Example

```yaml
name: Bepemito
id: HLW/BEPEMITO
active: true
provider: OpenAI
model: gpt-4o-mini
instructions: |
  Tu nombre es Bepemito.
  Hablas en español rioplatense semi formal.
  Eres un asistente útil y creativo que ayuda a los usuarios con sus consultas.
streaming: true
memoryEnabled: true
tools: []
mdResources:
  - Manual_BPM.md
  - testBase.md
targetUsers: []
targetGroups:
  - ADMIN
properties:
  telegramBotToken: "1711141100:AAEcPjqZ_XXXJZLRDdofqOA97hvoTkJugsg"
  telegramBotUsername: "gdressino_bot"
  telegramEnabledUsers:
    - 1851620352
  documentEnabled: true
  audioEnabled: true
  supportsAudio: true
  embeddingModelName: "text-embedding-3-large"
  ragEmbeddingStoreType: inmemory
```

---

# Prompts [prompts]
> Use this chapter when you need to create conversational forms that use an Agent to collect data through natural language interactions, execute prompts as scripts using bpm.service.execute(), or use prompts in Flowable Script Tasks. Prompts combine Agents with JSON Schema for dynamic form generation and can be executed programmatically.

Prompts are AI-powered form generators that use an Agent to collect information from users through conversational interactions. They combine an Agent with a JSON Schema to create dynamic, intelligent forms that guide users through data collection.

**Related Topics:** Prompts use [Agents](#agents) for conversation and follow the same JSON Schema structure as [Forms](#forms).

## YAML Structure

The recommended field order follows the `@JsonPropertyOrder` annotation:

```yaml
agent: "PROJECT/AGENT_ID"       # Agent to use for the conversation
prompt: |                       # Initial prompt/instructions for the agent
  Your task is to gather information from the user to complete the task.
  Act as if you've started a conversation with the user and use a friendly and professional tone.
  At the end, communicate the result of the task to the user.
content: []                     # Optional list of content IDs or references
schema:                         # JSON Schema defining the form structure
  type: object
  description: |
    This form collects basic user information,
    including their name, preferred treatment type, and the client they work with.
    Make sure the user feels comfortable and welcome when providing their data.
  properties:
    cliente:
      type: string
      title: Cliente
      description: Choose the client you're working with.
        Use the valueProvider "PROJECT/VP_CLIENTES" to get a list of clients.
    nombre:
      type: string
      title: Nombre
      description: Please provide your full name.
    tratamiento:
      type: string
      title: Tratamiento
      description: How do you prefer to be addressed? Only accepts "Formal" or "Informal".
      enum:
        - Formal
        - Informal
  required:
    - nombre
    - tratamiento
    - cliente
```

## Field Descriptions

- **`agent`**: Required. The agent ID (`PROJECT/AGENT_ID`) to use for the conversation.
- **`prompt`**: Required. Initial instructions/prompt for the agent describing the task and conversation style.
- **`content`**: Optional. List of content IDs or references to include in the prompt context.
- **`schema`**: Required. JSON Schema object defining the form structure, fields, validation rules, and requirements.

## Alternative Format: Script with Shebang

Prompts can also be defined as a script file with a shebang format:

```python
#!PROJECT/AGENT_ID

Your task is to gather information from the user to complete the task.
Act as if you've started a conversation with the user and use a friendly and professional tone.
At the end, communicate the result of the task to the user.
```

The first line `#!PROJECT/AGENT_ID` specifies the agent, and the rest is the prompt text. The schema must be defined separately in the YAML structure or via the `.config` file.

## Configuration

The `.config` file for Prompts:

```yaml
project: "PROJECT_CODE"
id: "PROJECT/PROMPT_ID"
lang: "YAML"                    # or "Python" if using script format
type: "Prompt"
description: "Optional description"
properties: null
form: false
```

## Usage

Prompts are used to create intelligent, conversational forms where an AI agent guides the user through filling out the form. The agent:

1. Uses the `prompt` to understand the task and conversation style
2. References the `schema` to know what information needs to be collected
3. Adapts the conversation flow dynamically based on user responses
4. Validates input against the schema requirements
5. Can use `content` references for additional context

The agent will naturally guide users through the form fields, asking questions, clarifying inputs, and ensuring all required fields are completed before submission.

## Using Prompts as Scripts

Prompts can be executed as scripts in two ways:

### 1. Using `bpm.service.execute()`

Prompts can be executed programmatically from other scripts using `bpm.service.execute()`:

```python
#!python3
import redflagbpm

bpm = redflagbpm.BPMService()

# Execute a prompt as a script
result = bpm.service.execute(
    script="PROJECT/PROMPT_ID",
    context={
        "user_message": "I need to collect customer information",
        # Additional context variables can be passed here
    }
)

# The result contains the structured data collected by the prompt
print(f"Collected data: {result}")
```

**Use Cases:**
- Execute prompts from other scripts or endpoints
- Integrate prompt-based data collection into automated workflows
- Chain multiple prompts together
- Use prompts in custom services or endpoints

### 2. Using Prompts in Flowable Script Tasks

Prompts can be used as Script Tasks in Flowable processes. To use a prompt in a Script Task:

1. **Set the Script field** to the prompt ID (e.g., `PROJECT/PROMPT_ID`)
2. **Set the Script Format** to `"code"` (this indicates the script comes from the BPM platform)

**Example Flowable Script Task Configuration:**

- **Script**: `PROJECT/PROMPT_ID`
- **Script Format**: `code`

When the Script Task executes, the prompt will:
- Use the agent specified in the prompt definition
- Collect data according to the schema
- Return the structured data as a result
- Make the collected data available as process variables

**Accessing Prompt Results in Flowable:**

After a prompt executes in a Script Task, the collected data is available as process variables. The variable names correspond to the schema properties:

```python
# In a subsequent Script Task or expression
# If the prompt collected: nombre, tratamiento, cliente
# These are available as process variables:
nombre = execution.getVariable("nombre")
tratamiento = execution.getVariable("tratamiento")
cliente = execution.getVariable("cliente")
```

**Important Notes:**

- Prompts executed as scripts require the agent to be properly configured and active
- The schema must be defined either in the YAML structure or in the `.config` file's `properties.schema`
- When using prompts in Script Tasks, ensure the agent has access to the necessary tools and resources
- Prompt execution may take longer than regular scripts due to AI processing time

## Complete Examples

**Complete Prompt Definition (YAML format):**

```yaml
agent: PROJECT/BEPEMITO
prompt: |
  Your task is to gather information about the user to complete the task.
  Act as if you've started a conversation with the user and use a friendly and professional tone.
  At the end, communicate the result of the task to the user.
content:                        # Optional: additional content references
  - PROJECT/DOCUMENT_ID
schema:
  type: object
  description: |
    This form collects basic user information,
    including their name, preferred treatment type, and the client they work with.
  properties:
    cliente:
      type: string
      title: Cliente
      description: Choose the client you're working with.
        Use the valueProvider "PROJECT/VP_CLIENTES" to get a list of clients.
    nombre:
      type: string
      title: Nombre
      description: Please provide your full name.
    tratamiento:
      type: string
      title: Tratamiento
      description: How do you prefer to be addressed? Only accepts "Formal" or "Informal".
      enum:
        - Formal
        - Informal
  required:
    - nombre
    - tratamiento
    - cliente
```

**Alternative Format (Script with Shebang):**

```text
#!PROJECT/BEPEMITO

Your task is to gather information about the user to complete the task.
Act as if you've started a conversation with the user and use a friendly and professional tone.
At the end, communicate the result of the task to the user.
```

When using the script format, the schema must be defined separately in the YAML structure or via the `.config` file's `properties.schema`.

---

# Menus [menus]
> Use this chapter when you need to organize application navigation, create menu hierarchies, group artifacts into menus, or configure menu items with icons, permissions, and targets.

Menus organize application navigation by grouping related artifacts (Queries, Services, Reports, Pages) into a hierarchical structure. The menu system combines two approaches: **explicit menu definitions** that create the menu structure, and **automatic menu items** from artifacts that populate those menus.

**Related Topics:** Menus are used by [Queries](#queries), [Services](#services), [Reports](#reports), and [Pages](#pages) to organize navigation. Menu items respect the same access control (`targetUsers`, `targetGroups`) as the artifacts they reference.

## Two Ways to Create Menus

### 1. Explicit Menu Definitions

Create standalone Menu artifacts (type `"Menu"`) that define the menu structure. These menus can contain submenus, separators, and custom menu items.

### 2. Automatic Menu Items from Artifacts

Artifacts (Queries, Services, Reports, Pages) can specify a `menus` array to automatically appear in those menu locations. The system creates menu items for these artifacts and adds them to the corresponding menus.

**How They Work Together:**

1. Explicit menus define the structure (parent menus, submenus, separators)
2. Artifacts specify which menus they belong to using the `menus` array
3. The system automatically populates the menus with the artifacts

## Explicit Menu Structure

Create a Menu artifact file (e.g., `main_menu.yaml`) with type `"Menu"` in its `.config` file:

```yaml
id: "MAIN_MENU"                  # Unique identifier for this menu
name: "Main Menu"                # Display name
icon: "HOME"                     # Icon name (Vaadin icons)
parentId: null                   # Parent menu ID (null for root-level menus)
target: null                     # Optional: URL or route (null for container menus)
separator: false                 # Add separator line before this item
sort: true                       # Automatically sort child items alphabetically
theme: null                      # Optional: CSS theme/class for styling
users: []                        # Optional: Specific users allowed (empty = all)
groups:                         # Optional: Groups allowed to access
  - "ADMIN"
items:                          # Child menu items (submenus or leaf items)
  - id: "ORDERS_SUBMENU"
    name: "Orders"
    icon: "SHOPPING_CART"
    parentId: "MAIN_MENU"
    sort: true
    items: []                    # Empty items array - will be populated by artifacts
  - id: "TOOLS_SUBMENU"
    name: "Tools"
    icon: "WRENCH"
    parentId: "MAIN_MENU"
    items: []
  - separator: true              # Add separator line
  - id: "ADMIN_MENU"
    name: "Administration"
    icon: "COGS"
    parentId: "MAIN_MENU"
    groups:
      - "ADMIN"
    items: []
```

**Menu Fields:**

- **`id`**: Unique identifier for the menu. Used by `parentId` to create hierarchy and by artifacts' `menus` array to reference this menu.
- **`parentId`**: ID of the parent menu. Use `null` for root-level menus. Creates submenus when set.
- **`name`**: Display name shown in the navigation menu.
- **`icon`**: Icon name (Vaadin icons or custom icon names).
- **`target`**: Optional URL or route. If set, clicking the menu item navigates to this target. Leave `null` for container menus that only contain submenus.
  
  **Target Formats:**
  - **Application routes**: Use route names like `"Query"`, `"Service"`, `"Queries"` (see [Application Routes](#routes) for complete list)
  - **Routes with parameters**: `"Query?__query__=PROJECT/QRY_ID"` or `"Service?__service__=PROJECT/SRV_ID&param1=value1"`
  - **REST API routes**: Use full paths like `"/api/test"` or `"/api/run/PROJECT/SCRIPT_ID"`
  - **External URLs**: Use full URLs like `"https://docs.example.com"`
  
  When artifacts specify `menus` array, their `target` is automatically generated based on artifact type (see [Automatic Menu Items from Artifacts](#automatic-menu-items-from-artifacts) below).
- **`separator`**: Boolean. If `true`, adds a separator line before this menu item.
- **`sort`**: Boolean. If `true`, automatically sorts child items alphabetically by name.
- **`theme`**: Optional CSS class or theme name for custom styling.
- **`users`**: Optional list of specific user IDs allowed to see this menu. Empty array means all users (subject to group restrictions).
- **`groups`**: Optional list of group names. Only users in these groups can see this menu.
- **`items`**: Array of child menu items. Can contain submenus or leaf items. If empty, artifacts with matching `menus` array values will be automatically added here.

**Menu Configuration File:**

```yaml
# main_menu.yaml.config
project: "MYPROJ"
id: "MYPROJ/MAIN_MENU"
lang: "YAML"
type: "Menu"
description: "Main application menu"
```

## Automatic Menu Items from Artifacts

Artifacts can automatically appear in menus by specifying a `menus` array. The values in this array must match the `id` of existing Menu artifacts.

**Queries:**

```yaml
name: "HLW/QRY_ORDERS"
caption: "Orders List"
icon: "LIST"
menus:                          # Appears in these menus
  - "ORDERS_SUBMENU"            # Must match a Menu's id
  - "SALES_MENU"
targetGroups:
  - "SALES"
  - "ADMIN"
```

**Services:**

```yaml
name: "HLW/SRV_SEARCH_ORDERS"
caption: "Search Orders"
icon: "SEARCH"
menus:
  - "ORDERS_SUBMENU"
  - "TOOLS_SUBMENU"
targetGroups:
  - "SALES"
```

**Reports:**

```yaml
# In report .config file
properties:
  menus:
    - "ORDERS_SUBMENU"
    - "REPORTS_MENU"
  targetGroups:
    - "ADMIN"
```

**Pages:**

```yaml
# In page .config file
properties:
  caption: "Dashboard"
  icon: "DASHBOARD"
  menus:
    - "MAIN_MENU"
  targetGroups:
    - "ALL_USERS"
```

**How It Works:**

1. When an artifact specifies `menus: ["ORDERS_SUBMENU"]`, the system looks for a Menu with `id: "ORDERS_SUBMENU"`
2. If found, the artifact is automatically added as a menu item under that menu
3. The menu item uses the artifact's `caption` (or `name`), `icon`, and respects its `targetUsers`/`targetGroups`
4. The menu item's target is automatically set based on artifact type:
   - Queries: `Query/q?__query__=ARTIFACT_ID`
   - Services: `Service/s?__service__=ARTIFACT_ID`
   - Reports: `Service/s?__service__=REPORT_ID`
   - Pages: `Page/p?__page__=ARTIFACT_ID`

## Creating Menu Hierarchies

Use `parentId` to create nested menu structures:

```yaml
# Root menu
id: "SALES_MENU"
name: "Sales"
icon: "CHART"
parentId: null
items: []

# Submenu under Sales
id: "ORDERS_SUBMENU"
name: "Orders"
icon: "SHOPPING_CART"
parentId: "SALES_MENU"          # References parent
items: []

# Another submenu under Sales
id: "CUSTOMERS_SUBMENU"
name: "Customers"
icon: "USERS"
parentId: "SALES_MENU"          # Same parent
items: []
```

**Resulting Structure:**

```
Sales
  ├─ Orders          (submenu)
  └─ Customers       (submenu)
```

## Using Separators

Add visual separators between menu items:

```yaml
items:
  - id: "ITEM_1"
    name: "First Item"
    items: []
  - separator: true              # Separator line
  - id: "ITEM_2"
    name: "Second Item"
    items: []
```

## Automatic Sorting

Enable automatic alphabetical sorting of child items:

```yaml
id: "TOOLS_MENU"
name: "Tools"
sort: true                       # Children will be sorted alphabetically
items: []
```

When `sort: true`, all child items (both explicit items and artifacts added via `menus` array) are automatically sorted by name.

## Access Control

Menus respect access control at multiple levels:

**Menu-Level Access:**

```yaml
id: "ADMIN_MENU"
name: "Administration"
groups:
  - "ADMIN"                      # Only ADMIN group can see this menu
users: []                        # Empty = all users in groups
items: []
```

**Artifact-Level Access:**

```yaml
# Query with access control
name: "HLW/QRY_SENSITIVE_DATA"
menus:
  - "ADMIN_MENU"
targetGroups:
  - "ADMIN"                      # Only ADMIN group can access
targetUsers: []                  # Empty = all users in groups
```

**Combined Access:**

- A user must have access to **both** the menu AND the artifact to see the menu item
- If a user doesn't have access to the menu, they won't see it or any of its items
- If a user has access to the menu but not the artifact, the artifact won't appear in the menu

## Complete Examples

**Example 1: Simple Menu Structure**

```yaml
# main_menu.yaml
id: "MAIN_MENU"
name: "Main Menu"
icon: "HOME"
items:
  - id: "ORDERS"
    name: "Orders"
    icon: "SHOPPING_CART"
    items: []
  - id: "CUSTOMERS"
    name: "Customers"
    icon: "USERS"
    items: []
```

```yaml
# Query that appears in Orders menu
name: "HLW/QRY_ORDERS"
caption: "Order List"
menus:
  - "ORDERS"                     # Matches menu id
```

**Example 2: Hierarchical Menu with Separators**

```yaml
# sales_menu.yaml
id: "SALES_MENU"
name: "Sales"
icon: "CHART"
items:
  - id: "ORDERS_SUBMENU"
    name: "Orders"
    icon: "SHOPPING_CART"
    sort: true                   # Auto-sort children
    items: []
  - separator: true              # Visual separator
  - id: "CUSTOMERS_SUBMENU"
    name: "Customers"
    icon: "USERS"
    items: []
  - separator: true
  - id: "REPORTS_SUBMENU"
    name: "Reports"
    icon: "FILE_TEXT"
    groups:
      - "MANAGER"                # Only managers see Reports submenu
    items: []
```

**Example 3: Menu with Custom Target**

```yaml
# external_link_menu.yaml
id: "EXTERNAL_LINKS"
name: "External Links"
icon: "EXTERNAL_LINK"
items:
  - id: "DOCS"
    name: "Documentation"
    icon: "BOOK"
    target: "https://docs.example.com"  # External URL
    items: []
  - id: "SUPPORT"
    name: "Support Portal"
    icon: "QUESTION_CIRCLE"
    target: "https://support.example.com"
    items: []
```

**Example 4: Multiple Artifacts in Same Menu**

```yaml
# Query 1
name: "HLW/QRY_ORDERS"
caption: "All Orders"
menus:
  - "ORDERS_SUBMENU"

# Query 2
name: "HLW/QRY_PENDING_ORDERS"
caption: "Pending Orders"
menus:
  - "ORDERS_SUBMENU"

# Service
name: "HLW/SRV_CREATE_ORDER"
caption: "Create Order"
menus:
  - "ORDERS_SUBMENU"
```

All three artifacts will appear under the "ORDERS_SUBMENU" menu, sorted if the menu has `sort: true`.

## Best Practices

1. **Use Descriptive IDs**: Menu IDs should be clear and unique (e.g., `"ORDERS_SUBMENU"` not `"MENU1"`)

2. **Organize by Function**: Group related artifacts under the same menu (e.g., all order-related queries and services under "Orders")

3. **Use Hierarchies Sparingly**: Deep nesting (more than 2-3 levels) can make navigation difficult

4. **Leverage Automatic Sorting**: Set `sort: true` on menus that contain many items to keep them organized

5. **Consistent Access Control**: Use menu-level `groups` for broad access control, and artifact-level `targetGroups` for fine-grained control

6. **Separators for Clarity**: Use separators to visually group related menu items

7. **Empty Items Arrays**: For menus that will be populated by artifacts, use `items: []` - the system will automatically add artifacts that reference this menu's `id`

---

# Application Routes [routes]
> Use this chapter when you need to find available application routes or REST API endpoints to use as menu targets, or understand URL patterns for navigation.

Application routes define the URLs used to navigate within the BPM application. These routes can be used as `target` values in menu definitions to create navigation links. Routes are organized into two categories: **Application Views** (Vaadin Flow routes) and **REST API Endpoints**.

**Related Topics:** Routes are used as `target` values in [Menus](#menus) to create navigation links. Routes can also be used in [Scripts](#scripts) response types like `OPEN` and `OPEN_NEW`.

## Application View Routes

Application view routes are defined using Vaadin Flow's `@Route` annotation. These routes render full-page views within the application layout.

### Artifact View Routes

These routes display specific artifacts (Queries, Services, Pages, Agents):

| Route | Description | Parameters | Example |
|-------|-------------|------------|---------|
| `Query` | Display a Query | `__query__`: Query ID (required) | `Query?__query__=PROJECT/QRY_ID` |
| `QueryW` | Display a Query (window layout) | `__query__`: Query ID (required) | `QueryW?__query__=PROJECT/QRY_ID` |
| `Service` | Display a Service | `__service__`: Service ID (required) | `Service?__service__=PROJECT/SRV_ID` |
| `ServiceW` | Display a Service (window layout) | `__service__`: Service ID (required) | `ServiceW?__service__=PROJECT/SRV_ID` |
| `Page` | Display a Page | `__page__`: Page ID (required) | `Page?__page__=PROJECT/PAGE_ID` |
| `PageW` | Display a Page (window layout) | `__page__`: Page ID (required) | `PageW?__page__=PROJECT/PAGE_ID` |
| `Agent` | Display an Agent chat interface | Agent ID as path parameter | `Agent/PROJECT/AGENT_ID` |

**Query Parameters:**

All artifact routes accept additional query parameters that are passed to the artifact's context:

```
Query?__query__=PROJECT/QRY_ID&param1=value1&param2=value2
```

### List View Routes

These routes display lists of artifacts:

| Route | Description | Parameters | Example |
|-------|-------------|------------|---------|
| `Queries` | List all available Queries | None | `Queries` |
| `Services` | List all available Services | None | `Services` |
| `Pages` | List all available Pages | None | `Pages` |
| `Reports` | List all available Reports | None | `Reports` |
| `Agents` | List all available Agents | None | `Agents` |
| `VectorStores` | List all Vector Stores | None | `VectorStores` |

### Process Management Routes

Routes for BPMN process management:

| Route | Description | Parameters | Example |
|-------|-------------|------------|---------|
| `ListTasks` | List user's tasks | App ID as path parameter | `ListTasks/APP_ID` |
| `ListProcesses` | List process instances | App ID as path parameter | `ListProcesses/APP_ID` |
| `ListProcessTasks` | List tasks for a process | App ID as path parameter | `ListProcessTasks/APP_ID` |
| `StartProcess` | Start a new process instance | App ID as path parameter | `StartProcess/APP_ID` |
| `StartProcessByName` | Start process by name | Process name as query parameter | `StartProcessByName?processName=PROCESS_NAME` |
| `StartTask` | Start a task by ID | Task ID as query parameter | `StartTask?taskId=TASK_ID` |
| `StartNextProcessTask` | Start next task in process | Process instance ID as query parameter | `StartNextProcessTask?processInstanceId=INSTANCE_ID` |
| `AdminTasks` | Admin task management | App ID as path parameter | `AdminTasks/APP_ID` |
| `AdminProcesses` | Admin process management | App ID as path parameter | `AdminProcesses/APP_ID` |
| `TestProcess` | Test process execution | App ID as path parameter | `TestProcess/APP_ID` |

### Configuration Routes

Administrative configuration views:

| Route | Description | Access |
|-------|-------------|--------|
| `ConfigCode` | Code repository management | ADMIN |
| `ConfigQueries` | Query management | ADMIN |
| `ConfigServices` | Service management | ADMIN |
| `ConfigReports` | Report management | ADMIN |
| `ConfigCollections` | Collection management | ADMIN |
| `ConfigProjects` | Project management | ADMIN |
| `ConfigApps` | Application management | ADMIN |
| `ConfigDatasets` | Dataset management | ADMIN |
| `ConfigDatasources` | Data source management | ADMIN |
| `ConfigMailsources` | Mail source management | ADMIN |
| `ConfigParameters` | Parameter management | ADMIN |
| `ConfigRepos` | Repository management | ADMIN |
| `ConfigVenvs` | Virtual environment management | ADMIN |
| `ConfigHandlers` | Handler management | ADMIN |
| `ConfigAssistants` | Assistant/Agent management | ADMIN |
| `LogView` | Log viewer | ADMIN |
| `LogConfig` | Log configuration | ADMIN |

### Other Routes

| Route | Description | Parameters |
|-------|-------------|------------|
| `kanban-example` | Kanban board example view | None |

## REST API Routes

REST API routes are defined using Spring's `@RequestMapping` annotations. These routes are prefixed with `/api` and are used for programmatic access and integrations.

### Script Execution

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/run/{project}/{code}` | GET/POST | Execute a script | Path: `project`, `code`; Query/Body: script parameters |

**Example:**

```
GET /api/run/MYPROJ/MY_SCRIPT?param1=value1&param2=value2
POST /api/run/MYPROJ/MY_SCRIPT
Content-Type: application/json
{"param1": "value1", "param2": "value2"}
```

### Authentication

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/authToken` | GET | Get authentication token | `user`, `password` |
| `/api/payloadToken` | POST | Sign data with token | `token` (query), JSON body |

### Service Proxy

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/service/**` | Any | Proxy to external service | Service ID in path |

### Assistant/Agent API

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/assistant/**` | Any | Agent/Assistant API | Agent ID in path |

### Resource Access

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/resource` | GET | Get resource file | `path` (query) |
| `/api/mailResource` | GET | Get mail resource | `path`, `cid` (query) |

### Task Management

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/taskComplete` | POST | Complete a task | `token`, `taskId` (query), JSON body |

### OAuth2 & SSO

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/oauth2callback` | GET | OAuth2 callback | OAuth2 parameters |
| `/api/sso/redirect/{channel}/{chatId}` | GET | SSO redirect | Path: `channel`, `chatId` |
| `/api/sso/start/{channel}/{chatId}` | GET | Start SSO flow | Path: `channel`, `chatId` |

### WhatsApp Integration

| Route | Method | Description | Parameters |
|-------|--------|-------------|------------|
| `/api/whatsapp/**/webhook` | POST | WhatsApp webhook | Agent ID in path |

### Utility

| Route | Method | Description |
|-------|--------|-------------|
| `/api/test` | GET | Test endpoint |
| `/api/put` | PUT | File upload |

## Using Routes in Menus

Routes can be used as `target` values in menu definitions. The format depends on the route type:

### Application View Routes

Use the route name directly, with query parameters if needed:

```yaml
id: "QUERIES_MENU"
name: "Queries"
target: "Queries"                    # Simple route
items: []

id: "ORDERS_QUERY"
name: "Orders"
target: "Query?__query__=HLW/QRY_ORDERS"  # Route with query parameter
items: []
```

### REST API Routes

For REST API routes, use the full path:

```yaml
id: "API_DOCS"
name: "API Documentation"
target: "/api/test"                  # Full path for REST routes
items: []
```

### External URLs

You can also use external URLs:

```yaml
id: "EXTERNAL_DOCS"
name: "Documentation"
target: "https://docs.example.com"   # External URL
items: []
```

## Route Parameters

### Query Parameters

Query parameters are appended to the route using `?` and `&`:

```
Query?__query__=PROJECT/QRY_ID&filter1=value1&filter2=value2
```

### Path Parameters

Some routes use path parameters (e.g., `ListTasks/APP_ID`):

```
ListTasks/myapp
```

### Special Parameters for Artifacts

When opening artifacts via routes, use these special parameters:

- **Queries**: `__query__=PROJECT/QUERY_ID`
- **Services**: `__service__=PROJECT/SERVICE_ID`
- **Pages**: `__page__=PROJECT/PAGE_ID`

Additional parameters are passed to the artifact's context:

```yaml
target: "Query?__query__=HLW/ORDERS&customer_id=12345&status=active"
```

## Examples

### Menu with Query Route

```yaml
id: "ORDERS_MENU"
name: "Orders"
icon: "SHOPPING_CART"
items:
  - id: "ALL_ORDERS"
    name: "All Orders"
    target: "Query?__query__=HLW/QRY_ALL_ORDERS"
    items: []
  - id: "PENDING_ORDERS"
    name: "Pending Orders"
    target: "Query?__query__=HLW/QRY_PENDING_ORDERS&status=pending"
    items: []
```

### Menu with Service Route

```yaml
id: "TOOLS_MENU"
name: "Tools"
items:
  - id: "CREATE_ORDER"
    name: "Create Order"
    target: "Service?__service__=HLW/SRV_CREATE_ORDER"
    items: []
```

### Menu with List View Route

```yaml
id: "MAIN_MENU"
name: "Main"
items:
  - id: "VIEW_QUERIES"
    name: "All Queries"
    target: "Queries"
    items: []
  - id: "VIEW_SERVICES"
    name: "All Services"
    target: "Services"
    items: []
```

### Menu with Process Route

```yaml
id: "BPM_MENU"
name: "Processes"
items:
  - id: "MY_TASKS"
    name: "My Tasks"
    target: "ListTasks/default"
    items: []
  - id: "START_PROCESS"
    name: "Start Process"
    target: "StartProcess/myapp"
    items: []
```

### Menu with REST API Route

```yaml
id: "API_MENU"
name: "API"
items:
  - id: "API_TEST"
    name: "Test API"
    target: "/api/test"
    items: []
```

---

# Configuration Files [configuration-files]
> Use this chapter when you need to create or modify .config files for artifacts, configure artifact metadata, set language types, define properties, or understand configuration file structure and naming conventions.

Every artifact (script, query, service, etc.) stored in the repository must have a corresponding `.config` file. This file is a YAML document (preferred) or JSON that defines metadata for the artifact.

## File Location and Naming

**Naming Convention:**

If the artifact file is named `myscript.py`, the config file must be named `myscript.py.config`.

**⚠️ IMPORTANT - File Location:**

The `.config` file **MUST be located in exactly the same directory** as the artifact file. The system looks for the config file by appending `.config` to the artifact file's path, so they must be in the same location.

**Example:**

```
repos/
  └── mi_proyecto/
      └── scripts/
          ├── mi_script.py          ← Artifact file
          └── mi_script.py.config   ← Config file (same directory)
```

**Incorrect locations (will NOT work):**

```
repos/
  └── mi_proyecto/
      ├── scripts/
      │   └── mi_script.py
      └── configs/                 ← WRONG: Different directory
          └── mi_script.py.config
```

## YAML Structure

```yaml
project: "PROJECT_CODE"        # Project identifier (e.g., "MYPROJ")
id: "PROJECT_CODE/ARTIFACT_ID" # Unique ID (e.g., "MYPROJ/MYSCRIPT")
lang: "Python"                 # Language: Python (Jython), JavaScript, YAML, JSON, Shell (shebang), Velocity, JUEL, Vertx, Prompt, Markdown, Binary
type: "Script"                 # Type: Script, Query, Service, Page, Report, Agent, Collection, Form, Menu, Public endpoint, User endpoint, Web service endpoint
description: "Optional description"
cron: null                     # Optional cron expression for scheduling
properties:                    # Additional properties (varies by type)
  caption: "My Script Action"  # For scripts used as actions
  selectionScope: 1            # For scripts used as actions
env:                           # Environment variables
  MY_VAR: "value"
form: false                    # true if the artifact has an associated form
```

## Common Fields

- **`project`**: The project code the artifact belongs to.
- **`id`**: The unique identifier used to reference the artifact in the system.
- **`type`**: The type of artifact. Valid values: `Script`, `Query`, `Service`, `Page`, `Report`, `Agent`, `Collection`, `Form`, `Menu`, `Public endpoint`, `User endpoint`, `Web service endpoint`.
- **`lang`**: The language of the file. Valid values: `Python`, `JavaScript`, `YAML`, `JSON`, `Shell`, `Velocity`, `JUEL`, `Vertx`, `Prompt`, `Markdown`, `Binary`.
- **`properties`**: Used to define specific behaviors, especially for scripts acting as actions in queries (defining `caption`, `icon`, `selectionScope`, etc.).

## Language Types (`lang` field)

The `lang` field specifies the programming language or format of the artifact file. Valid values are:

- **`Python`**: Python scripts (`.py` files)
  - **Interpreter**: Uses **Jython** (Java-based Python interpreter)
  - The code is executed using the Jython script engine
  - Example: `lang: "Python"`

- **`JavaScript`**: JavaScript files (`.js` files)
  - Executed using Nashorn or the default JavaScript engine
  - Example: `lang: "JavaScript"`

- **`Shell`**: Shell scripts (`.sh` files) or scripts with shebang
  - **Important**: When `lang: "Shell"` is used, the system processes the **shebang** (`#!`) line at the beginning of the file
  - The shebang determines the actual interpreter to use (e.g., `#!/bin/bash`, `#!/usr/bin/python3`, `#!/usr/bin/env node`)
  - If the file starts with `#!` but doesn't match any known language pattern, it's treated as Shell
  - Example: `lang: "Shell"` for a file starting with `#!/bin/bash`

- **`YAML`**: YAML configuration files (`.yaml` or `.yml` files)
  - Parsed as YAML and converted to JSON
  - Example: `lang: "YAML"`

- **`JSON`**: JSON configuration files (`.json` files)
  - Parsed as JSON
  - Example: `lang: "JSON"`

- **`Velocity`**: Velocity templates (`.vtl`, `.vhtml`, `.html`, `.htm` files)
  - Uses Apache Velocity template engine
  - Example: `lang: "Velocity"`

- **`JUEL`**: Java Unified Expression Language
  - Used for expression evaluation
  - Example: `lang: "JUEL"`

- **`Vertx`**: Vert.x scripts
  - Uses Vert.x script engine
  - Example: `lang: "Vertx"`

- **`Prompt`**: Prompt files (`.prompt` files)
  - Processed as Shell scripts
  - Example: `lang: "Prompt"`

- **`Markdown`**: Markdown files (`.md` files)
  - Rendered as HTML
  - Example: `lang: "Markdown"`

- **`Binary`**: Binary files (e.g., `.pdf`, `.docx`, `.odt`, `.jasper`, `.jrxml`, `.zip`, images)
  - Stored as binary data, not executed
  - Example: `lang: "Binary"`

**Important Notes:**

1. **Python vs Shell**: 
   - When `lang: "Python"` is specified, the code is executed using **Jython** (Java-based Python interpreter)
   - When `lang: "Shell"` is specified, the system reads the **shebang** (`#!`) from the file to determine the actual interpreter
   - For Python3 scripts that need to use the system Python interpreter, use `lang: "Shell"` with a shebang like `#!python3`

2. **Shebang Processing** (for `lang: "Shell"`):
   - The system reads the first line of the file if it starts with `#!`
   - Common shebangs:
     - `#!/bin/bash` - Bash shell
     - `#!/usr/bin/python3` - System Python 3
     - `#!/path/to/venv/bin/python3` - Venv Python 3
     - `#!/usr/bin/env node` - Node.js
     - Any other `#!` line is passed to the shell engine

3. **Default Language**:
   - If no `lang` is specified and the file doesn't have a shebang, JavaScript is used as default

## Type-Specific Properties Examples

**Scripts (as Actions):**

```yaml
properties:
  caption: "Action Name"
  icon: "EDIT"                  # Icon name (Vaadin icons)
  type: "call"                  # Action type: "create", "update", "delete", "query", "display", "open", "call"
  selectionScope: 1              # 0: none, 1: single, -1: any, -2: multiple, -3: group, -4: filter
  displayHints:                 # Array of display hints
    - "default"                 # Default action (double-click)
    - "main"                    # Main button in toolbar
    - "confirm"                 # Require confirmation
    - "small"                   # Display size (for type: "display"): "small", "medium", "large", "x-large"
    - "refresh-on-close"        # Refresh query on close
  reusable: false               # Keep open after execution
  asynchronous: false           # Execute asynchronously
  groupName: "group1"           # Group name for organizing actions
  shortcuts:                    # Keyboard shortcuts
    ok: "Enter"
    cancel: "Escape"
  name: "SCRIPT_NAME"           # Optional script name (legacy)
  inputSchema:                  # Optional form schema
    type: object
    properties: {...}
```

**Note:** For complete documentation of action properties, see the [Actions section](#actions) in the Queries chapter.

**Reports:**

```yaml
properties:
  resources:                    # Resource files used by report
    - "image.jpg"
    - "logo.png"
  targetGroups:                 # Access control
    - "ADMIN"
```

**Pages:**

```yaml
properties:
  title: "Page Title"           # Page title displayed in browser
```

**Queries:**

```yaml
# Usually no properties needed, configuration is in the YAML file itself
properties: null
```

## Complete Examples

**Script .config:**

```yaml
project: "HLW"
id: "HLW/MY_SCRIPT"
lang: "Python"
type: "Script"
description: "My script description"
properties:
  caption: "Execute Action"
  selectionScope: 1
env:
  API_KEY: "secret123"
form: false
```

**Query .config:**

```yaml
project: "HLW"
id: "HLW/QRY_ORDERS"
lang: "YAML"
type: "Query"
description: "Orders query"
properties: null
form: false
```

**Service .config:**

```yaml
project: "HLW"
id: "HLW/SRV_SEARCH"
lang: "YAML"
type: "Service"
description: "Search service"
properties: null
form: false
```

**Report .config:**

```yaml
project: "HLW"
id: "HLW/RPT_SALES"
lang: "Binary"
type: "Report"
description: "Sales report"
properties:
  resources:
    - "logo.png"
    - "header.jpg"
  targetGroups:
    - "ADMIN"
    - "SALES"
form: false
```

**Page .config:**

```yaml
project: "HLW"
id: "HLW/PAGE_DASHBOARD"
lang: "Velocity"
type: "Page"
description: "Dashboard page"
properties:
  title: "Dashboard"
form: false
```

**Endpoint .config:**

```yaml
project: "HLW"
id: "HLW/ENDPOINT_API"
lang: "Python"
type: "Public endpoint"  # or "User endpoint" or "Web service endpoint"
description: "API endpoint"
properties: null
form: false
```

---

