Overview
Annotators
Instances
Questions
Behavioral
Crowdsourcing
{% if mace_enabled %}
MACE
{% endif %}
{% if embedding_viz_enabled %}
Embeddings
{% endif %}
Configuration
System Information
| User ID | Phase | Annotations | Working Time | Avg Time/Annotation | Speed (per hour) | Completion % | Last Activity |
|---|
| Instance ID | Text Preview | Annotations | Completion % | Most Frequent Label | Disagreement | Avg Time | Annotators | Num AI Used |
|---|
AI Assistance Usage
Quality Indicators
| User ID | Instances | Avg Time (s) | Interactions | Changes | AI Requests | AI Accept Rate | Suspicion |
|---|
Loading...
INSTANCE PREVIEW
Hover over a point to see details
SELECTION QUEUE
Use lasso or box selection to select points
Selected items will be prioritized for annotation
CONTROLS
Loading MACE data...
Annotator Competence
MACE estimates each annotator's reliability (0 = random guessing, 1 = always correct).
| Annotator | Competence | Reliability |
|---|---|---|
| Click "Run MACE" to compute scores | ||
Predicted Labels
MACE's best estimate of the true label for each item, weighted by annotator competence.
Run MACE to see predictions.