[Remote] Generative AI Data Analyst - USA (Remote)
Note: The job is a remote job and is open to candidates in USA. Welo Data is seeking a Generative AI Analyst to support a high-impact machine learning project. This role involves creating high-quality prompts and responses, leading labeling initiatives, and training teams on best practices for LLM development.
Responsibilities
- Creatively writing prompts and responses to a variety of diverse topics
- Perform LLM annotation and evaluation tasks (ranking, scoring, labeling, tagging)
- Evaluate model outputs for accuracy, relevance, and instruction-following
- Identify and document issues such as hallucinations and inconsistencies
- Participating in and/or supporting labeling workflows, including hands-on annotation and collaboration with internal or external teams
- Training teams on best practices for creating Large Language Models/Data sets
Skills
- Hands-on experience performing data annotation or evaluation tasks (e.g., labeling, ranking, scoring, or tagging LLM outputs)
- Native or near-native English with excellent writing skills
- Strong attention to detail and ability to follow guidelines consistently
- Self-driven, motivated and enthusiastic to work on state-of-art machine learning tools
- 4 year Accredited College degree or equivalent experience
- Must have valid work authorization in the US (Welo Data does not sponsor VISAs at this time)
- College Degree or experience in Linguistics, English Literature, Creative Writing, Journalism, and domain knowledge (Law/Medical/Math/Coding/etc.)
- Experience working in annotation platforms or structured labeling environments is a plus
- Deep understanding of Large Language Models/RLHF
- Experience in labeling/tagging of frames/tasks/prompts to prepare for DNNQA/testing experience
Benefits
- Full-time W-2 employee with benefits
Company Overview