• Leena Vu
  • AI Machine Learning Operations (MLOps)
Urgent

AI Machine Learning Operations (MLOps)

Consultant Engineering Data Analysis AI/Artificial Intelligence AI/ML Machine Learning

Icon salary 年収
交渉可能
Icon Location Location
Ho Chi Minh

Benefits

13ヶ月目の給与 13ヶ月目の給与
仕事用のノートパソコン/デスクトップ 仕事用のノートパソコン/デスクトップ
その他の福利厚生 その他の福利厚生
在宅勤務 在宅勤務

Job Overview And Responsibility

The Opportunity This role owns end-to-end reliability for AI agents in production. Our agents power real user decisions and must be correct, safe, and trustworthy every day. As a startup, we don’t separate QA, evals, safety, and reliability into different teams—you are the single owner of agent quality in the real world. You’ll define what “good” means, build the evaluation and monitoring systems that enforce it, and close the loop from production failures back into fixes. This is a hands-on, high-impact role for engineers who want to focus deeply on agent correctness, trust, and reliability at scale. Core Responsibilities - Own agent correctness and quality in production, from definition of success to enforcement and continuous improvement - Build and maintain automated evaluation systems (golden datasets, regression tests, LLM-as-judge, rule-based checks) - Design and operate production monitoring and observability for agent quality, failures, drift, and regressions - Define and enforce guardrails and safety boundaries across prompts, tools, permissions, and escalation paths - Run tight feedback loops: turn real failures into tests, fixes, and lasting reliability improvements

Required Skills and Experience

- 4+ years of experience in AI engineering, data analytics engineering, ML systems, or reliability-focused roles (SRE, MLOps, ML QA) - Strong instincts for quality, failure modes, metrics, and regression testing in probabilistic systems - Hands-on experience with Python for evaluation pipelines, automation, and data analysis - Experience working close to production systems, including logs, metrics, dashboards, and incident response - Ability to translate real-world workflows into clear success criteria, KPIs, and test cases - Strong communication skills—you can explain quality tradeoffs to engineers, product, and leadership

Why Candidate should apply this position

- Build from scratch: Be part of creating the customer insights function for a major ecommerce platform - Real impact: Your analyses will directly inform strategy and drive business decisions - Learn and grow: Work with experienced entrepreneurs, consultants from Bain, data engineers, and leadership - Modern tools: Access to AI tools and data infrastructure to do your best work - International exposure: Opportunity for US office rotations or visits - Competitive package - Hybrid working model - Macbook Pro provided for work - Bonus: 13th month salary - Weekly learning sessions and offsites - Close-knit small team culture

Preferred skills and experiences

- Experience building or evaluating LLM-powered or agentic systems (prompting, RAG, tool calling, memory) - Familiarity with LLM evaluation techniques, including offline evals, trajectory analysis, and LLM-as-judge - Experience with agent observability: tracing, sampling, confidence signals, or grounding - Background in data systems or analytics platforms, where correctness and trust are critical - SRE or MLOps experience: production monitoring, incident response, on-call rotation - Experience with A/B testing, experimentation frameworks, or statistical evaluation methods - Comfortable working in ambiguity and iterating toward clarity using data and examples

Report to

CDO

Interview process

Overview interview -> Technical interview, 90mins (at office) -> Fit interview

Leena Vu

Headhunter | Recruiter
Verified
employee 945 件の履歴書
cup 249 件の面接
health 38 件のオファー

Apply for this job

Successfully!

Thank you, you have sent the information successfully.

← View more Leena Vu's jobs
upload Click or drag file to this area to upload PDF only (3MB), You can update only 1 CV

Leena Vu

Headhunter | Recruiter
Verified
Icon employee 945 件の履歴書
Icon cup 249 件の面接
Icon health 38 件のオファー
Chat
ご成約済みの案件 (38)
  • Check Sales Logistics への配置
  • Check Social Marketing Leader への配置
  • Check Fullstack Software Engineer への配置
  • Check [HCM] Business Development Manager への配置
  • Check OutSystems Developer への配置
  • Check Senior Software Developer (remote) への配置
  • Check Software Engineer (JavaScript) への配置
  • Check Software Technical Architect への配置
  • Check Senior SRE/ DevOps への配置
  • Check Senior Front-end Engineer (Full-time remote) への配置
  • Check Procure-to-Pay (PTP) Team Lead への配置
  • Check Finance Executive (Financial Reporting) への配置
  • Check Finance Executive (Financial Reporting) への配置
  • Check Senior Software Engineering への配置
  • Check Finance Executive (Consolidation) - role 2 への配置
  • Check Senior Software Engineer (Full-time Remote) への配置
  • Check Frontend Engineer (Full-time remote) への配置
  • Check Senior Back-End Engineer (Full-time remote) への配置
  • Check #Web3 Developer への配置
  • Check Senior Fullstack Engineer (update reward) への配置
  • Check Senior Ruby on Rails Developer への配置
  • Check Frontend Engineer (Full-time Remote) への配置
  • Check Senior Software Engineer (Full-time Remote) への配置
  • Check Senior Software Engineer (Full-time Remote) への配置
  • Check #Senior Blockchain Developer (Solidity/Rust) への配置
  • Check [HN] Senior Frontend (Full-time Remote) への配置
  • Check Senior Software Engineer (Full-time Remote) への配置
  • Check Senior Software Engineer (Full-time Remote) への配置
  • Check [HN] Senior PHP Laravel Backend (Full-time Remote) への配置
  • Check [HCM] Senior Frontend (Full-time Remote) への配置
  • Check Junior Software Engineer への配置
  • Check Lead Fullstack Engineer (Full-time Remote) への配置
  • Check [HCM] Senior Backend developer (PHP Laravel) (Full-time Remote) への配置
  • Check Senior Software Engineer (Full-time Remote) への配置
  • Check Middle/ Senior Fullstack Developer (Strong C#, .NET Core) への配置
  • Check Senior Backend Engineer への配置
  • Check DevOps engineer (AWS/Kubernetes/WP) への配置
  • Check Product Owner への配置
View More
View Less