Intermediate Machine Learning Engineer, GitLab

• A relevant Master’s degree and 2 or more years of experience in ML or PhD degree with a focus on Machine Learning or Data Science
• Professional experience with Python
• Experience with performance and optimization problems and a demonstrated ability to both diagnose and prevent these problems
• Comfort working in a highly agile, intensely iterative software development process
• Demonstrated ability to onboard and integrate with an organization long-term
• Positive and solution-oriented mindset
• Effective communication skills: Regularly achieve consensus with peers, and clear status updates
• An inclination towards communication, inclusion, and visibility
• Experience owning a project from concept to production, including proposal, discussion, and execution
• Self-motivated and self-managing, with strong organizational skills
• Demonstrated ability to work closely with other parts of the organization
• Share our Values, and work in accordance with those values
• Ability to thrive in a fully remote organization
• Professional experience with prompt engineering and Retrieval Augmented Generation (RAG)
• Experience with evaluation of Machine Learning models or AI features
• Experience with model deployments on cloud platforms like AWS and GCP
• Proven experience designing and implementing LLM evaluation systems
• Strong understanding of ML model architectures
• Expertise in ML evaluation metrics and dataset management
• Demonstrated ability to build production-grade ML infrastructure
• Practical experience with Python-based ML frameworks and evaluation tools (e.g., Langsmith, DSpy)

Desirable

• The Custom Models team is responsible for Duo Self-Hosted, a key component in GitLab AI that allows customers to run GitLab Duo features on completely private environments, connecting GitLab to their own AI models
• They work collaboratively with numerous teams to ensure a complete lifecycle of assessing models, evaluating features, implementing controls for customization and guaranteeing a smooth experience for our largest customers
• Why us? This isn't just a job; it's your chance to shape the future of AI at GitLab
• Your expertise in backend development will be critical to your success
• Develop evaluation techniques to assist feature teams on guaranteeing the quality of their features on new models
• Evolve the Evaluation Runner, our internal tool for scaling AI Feature evaluation
• Keep up with the industry to explore brand new models
• Deploy and manage LLMs internally for evaluation and development
• Collaborate with product managers, engineers, and other stakeholders as a machine learning specialist, guiding them on effective implementation of AI features
• Advocate for improvements to product quality, security, and performance
• Solve technical problems of moderate scope and complexity
• Craft code that meets our internal standards for style, maintainability, and best practices for a high-scale machine-learning environment. Maintain and advocate for these standards through code review
• Confidently ship small features and improvements with minimal guidance and support from other team members. Collaborate with the team on larger projects
• Participate as a reviewer or project maintainer in one or more engineering projects
• All team members are encouraged and expected to incorporate AI into their daily workflows to drive efficiency, innovation, and impact across our global organisation
• The team works closely with these other teams within the organization:
• AI Framework
• MLOps
• Model Validation
• Duo Chat

• We offer benefits to manage your health, wealth, and well-being regardless of location
• Flexibility in schedule to be there for life’s important moments
• Equity compensation & Employee Stock Purchase Plan offered
• Generous Paid Time Off

Total funding: $434.9m

GitLab is an open-source collaboration platform for developers. It enables teams to build code quickly from anywhere without compromising on security or quality. GitLab already has an estimated 30M+ users from all over the world.