Development Engineer in Test (SDET) – ML & LLM Systems
Washington D.C., District of Columbia
Overview
Make your mark at Comcast -- a Fortune 30 global media and technology company. Become part of our award-winning, international engineering team that turns big ideas into cutting-edge products, platforms, and solutions that our customers love. We create space to innovate, and we recognize, reward, and invest in your ideas, while ensuring you can proudly bring your authentic self to the workplace. You’ll do the best work of your career right here.
Success Profile
What makes a successful Development Engineer in Test (SDET) – ML & LLM Systems at Comcast? Check out these top traits and explore role-specific skills in the job description below.
- Results-driven
- Adaptable
- Inventive
- Entrepreneurial
- Team Player
- Problem-Solver
Responsibilities
Development Engineer in Test (SDET) – ML & LLM Systems
Job Summary
This team operates at the forefront of applied machine learning, focusing on niche and highly specialized technologies such as NLP pipelines, large language model (LLM) evaluations, and ML quality frameworks. Unlike teams that primarily train or consume models, this group specializes in evaluating, validating, and measuring LLM behavior, playing a critical role in ensuring model reliability, performance, and responsible deployment. Engineers on the team gain deep, market‑relevant expertise by continuously adapting to rapidly evolving LLM architectures, evaluation metrics, and benchmarks. The work is fast‑moving, highly technical, and offers exposure to cutting‑edge challenges that few teams encounter, making it an exciting environment for engineers who want to stay ahead of the curve in the LLM and AI evaluation space.Job Description
Key Responsibilities
Partner with product owners, engineers, and internal stakeholders to understand product functionality, technical requirements, and quality expectations; analyze testing needs and define appropriate automation strategies.
Design, develop, and maintain robust automated test code using software engineering best practices, ensuring reusability, reliability, and scalability.
Develop and enhance automated test suites for functional, regression, and performance testing across machine learning models, NLP systems, and supporting backend services.
Execute automated and manual tests as part of build validation to ensure expected behavior and maintain release quality.
Design and implement robust test strategies and automation frameworks to support evolving services, workflows, and ML model testing requirements.
Develop and maintain automated validation pipelines for data ingestion, data quality checks, schema validation, and anomaly detection across large-scale datasets.
Perform model validation and verification, including accuracy benchmarking, regression testing, drift detection, and reproducibility checks before and after deployment.
Investigate failing tests, diagnose root causes, and implement fixes to improve test stability and confidence.
Review automation code contributed by peers, providing constructive feedback aligned with internal coding and quality standards.
Coordinate QA and production build deployments in collaboration with engineering and infrastructure teams.
Support triage and resolution of integration and production issues to ensure system reliability and rapid recovery.
Communicate clearly and effectively with team members and leadership regarding risks, test results, priorities, and quality improvements.
Collaborate with engineering, product, and leadership partners to continuously improve test coverage, automation effectiveness, and end‑to‑end customer experience.
Stay current with emerging test tools, frameworks, and industry trends, and help evolve internal testing practices.
Contribute to additional team initiatives and responsibilities as business needs evolve.
Regular, consistent and punctual attendance. Must be able to work nights and weekends, variable schedule(s) and overtime as necessary.
Must Have Skills:
2+ years of industry experience as a Software Development Engineer in Test (SDET).
Strong experience with API testing and solid understanding of RESTful APIs.
Hands-on experience with test processes, test design, test case creation, and defect management systems.
Strong coding skills with hands-on experience in Java and Spring Boot, and Python.
Prior Experience with testing Machine Learning models, NLP services, Large Language Models (LLMs) and LLM evaluation frameworks.
Demonstrated ability to design, build, and maintain test automation frameworks.
Comfortable working in Linux environments; hands-on experience with container technologies such as Docker and Kubernetes.
Experience building or working with CI/CD pipelines using tools such as Jenkins.
Working knowledge of AWS services, including S3.
Experience with NoSQL databases such as MongoDB.
Familiarity with Agile development methodologies and working within cross-functional Scrum teams.
Strong written and verbal communication skills, with the ability to collaborate effectively across teams.
Highly detail-oriented, analytical, and proactive, with a strong passion for quality, automation, and continuous improvement.
Prior Comcast experience preferred
Here's a look at just some of the perks and benefits we make available to our US-based employees:
Medical & Dental
401(k) Savings Plan
Generous paid time off
Life Milestones - from adoption assistance, childcare resources, pet insurance, and more, Comcast supports you at all life stages.
Courtesy Services - We offer all of our full-time employees in serviceable areas free digital TV and internet.
Learn more atjobs.comcast.com/life-at-comcast/benefits
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive other benefits and privileges of employment. Please contact us to request an accommodation.
This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.
Comcast is an EOE/Veterans/Disabled/LGBT employer.
Discount tickets for Universal Resorts, including theme park tickets and onsite hotel rooms.
Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.
Skills:
CI/CD; Kubernetes; Machine Learning (ML); Software Coding
Salary:
Primary Location Pay Range: $82,288.02 - $123,432.04
Comcast intends to offer the selected candidate base pay within this range, dependent on job-related, non-discriminatory factors such as experience. The application window is 30 days from the date job is posted, unless the number of applicants requires it to close sooner or later.
Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits to eligible employees. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That’s why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary on our careers site for more details.
Education
Bachelor's Degree
While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.
Relevant Work Experience
2-5 Years
Hear from our people
“Comcast brings to the table the leading edge in the technology, so that sets the baseline for the features that are available to our customers”
Albert Vice President, Product Management
“I mean, that's one of the best things about working here is that there have been very few times when I've been set, when I've been told, No, you can't do this.”
Greg Engineer
“It is incredibly rewarding to see the work that we've done impact people in such a meaningful way, and we're bringing joy to people's lives.”
Faisal Executive Director, Machine Learning
Culture
At Comcast you can have work/life balance in an inclusive environment where you can be your authentic self.
Comcast provides many opportunities to build skills and grow in your career and a top-notch benefits, time off and perks package.
We are committed to delivering reliable, effortless experiences to our customers no matter how they choose to interact with us.
You will build personal relationships on a growing team that while making Xfinity a best-in-class, most-loved brand.
More About Comcast
-
Comcast Unveils Industry-First Gateway Designed for Apartment Buildings Enabling ‘Instant’ Connectiity for Residents
Comcast has launched a new, innovative gateway with Tri-Band WiFi 6E technology that delivers faster speeds, lower latency, and ‘Instant On’ connectivity to apartment residents.
-
Xfinity Mobile, Comcast Business Mobile Customers Unlock PowerBoost – Delivering WiFi Speeds of Up to a Gig – with the New Samsung Galaxy S25 Series
Android fans have something to celebrate now that the new Samsung Galaxy S25 Series can unlock PowerBoost – a unique feature that delivers speeds of up to 1 gigabit-per-second (Gbps) over WiFi to Xfinity Mobile and Comcast Business Mobile customers.
-
Network Innovation Delivering Business Solutions at Scale
Comcast Business is hyper scaling the availability of its Dedicated Internet connectivity solution, which provides business customers with unparalleled speeds, 99.99 percent uptime, and proactive network monitoring.
Tech Engineering in Action
Talent Community
Join our talent community so we can get to know you better, learn more about your skills and experience, and how they could align with future open positions at Comcast.
Job Alerts
Sign up for Job Alerts to be the first to know about new opportunities. After signing up or logging in to Workday, select Job Alerts in the top right corner to create a new alert or edit an existing one.