Find your next role

Introduction

At IBM, work is more than a job — it’s a calling: to build, to design, to code, to consult, to think along with clients and partners. To make markets, invent, and collaborate. To do things you thought were impossible.

IBM Infrastructure is seeking an early‑career AI Engineer to join our System Z team. This role is ideal for recent graduates or professionals at the start of their careers who want to apply modern AI techniques to enterprise‑grade systems that power the world’s most critical institutions, including banks, healthcare providers, and government agencies.

You’ll work with Spyre accelerator cards, IBM‑designed hardware optimised for running Large Language Models (LLMs) on System Z. This is a unique opportunity to gain experience across the full technology stack, from low‑level hardware and firmware interfaces to AI runtimes, APIs, and monitoring systems — all while being supported by experienced engineers and architects.

Why You’ll Love Working Here

• Opportunity to work on mission‑critical systems at global scale
• Hands‑on experience across hardware, systems software, and AI
• Structured mentorship and learning from experienced IBM engineers
• A collaborative, inclusive environment that values curiosity and continuous learning
• Access to IBM training, resources, and career development opportunities

Your role and responsibilities

LLM Integration and Deployment

• Assist in developing and integrating Large Language Models on System Z using Spyre accelerator cards
• Work with senior engineers to support model loading, inference execution, and resource management
• Help enable AI workloads for both traditional mainframe applications and modern, cloud‑native services
• Gain exposure to system‑level design considerations across hardware, firmware, runtime, and application layers

Performance Profiling and Optimisation

• Learn how to profile LLM inference workloads to measure latency, throughput, memory usage, and power consumption
• Use performance analysis tools to help identify bottlenecks and optimisation opportunities
• Contribute to optimisation efforts related to:
  • Hardware utilisation
  • Memory management
  • Kernel and runtime efficiency
  • Batch and workload scheduling
• Document performance findings and assist with creating internal best‑practice guides

Failure Analysis and Debugging

• Support investigation and debugging of inference and system‑level issues
• Learn how to trace problems across multiple layers of the stack, including firmware, drivers, runtimes, and applications
• Collaborate with hardware engineers, firmware developers, and system architects to resolve complex issues
• Help improve test coverage and automated regression testing

Observability and Telemetry

• Assist with building and maintaining monitoring solutions for AI workloads in production
• Help instrument systems to capture metrics such as:
  • Model performance
  • Hardware utilisation
  • Error rates
  • Overall system health
• Support the development of dashboards, alerts, and reports for operational teams
• Gain experience with time‑series data, logging systems, and monitoring platforms

Collaboration and Documentation

• Work closely with cross‑functional teams including hardware, firmware, system software, AI research, and applications
• Participate in technical reviews and design discussions with guidance from senior engineers
• Create clear technical documentation and contribute to shared team knowledge
• Communicate technical ideas clearly across different audiences

Required education

None

Preferred education

Bachelor's Degree

Required technical and professional expertise

• Bachelor’s degree in Computer Science, Computer Engineering, or a related technical field
• Strong programming fundamentals in Python, with some exposure to C/C++
• Understanding of machine learning fundamentals, particularly neural networks and transformers
• Basic knowledge of computer architecture (memory systems, parallel processing, I/O)
• Comfortable working in Linux environments using command‑line tools and scripts
• Exposure to debugging and problem‑solving techniques
• Familiarity with basic monitoring or logging concepts
• Strong attention to detail and analytical thinking
• Good written and verbal communication skills
• Curiosity about how systems work across different layers
• Willingness to learn, ask questions, and collaborate as part of a team

Preferred technical and professional experience

• Master’s degree or coursework in systems, AI, machine learning, or computer architecture
• Experience with frameworks such as PyTorch, TensorFlow, or Hugging Face Transformers
• Exposure to model optimisation techniques such as quantisation, pruning, or distillation
• Familiarity with inference or serving frameworks (e.g., ONNX Runtime, TensorRT, TorchServe)
• Some knowledge of hardware acceleration or GPU programming concepts
• Experience with monitoring tools such as Prometheus, Grafana, or ELK Stack
• Exposure to profiling tools (e.g., perf, VTune, Nsight)
• Familiarity with Docker, Kubernetes, or CI/CD pipelines
• Any exposure to mainframe systems, z/OS, or enterprise computing environments
• Coursework, research projects, or open‑source contributions related to ML systems or performance engineering