Software Architect - AI/ML

Heroku is seeking a software architect to be the tech lead for exciting new AI, machine learning, and LLM offerings for our customers. Delivering on the promise of AI involves a lot of data, security, and privacy concerns, and you must be able to simplify these complexities and deliver delightful AI capabilities in Heroku’s opinionated and powerful developer experience. You will focus on a range of AI opportunities: from hosting our customers’ AI workloads within a fully managed service, to delivering Heroku assistants for operators and developers, and using AI to improve the efficiency of Heroku’s internal platform.

This is a senior technical individual contributor role at Heroku. As a Heroku software architect focused on AI, you bring deep technical knowledge of programming, architecture patterns, machine learning fundamentals, AI frameworks, compliance, costs, and the latest technologies associated with delivering managed AI services to developers. This role partners with Heroku AI engineering director and product manager to define and deliver AI offerings. This role mentors and grows AI, machine learning, and LLM know-how across Heroku engineering teams.

This position require top-notch technical skills and outstanding analytical and influencing skills paired with extraordinary business insight. It is a multi-functional role that requires alignment building and communication within a large engineering organization. Software architects at Heroku write code.

Responsibilities

• Coordinate and drive the end-to-end architecture of AI, ML, and LLM capabilities across the Heroku platform
• Craft overall architectural principles, frameworks, and standards
• Create solutions that are enterprise scalable and maintainable
• Drive strategic technology initiatives, working with product managers, other architects, engineers, subject-matter experts, and the broader Salesforce organization
• Work in a consultative fashion to improve communication, teamwork and alignment among teams inside and outside of the organization
• Analyze and provide feedback on product strategy
• Provide input into long-range product requirements, standard methodologies, and operational guidelines, with focus on continuous improvement of platform reliability and serviceability
• Perform engineering research, case studies, and build prototypes for innovative technologies and their applications at Heroku
• Build product and share knowledge by authoring, maintaining, and/or reviewing code.
• Ensure that we are continuously raising our standard of engineering excellence in AI
• Mentor and guide engineers in technical skills and career advancement
• Participate in the AI team’s on-call rotation to address complex problems in real-time and keep services operational and highly available

Qualifications

• BS or MS in Engineering, Computer Science, Artificial Intelligence, Machine Learning, or related disciplines or equivalent experience in technical leadership roles
• Minimum of 5+ years of proven track record in individual-contributor tech leadership roles in software engineering teams, with proven expertise in AI, including generative AI, machine learning, and enterprise / PaaS environments. Experience in multiple AI technologies, services and products, as well as legal and privacy requirements faced by organizations wanting to build AI experiences.
• Experience building, delivering, and operating AI capabilities in AWS infrastructure
• Experience with machine learning software frameworks including PyTorch, pgvector, JAX, Neuron, MXNet, TensorFlow, or similar technologies
• Mastery of multiple programming languages and platforms, especially for machine learning use cases
• Familiarity with hardware optimizations for AI, including Inferentia, Trainium, and GPUs in general
• 15+ years of software development experience
• Ability to independently craft and deliver large sophisticated projects
• Ability to mentor to team members to support their understanding of the architecture and aid in their technical and professional development
• Experience with agile development methodologies (e.g., scrum)
• Ability to work with other architects, principal architects, engineering managers, and the CTO to craft the architectural vision and roadmap for the organization
• Develops technical performance architecture that solves most complex performance issues
• Ability to drive long-term design strategies that span multiple complex projects, deliver technical reports and presentations to customers and at industry events
• Ability to build an environment for clear and open discussion of all issues. You involve the right people, from your team and others, to resolve critical issues.
• Experience in continuous delivery and short release cycles.

Preferred Skills

• Familiarity with Heroku and Salesforce platforms