Research Assistant 2 – AI Solutions Architect (LLM prompt engineering, training, RAG)

Job title:

Research Assistant 2 – AI Solutions Architect (LLM prompt engineering, training, RAG)

Company

McGill University

Job description

Please refer to the job aid for instructions on how to apply.If you are an active McGill employee (ie: currently in an active contract or position at McGill University), do not apply through this Career Site. Login to your McGill Workday account and apply to this posting using the Find Jobs report (type Find Jobs in the search bar).Position Summary:We are seeking a part-time (20 hours/week) Research Assistant 2 to take on a research & development role in the Shared Reality Lab’s open source IMAGE project (image.a11y.mcgill.ca), focused on making photos, charts, and maps available to people who are blind or low vision. We have funding for through the end of 2025, with possibility for renewal, for a project with Humanware, a leading provider of solutions for people who are blind or low vision, to prototype compelling experiences for users of their upcoming Monarch Android pin array tablet (https://www.humanware.com/en-usa/monarch). You will be responsible for researching, selecting, and integrating machine learning components into the IMAGE server architecture to enable the team’s designers and developers to create useful end-user experiences on the Monarch.Key responsibilities include:

• Integrate multimodal LLMs (e.g., llama3.2-vision) into IMAGE by using prompt engineering, RAG, few-shot training, and other techniques to reliably extract key information from textbook diagrams.
• Make recommendations regarding tradeoffs in accuracy, performance, and cost between models, model sizes, and inferencing locally or in the cloud.
• Where appropriate, proactively seek out machine learning tools and components that will provide useful information, including traditional models, as well as integrating LLM components.
• Review and revise the existing IMAGE machine learning components to optimize for efficiency/scalability and quality of results. Convert them to LLMs instead of local models where appropriate.
• Document and create example code to help others use the IMAGE platform efficiently.
• Review and merge code from students and other team members.
• Contribute to academic publications resulting from work on the project.

Qualifications:Master’s degree.Requirements (we expect that qualified candidates will not match with all of these, so please apply, regardless, since we can likely assist you with coming up to speed in some areas):

• Solid command of the current machine learning landscape in terms of deployable models, cloud services, and LLMs, to be able to find and integrate solutions focused on extracting information from photographs and diagrams (e.g., scientific textbook figures).
• Experience deploying machine learning tools in a production environment including running LLMs both locally and in the cloud via tools such as ollama and open-webui
• Experience with inferencing, and not just model building, including profiling, optimizing and scaling machine learning solutions to meet performance and quality targets.
• Linux command line
• Familiar with (or able to quickly learn) Docker, containerization, network layer designs, client/server design
• Git, and Github flow method of raising and merging pull requests
• Python
• JSON schema definition
• Mentoring junior designers/developers
• Quebec resident (mandatory)
• Able to work on-site at McGill at least one day/week (entirely remote possible for exceptional candidates, but must reside in Quebec)
• Bonus: Experience writing conference and journal papers and presenting results in academic venues
• Bonus: French, spoken and/or written
• Bonus: Experience working with the blind/low vision community

To apply for any of the above positions, please include the following items in your application:

• A brief letter of application, describing your qualifications and relevant experience to the position of interest, along with your dates of availability.
• Detailed CV with links to online papers and/or project portfolios.
• Two (2) reference letters (sent separately).

The positions are available immediately, with an initial appointment of up to one year.Informal inquiries are welcome, please email .The conducts research in audio, video, and haptic technologies, building systems that leverage their capabilities to facilitate and enrich both human-computer and computer-mediated human-human interaction. The lab is part of the and of . McGill, one of Canada’s most prestigious universities, is located in Montreal, a , especially for students.Hourly Salary: $30.75Hours per Week: 20 (Part time)Location: McConnell Eng. Bldg.Supervisor: ProfessorPosition Start Date: 2024-12-24Position End Date: 2025-12-31Deadline to Apply: 2025-01-31This position is covered by the Association of McGill University Research Employees (AMURE) collective agreement.McGill University hires on the basis of merit and is strongly committed to equity and diversity within its community. We welcome applications from racialized persons/visible minorities, women, Indigenous persons, persons with disabilities, ethnic minorities, and persons of minority sexual orientations and gender identities, as well as from all qualified candidates with the skills and knowledge to productively engage with diverse communities. McGill implements an employment equity program and encourages members of designated groups to self-identify. Persons with disabilities who anticipate needing accommodations for any part of the application process may contact, in confidence, .

Expected salary

Location

Canada

Job date

Sun, 05 Jan 2025 04:57:54 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (jobsnearcanada.com) you saw this job posting.