Applied LLM Prompt Engineer (Data Science)

ICF is a mission-driven company filled with people who care deeply about improving the lives of others and making the world a better place. Our core values include Embracing Difference; we seek candidates who are passionate about building a culture that encourages, embraces, and hires dimensions of difference. @ ICF we are fearless in finding new ways to solve problems, relentless in making sure it pays off for our clients and committed to making a positive change in the world. Join our community of mission-driven creatives, communicators, strategists, and technologists to challenge the status quo. 

As a Prompt Engineer, you will be responsible for designing and implementing the natural language processing (NLP) and machine learning (ML) models that power our web3 documentation chatbot. You will work closely with our product and engineering teams to ensure that our chatbot provides accurate and helpful responses to user inquiries. 

Responsibilities 

• Design, develop, and implement custom generative LLMs (e.g. models similar to GPT ) and discriminative LLMs (e.g. models similar to BERT) 

• Create wrapper apps based on ChatGTP and other LLMs 

• Design, develop, and implement systems for AI alignment, Reinforcement Learning with Human Feedback (RLHF) instruction models, and AI guardrails. 

• Conduct rigorous tests to evaluate LLMs across standardized performance benchmarks and custom evaluations. 

• Employ cutting edge Natural Language Processing (NLP) and Machine Learning (ML) techniques to solve complex natural language problems. 

• Translate technical findings into clear, actionable insights for a non-technical audience 

• Develop and maintain documentation and best practices for our NLP and ML models. 

• Collaboratively develop distributed training infrastructure to train and fine tune LLMs 

• Collaborating closely with our data science and core development teams, you will lead efforts in designing and implementing products from scratch 

• Collaborate with data science and core development teams to integrate AI technologies into both internal and external tools 

• Provide technical leadership in AI system audits, model accuracy improvements, and tool deployment 

• Fine-tune AI models based on real-world performance data and stakeholder feedback 

Minimum Qualifications: 

• Bachelor's degree in computer science or related field  

• 8+ years of work experience as a software engineer or relevant experience 

• 5+ Strong background in natural language processing, machine learning, and data analysis 

• 5+ years of experience with Python 

• 5+ Experience with Azure  

• Must be a U.S. citizen (required by federal government for the position) 

• Active clearance required 

Professional Skills: 

• Familiarity with popular machine learning (ML) frameworks such as TensorFlow and PyTorch. 

• Excellent problem-solving skills and the ability to work independently and collaboratively. 

• Strong communication and interpersonal skills, with the ability to explain complex technical concepts to non-technical stakeholders. 

• Capacity for problem solving 

• Comfortable with collaboration and open communication across distributed teams 

• Experience with generative large language model (LLM) fine-tuning and prompt engineering 

• Experience with deep learning frameworks 

• Ability to work with data scientists, business analysts and machine learning infrastructure to connect the dots between business and technology partners 

• Demonstrates passion for developing LLM-powered products whether that is through commercial experience or open source projects you have worked on in your own time 

• Proven track record of shipping software and successfully released apps (please include names and links on your resume) 

• Experience in cloud technologies and edge computing relevant to AI deployment 

Working at ICF

We can only solve the world's toughest challenges by building an inclusive workplace that allows everyone to thrive. We are an equal opportunity employer, committed to hiring regardless of any protected characteristic, such as race, ethnicity, national origin, color, sex, gender identity/expression, sexual orientation, religion, age, disability status, or military/veteran status. Together, our employees are empowered to share their expertise and collaborate with others to achieve personal and professional goals. For more information, please read our EEO & AA policy.

Reasonable Accommodations are available, including, but not limited to, for disabled veterans, individuals with disabilities, and individuals with sincerely held religious beliefs, in all phases of the application and employment process. To request an accommodation please email icfcareercenter@icf.com and we will be happy to assist. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.

Read more about workplace discrimination rights, the Pay Transparency Statement, or our benefit offerings which are included in the Transparency in (Benefits) Coverage Act.

Pay Range - There are multiple factors that are considered in determining final pay for a position, including, but not limited to, relevant work experience, skills, certifications and competencies that align to the specified role, geographic location, education and certifications as well as contract provisions regarding labor categories that are specific to the position. The pay range for this position is: