AI Conversation Research & Evaluation Specialist

Tugas kerja

About Gramian Consulting Gramian Consulting is a boutique consultancy specializing in IT professional services and engineering talent solutions. The firm leverages a strong background in software engineering and leadership to connect companies with professionals who meet their specific needs, thereby supporting the development of high-performing teams. The Opportunity: AI Conversation Research & Evaluation Specialist Gramian Consulting is seeking an AI Conversation Research & Evaluation Specialist to support AI conversation evaluation research. This full-time role, based in Saudi Arabia, involves reviewing interactions between individuals and AI models to assess contextual understanding and identify opportunities for improvement. The ideal candidate will possess 2-5 years of relevant professional experience and strong analytical skills. Key Responsibilities Review AI conversations that involve real-world interpersonal situations and everyday decision-making. Identify missing context and recommend relevant follow-up questions to enhance the quality of AI responses. Evaluate whether gathering additional information would significantly alter an AI-generated recommendation. Distinguish between critical information that impacts a decision and non-essential or merely descriptive details. Conduct qualitative reviews and document findings using structured evaluation guidelines. Assess if conversations adequately consider relevant personal, psychological, social, financial, practical, and organizational factors. Identify missed opportunities for clarification, deeper understanding, and information gathering. Review conversations related to relationships, work, education, family, wellbeing, planning, and resource constraints. Collaborate with research teams to ensure consistent, evidence-based assessments. Maintain high standards of accuracy, clarity, and consistency while adhering to project deadlines. Required Qualifications and Experience Candidates must have at least 3-5 years of relevant direct professional experience in one or more of the following domains: Counseling, Therapy, or Psychology: Master's degree in Social Work (MSW) or equivalent graduate-level qualification, with LCSW, LICSW, LMSW, or an equivalent applicable social work license/credential. Experience in client assessment, case review, interviewing, or counseling. Master's degree in a relevant discipline with an LMFT, LPC, LCPC, or equivalent counseling/therapy license/credential. PhD or PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology. Relevant experience in counseling, therapy, psychological assessment, interviewing, or client-facing professional work. Qualitative Research, HCI, or Communication: MA, PhD, or equivalent advanced expertise in Human-Computer Interaction, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline. Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, or qualitative coding. Ability to systematically analyze conversations and identify missing context, communication patterns, and opportunities for clarification. Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields: Relevant professional qualification, credential, or demonstrated subject-matter expertise. At least 5 years of direct advisory or client-facing experience for candidates qualifying through this pathway. Experience in coaching, mediation, student advising, organizational consulting, or comparable professional work. Demonstrated ability to gather context, understand individual or organizational needs, and provide informed recommendations. Skills and Attributes Strong analytical skills with the ability to distinguish information that could materially change a recommendation from less relevant details. Ability to provide structured feedback to improve conversational quality. Work Type This is a full-time position.