Data Scientist

Overview

We are seeking a Data Scientist to support applied machine learning, natural language processing, AI evaluation, data quality, and analytical product development for a federal permitting modernization initiative.

The Data Scientist will work across structured and unstructured data, including public comments, permitting records, compliance information, documents, and geospatially linked datasets. A key focus of the role will be developing representative evaluation datasets and quantitative methods to measure the performance, reliability, and limitations of AI-enabled and agentic solutions.

This position is remote.

Duties & Responsibilities

• Develop analytical and machine-learning approaches supporting permitting and natural-resource use cases.
• Build and maintain labeled datasets, evaluation corpora, and benchmark datasets for AI-enabled services.
• Define quantitative evaluation methodologies for classification, extraction, retrieval, summarization, entity matching, and other AI/NLP capabilities.
• Evaluate LLM and agentic workflows, including output quality, retrieval performance, task completion, failure modes, and reliability.
• Analyze public-comment datasets and develop reusable features and evaluation datasets for comment-processing services.
• Apply NLP and machine-learning techniques to structured and unstructured data.
• Evaluate embedding models, semantic retrieval strategies, document-ranking techniques, and AI-generated outputs.
• Develop statistical approaches for detecting anomalies, reconciliation issues, missing data, and inconsistent records.
• Partner with Content Pipeline Engineers to measure OCR, extraction, and document-retrieval performance.
• Partner with Data Engineers to ensure analytical datasets and evaluation pipelines are reproducible and production-ready.
• Support entity matching and harmonization across data originating in multiple source systems.
• Design experiments and validation approaches with clearly defined success metrics.
• Contribute to certified analytical data products, metrics, and governed datasets.
• Develop reproducible Python analyses, models, evaluation pipelines, and technical documentation.
• Communicate model performance, limitations, uncertainty, and failure modes to technical and non-technical stakeholders.

Educational Requirements

• Bachelor's degree in Data Science, Statistics, Computer Science, Mathematics, Engineering, or a related quantitative discipline.

Requirements

• 5+ years of relevant data science, applied machine learning, or quantitative analytics experience.
• Strong Python and SQL skills.
• Experience with statistical analysis, machine learning, NLP, and unstructured text data.
• Experience designing labeled datasets, benchmark datasets, and train/test evaluation frameworks.
• Experience evaluating LLM, RAG, or agentic AI solutions, including retrieval quality, output accuracy, and failure modes.
• Strong understanding of model-evaluation techniques, including precision, recall, F1, ranking metrics, confusion matrices, and related measures.
• Experience with embeddings, semantic retrieval, or vector-based search.
• Experience developing reproducible analytical and evaluation workflows.
• Ability to distinguish data-quality issues from model-performance issues.
• Strong ability to communicate analytical results, limitations, and uncertainty.

Bonus Skills

• Experience with agent evaluation frameworks or LLM observability tools.
• Experience with agentic frameworks such as LangChain, LangGraph, or similar orchestration frameworks.
• Experience with OpenSearch, vector databases, or enterprise search platforms.
• Experience with document AI or OCR pipelines.
• Experience with geospatial analysis.
• Experience developing entity-resolution, record-linkage, or data-quality models.
• Experience working in the U.S. Federal Government space.
• Experience with permitting, regulatory, environmental, energy, or natural-resource datasets.