Your Role

As a Data Engineer, you will be responsible for designing, building, and maintaining the data infrastructure that powers Basalt’s monitoring, analytics, and AI evaluation systems. You will work with large volumes of real-time and batch data coming from our AI agents, ensuring it is reliable, scalable, queryable, and actionable for both internal teams and customers.

You will work with technologies such as Python, ClickHouse, data ingestion frameworks, streaming pipelines, OLAP architectures, and observability tooling. Your work will directly impact how teams analyze agent performance, understand user behavior, and ship reliable AI features at scale.

Your responsibilities will include:

• Designing, maintaining, and improving data pipelines that collect, transform, and serve large volumes of agent monitoring data
• Managing and optimizing our ClickHouse clusters and OLAP storage to ensure speed, reliability, and scalability
• Building efficient data models and schemas that enable fast querying and deep analytics
• Implementing ingestion, transformation, and export mechanisms for both batch and streaming data
• Collaborating with AI, ML, and Product teams to expose meaningful insights, metrics, and evaluation signals
• Ensuring data quality, lineage, documentation, and observability across all pipelines
• Developing internal tools to help teams explore, analyze, and leverage monitoring data
• and bringing your expertise to push our data platform to the next level ⚙️

Ideal candidate

• 3–5+ years of experience as a Data Engineer or similar role
• Strong proficiency in Python for data processing, automation, and tooling
• Solid experience with ClickHouse (or other OLAP databases), including schema design & performance optimization
• Strong understanding of data modeling, ETL/ELT pipelines, and distributed data systems
• Experience with streaming/real-time data frameworks (Kafka, Redpanda, or equivalents) is a plus
• Ability to work with large-scale datasets and high-throughput ingestion systems
• Strong analytical mindset with the ability to transform raw data into actionable insights