About the role
As a Data Engineer at Brillio, you will be at the forefront of building robust, scalable data architectures that power complex business intelligence and AI initiatives. You will work closely with cross-functional teams, including AI/ML engineers and product stakeholders, to transform raw data into reliable, high-quality assets. Your day-to-day will involve architecting sophisticated data pipelines, implementing advanced modeling techniques, and ensuring the seamless flow of information across the enterprise, directly influencing the data-driven decision-making capabilities of our clients.
Primary Skills
- Snowflake, Airflow, DBT
- SQL
- Python/PySpark
- ETL
- CDC
Specialization
- Snowflake
- SnowPro
Job requirements
Key Responsibilities
Project Type: L3 Support
Flexibility to shift working is Mandatory.
Development Responsibilities
- Design and develop scalable data pipelines and modern data architectures using Snowflake.
- Lead migration initiatives from legacy databases to Snowflake.
- Build and optimize ELT frameworks using DBT.
- Develop and orchestrate workflows using Apache Airflow.
- Design and implement data models, including Star Schema and Snowflake Schema.
- Develop ingestion pipelines for structured and semi-structured data.
- Create reusable frameworks for data validation, reconciliation, auditing, and monitoring.
- Optimize Snowflake objects, including:
- Virtual Warehouses
- Clustering Keys
- Materialized Views
- Streams & Tasks
- Dynamic Tables
- Develop technical design documents and migration runbooks.
- Perform code reviews and enforce engineering best practices.
Migration Responsibilities
- Assess and migrate data from legacy platforms, including:
- Oracle
- SQL Server
- Teradata
- Netezza
- DB2
- SAP HANA
- On-Prem Data Warehouses
- Perform source system analysis and database rationalization.
- Design migration strategies, including:
- Full Data Loads
- Incremental Loads
- CDC (Change Data Capture)
- Historical Data Migration
- Conduct data reconciliation and validation during migration.
- Optimize transformed workloads for Snowflake architecture.
- Execute cutover planning and post-migration stabilization.
Good to have
AMS / Production Support Responsibilities
- Provide L3 support for Snowflake data platforms and pipelines.
- Monitor and manage production data workloads.
- Troubleshoot data failures, performance issues, and pipeline disruptions.
- Perform root cause analysis (RCA) and implement permanent fixes.
- Support incident, problem, change, and release management activities.
- Work closely with business users and stakeholders to resolve production issues.
- Ensure adherence to SLA and operational support commitments.
- Participate in production releases and deployment activities.