Data Automation Architect

Job description

The Data Automation Architect will work with the team that created Biml on their next-generation metadata-driven data warehouse automation framework. BimlFlex is an end-to-end solution that

accelerates the data warehouse development process: everything from modelling through deployment and operations management. We are looking for a Data Architect to lead the technical development of new features in the BimlFlex framework. It’s a big job, and it will have the potential to change the course of data warehouse development technology

Responsibilities:
• Lead Technical Development:
• Drive the design and implementation of new features in the BimlFlex framework using Microsoft Fabric or LakeFlow, ensuring high-quality, scalable, and efficient data architecture solutions.
• Data Modeling:
• Design data models and architectures that support automated data warehousing and ETL processes, ensuring alignment with business and technical requirements.
• BimlFlex Framework Expertise:
• Leverage deep expertise in Biml and BimlFlex to extend the framework’s capabilities and improve its effectiveness in automating data warehouse solutions.
• Collaboration:
• Work closely with cross-functional teams, including data engineers, software developers, and product managers, to deliver new features that enhance the BimlFlex framework.
• Optimization and Automation:
• Develop automated solutions for data integration, transformation, and loading processes using BimlFlex, improving the overall efficiency of data pipelines.
• Performance Tuning:
• Analyze and optimize the performance of BimlFlex-generated solutions to ensure they meet performance requirements in large-scale data environments.
• Architectural Best Practices:
• Define and promote architectural best practices for data warehousing, ETL, and data integration solutions, ensuring that solutions are aligned with industry standards and business goals.
• Documentation and Knowledge Sharing:
• Create and maintain comprehensive documentation for new features, providing technical guidance and training to internal teams on the usage of BimlFlex.

To be successful in this role, you will have:
• Essential Systems/Software:
• 3+ years Dimensional Modeling and Automation Patterns
• 5+ years of ETL Development experience
• Expert knowledge of MS SQL
• Expert knowledge Azure Data Factory, ETL workflows with SSIS automation
• Working knowledge of C#
• Experience:
• Excellent communications skills – both written and verbal
• Inquisitive, detail oriented, self-starter, can effectively work independently or in a team
• Nice to Have:
• Data Vault Modeling and Automation Patterns
• Auditing, logging, and notification of ETL processes
• Azure SQLDW
• Snowflake DB
• Test Automation and Continuous Integration
• Biml
• SSIS