Pricing For Talent
Login Free Trial Book a Demo

Databricks / Unity Catalog Lead Architect

New

Skills

Advanced SQL Azure Data Lake Basic SQL Databricks DataBricks Delta Lake PySpark Terraform Unity Catalog

Reports To: Data Governance / Purview Solution Architect

Engagement: MVP: 4-month engagement· Future State: 12-month engagement

Role Summary

Owns the Unity Catalog technical design - metastore structure, catalog/schema conventions, access model, and lineage capture - ensuring UC is a reliable technical-metadata source feeding Purview.

Key Responsibilities

  • Design the UC metastore, catalog-per-domain structure, and naming standards
  • Define the grants/access model in alignment with domain data owners
  • Ensure UC-native lineage is captured correctly for curated, analytics-ready tables
  • Own the technical integration point between UC and Purview scanning
  • Guide the engineering team on notebook/pipeline patterns that preserve lineage
  • Troubleshoot metastore/connector issues encountered during scanning

Required Qualifications

  • 6+ years data platform engineering experience, including 2+ years hands-on Unity Catalog
  • Deep knowledge of Databricks (clusters, jobs, Delta Lake, lineage APIs)
  • Experience integrating UC with external catalogs (Purview, Collibra, or similar)
  • Strong SQL and Python/PySpark

Preferred Qualifications

  • Databricks certification (Data Engineer Professional or similar)
  • Experience in banking/financial-services data platforms

Key Skills & Tools

Databricks, Unity Catalog, Delta Lake, PySpark, SQL, Azure Data Lake Storage, Terraform (a plus)

No forms. Your profile is generated instantly.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: 4 Months

Share this job:

Similar Jobs

Software Engineer at Acquia

Posted 140 days ago

Develop and maintain customer-facing applications and microservices on Kubernetes

Collaborate with cross-functional teams to document product and development details

Ansible AWS Devops Engineer

Principal AI Engineer Role

Posted 140 days ago

Hire a remote Principal AI Engineer

Develop customer experience automation solutions

Ai Automation AWS Cloud Computing

Forward Deployed Software Engineer

Posted 140 days ago

Lead technical implementation and optimization of data platform

Serve as primary technical contact for key accounts

Airflow AWS Databricks Engineer

Staff Software Engineer Role

Posted 140 days ago

Lead backend development for AI-powered data solutions

Architect scalable systems and APIs for enterprise clients

Agile Methodologies Angular AWS CI/CD

Staff Software Engineer

Posted 140 days ago

Revolutionize enterprise data operations through AI solutions.

Automate and accelerate data tasks for overworked data teams.

Ai Airflow Ansible Api Development

Staff ML Engineer, Apollo

Posted 140 days ago

Lead development of scalable ML systems

Advance Apollo's AI-native product features

Airflow Architecture Databricks Engineer

Senior ML Engineer, Remote

Posted 140 days ago

Design and productionize scalable machine learning systems

Personalize user experiences using data-driven models

Cloud Computer science Databricks Engineer

Backend Engineer Role Overview

Posted 140 days ago

Develop scalable backend solutions

Collaborate cross-functionally with teams

Docker Elasticsearch Engineer Kubernetes

Senior ML Engineer II at Apollo

Posted 140 days ago

Build and productionize Machine Learning models for Apollo products

Optimize users' experience at all stages of their product journey

Airflow Ai Systems Cloud Computer science

Senior Frontend Engineer

Posted 140 days ago

Implement best practices

Drive quality and innovation

Android AWS CI/CD Cloud

Senior Fullstack Engineer

Posted 140 days ago

Work with Apollo’s 200M+ Leads data and build features on top of it

Improve data quality and build customer-facing products

Android API Design Automated Testing AWS

Collaborative Frontend Engineer

Posted 140 days ago

Drive best practices in technology and processes

Collaborate with cross-functional teams

Docker Elasticsearch Engineer Front end

ML Engineer on Apollo Team

Posted 140 days ago

Build and deploy ML models for Apollo products.

Enhance user experience through data-driven insights.

Airflow Cloud Computer science Databricks

Senior Analytics Engineer

Posted 140 days ago

Hiring a Senior Analytics Engineer remotely

Axios - Smart brevity

Airflow Analytics BigQuery Bi tools

Principal Data Engineer

Posted 140 days ago

Set technical direction for data initiatives

Design and build scalable data pipelines

Architecture Databricks Data Compliance Data Modeling

Principal Product Manager - AI Platform

Posted 140 days ago

Lead and drive product initiatives for AI and Data Product Platform

Adopt and implement data mesh framework across the organization

Agile Agile Development Databricks Financial Acumen

Senior DevOps Engineer

Posted 140 days ago

Build automation processes and best practices in software development.

Contribute to system efficiencies through reusable automation scripts and tools.

Ai Architecture Automated Testing AWS

Platform Engineer Role

Posted 140 days ago

Support Windows systems infrastructure

Automate tasks using scripting languages

Active directory Ansible AWS Cloudformation

Windows System Platform Engineer

Posted 140 days ago

Support and manage Windows infrastructure effectively

Automate daily tasks using scripting and programming languages

Automation AWS Cloudformation Engineer

Cloud Network Architect

Posted 140 days ago

Design and architect secure cloud network solutions.

Develop and enforce cloud network best practices.

Architecture Automation AWS Cisco

Principal Product Manager

Posted 140 days ago

Lead multiple product initiatives from concept to delivery.

Drive data business transformation and enhance customer outcomes.

Agile Agile Development Analytics Databricks

Senior Data Architect for Blackbaud

Posted 140 days ago

Lead data strategy and architecture for Blackbaud

Design breakthrough products in Data Intelligence

Big Data Databricks Data Modeling Machine Learning

Enterprise Data Architect Role

Posted 140 days ago

Design and oversee enterprise-wide data architecture

Develop strategies for data platforms and analytics

AWS Big Data Databricks Data Modeling

Windows Platform Engineer

Posted 140 days ago

Automate and manage Windows infrastructure

Leverage cloud and IaC technologies

Active directory Ansible AWS Cloudformation

Cloud DevOps Security Manager

Posted 140 days ago

Lead vulnerability and patch management programs

Drive automation and operational excellence

Ansible Architecture AWS CI/CD

Principal Data Engineer Role

Posted 140 days ago

Modernize and optimize legacy data warehouse systems

Design and implement scalable data pipelines

Architecture AWS Databricks Devops

Senior AI/ML Engineer

Posted 140 days ago

Design and optimize scalable AI/ML models

Integrate AI solutions across business units

Databricks Hadoop Java Kafka

Principal Product Manager, Data

Posted 140 days ago

Lead strategic product initiatives for data and AI platforms

Drive organizational adoption of data mesh framework

Agile Agile Methodologies Databricks Financial Acumen

Windows Platform Engineer Staff

Posted 140 days ago

Manage and optimize Windows infrastructure

Automate system tasks using scripting and IaC tools

Active directory AWS Cloudformation Engineer

AI/ML Senior Engineer Role

Posted 140 days ago

Design and optimize scalable AI/ML models

Integrate AI solutions across business units

Databricks Engineer Hadoop Java

Site Reliability Engineer

Posted 140 days ago

Ensure reliability and scalability of blockchain infrastructure.

Automate deployments and operational processes across multi-cloud environments.

Automation AWS CI/CD Cloud

Efficient Supply Chain Solution

Posted 140 days ago

Enhance supply chain efficiency

Promote teamwork and collaboration

AWS CI/CD Engineer Express.js

Full Stack Software Engineer

Posted 140 days ago

Join a growing team in India

Help build the future of low code software development

Ansible API Design Automation Ci/cd Pipelines

Senior Juju Software Engineer (Go)

Posted 140 days ago

Seeking a Senior Software Engineer proficient in Go for Canonical's Juju software

Responsibilities include design, delivery, support, collaboration, coaching, and development

Ansible C++ Devops Docker

Distributed Systems Testing Engineer

Posted 140 days ago

Validate and enhance the quality of distributed systems

Automate testing approaches for reliability and resilience

Ci/cd Pipelines CircleCI Cloud Technologies Continuous integration

Senior Site Reliability Engineer

Posted 140 days ago

Architect and run OpenStack and Kubernetes infrastructure

Enable devsecops for applications on the infrastructure

Ansible Architecture Automation AWS

Distributed Systems Test Engineer

Posted 140 days ago

Automate distributed systems testing and validation

Enhance CI/CD pipelines for cloud product integration

Architecture AWS CircleCI Data Analytics

Site Reliability Engineer Role

Posted 140 days ago

Enhance reliability of open source infrastructure

Automate operations for scalability

Architecture Automation AWS CI/CD

AI Engineering Manager

Posted 140 days ago

Lead AI agency for high-impact solutions

Improve engineering practices with AI applications

Architecture AWS Aws Sagemaker Deep Learning

Senior AI DevOps Engineer

Posted 140 days ago

Transform education and productivity using the latest AI technologies

Develop tools for Goodnotes AI and foundational technologies

AWS CircleCI Datadog Devops

DevOps Engineer - AI Infrastructure

Posted 140 days ago

Transform education and productivity using AI technologies

Build and maintain infrastructure for AI technology

AWS CircleCI Datadog Devops

Senior DevOps Engineer - AI & Data Infrastructure

Posted 140 days ago

Transform education and productivity through AI technology

Build and maintain infrastructure for running end-to-end AI technology

AWS CircleCI Datadog Devops

Fullstack Engineer, Cross-Platform

Posted 140 days ago

Building cross-platform apps for iOS, Web, Android, and Windows

Improving UI performance and memory optimizations

Android AWS Datadog Engineer

Senior Web Cross-platform Engineer

Posted 140 days ago

Deliver cross-platform product experiences

Drive innovation and implement new technologies

Android AWS B2b Engineer

AI Engineering Manager Role

Posted 140 days ago

Lead development of advanced AI features

Manage and mentor engineering teams

Architecture AWS Deep Learning Full-stack Development

Lead Backend Engineer Kotlin

Posted 140 days ago

Architect scalable backend cloud solutions

Enable secure authentication and identity management

Android AWS Aws Cdk Cloud

AI Engineering Lead Role

Posted 140 days ago

Lead and mentor AI engineering team

Develop and deploy AI productivity solutions

AWS Cloud Kubernetes LangChain

AI Engineering Team Lead

Posted 140 days ago

Lead and mentor an AI engineering team

Architect and deploy AI-driven productivity solutions

AWS Cloud Kubernetes LangChain

Senior JVM Backend Engineer

Posted 140 days ago

Architect and scale backend systems

Implement and maintain clean, robust code

Android AWS Aws Cdk Cloud