Azure data engineer (Pittsburgh) Job at Cognizant, Pittsburgh, PA

bjhia3I0akpVT1NQVDhmdWVLd2JZVmpUeUE9PQ==
  • Cognizant
  • Pittsburgh, PA

Job Description

Job Title - DataBricks Data Engineer

Location - Pittsburg, PA

**Must have 8+ years of real hands on experience**

We are specifically seeking a Data Engineer-Lead with strong expertise in Databricks development.

The role involves:

  • Building and testing data pipelines using Python/Scala on Databricks
  • Hands on experience to develop and lead the offshore team to perform development/testing work in Azure data bricks
  • Architect data platforms using Azure services such as Azure Data Factory (ADF), Azure Databricks (ADB), Azure SQL Database, and PySpark.
  • Collaborate with stakeholders to understand business needs and translate them into technical solutions.
  • Provide technical leadership and guidance to the data engineering team and need to perform development.
  • Familiar with Safe Agile concepts and good to have working experience in agile model.
  • Develop and maintain data pipelines for efficient data movement and transformation.
  • Onsite and offshore team communication and co-ordination.
  • Create and update the documentation to facilitate cross-training and troubleshooting
  • Hands on experience in scheduling tools like BMC control-M and setup jobs and test the schedules.
  • Understand the data models and schemas to support the development work and help in creation of tables in databricks
  • Proficiency in Azure Data Factory (ADF), Azure Databricks (ADB), SQL, NoSQL, PySpark, Power BI and other Azure data tools.
  • Implementing automated data validation frameworks such as Great Expectations or Deequ
  • Reconciling large-scale datasets
  • Ensuring data reliability across both batch and streaming processes

The ideal candidate will have hands-on experience with:

  • PySpark, Scala, Delta Lake, and Unity Catalog
  • Devops CI/CD automation
  • Cloud-native data services
  • Azure databricks/Oracle
  • BMC Control-M

Location: Pittsburgh, PA

Job Tags

Part time, Work experience placement,

Similar Jobs

Cyntergy

Program Coordinator Job at Cyntergy

 ...Coordinator Were looking for self-starters and eager learners. Beyond that, youll need Working knowledge of computers and MS Office. Working knowledge of Deltek Vision, Newforma, Smartsheet, and SharePoint is a plus. Ability to handle multiple assignments... 

Phoenix Loss control

Field Claims Investigator Job at Phoenix Loss control

Job Description Job Description Job Type: Contract Workplace Type: Hybrid (50% remote, 50% fieldwork) Compensation: $20/hrplus$.50/mi COMPANY OVERVIEW Phoenix Loss Control (PLC) is aUS-based business servicesproviderin the cable, telecom, and utilities...

Loram Maintenance of Way

Technical Instructional Designer Job at Loram Maintenance of Way

 ...Job Description Job Title: Technical Instructional Designer \n FLSA Status: Exempt \n Department: Human Resources \n Reports...  ...development, and learning program production while providing expert consultation on program measurement, visualization, and multi-modal... 

SAVI EHS

Environment, Health and Safety Manager Job at SAVI EHS

 ...Job Description Environmental, Health and Safety Manager SAVI EHS Charleston, South Carolina (On-site) Compensation $45.00 $50.00 per hour $800 weekly per diem Employment Type Full-Time Job Description EHS Manager SAVI EHS is seeking an... 

Multipurpose Arena Fort Worth

Roving Event Security (Part-Time) Job at Multipurpose Arena Fort Worth

Security: Roving Event Security (Part-Time)Trail Drive Management Corporation Dickies Arena of Fort Worth, TX Dickies Arena is a 14,000 seat, spectacular multipurpose venue located adjacent to the Will Rogers Memorial Center campus in Fort Worth, Texas. The arena...