Azure data engineer (Pittsburgh) Job at Cognizant, Pittsburgh, PA

bjhia3I0akpVT1NQVDhmdWVLd2JZVmpUeUE9PQ==
  • Cognizant
  • Pittsburgh, PA

Job Description

Job Title - DataBricks Data Engineer

Location - Pittsburg, PA

**Must have 8+ years of real hands on experience**

We are specifically seeking a Data Engineer-Lead with strong expertise in Databricks development.

The role involves:

  • Building and testing data pipelines using Python/Scala on Databricks
  • Hands on experience to develop and lead the offshore team to perform development/testing work in Azure data bricks
  • Architect data platforms using Azure services such as Azure Data Factory (ADF), Azure Databricks (ADB), Azure SQL Database, and PySpark.
  • Collaborate with stakeholders to understand business needs and translate them into technical solutions.
  • Provide technical leadership and guidance to the data engineering team and need to perform development.
  • Familiar with Safe Agile concepts and good to have working experience in agile model.
  • Develop and maintain data pipelines for efficient data movement and transformation.
  • Onsite and offshore team communication and co-ordination.
  • Create and update the documentation to facilitate cross-training and troubleshooting
  • Hands on experience in scheduling tools like BMC control-M and setup jobs and test the schedules.
  • Understand the data models and schemas to support the development work and help in creation of tables in databricks
  • Proficiency in Azure Data Factory (ADF), Azure Databricks (ADB), SQL, NoSQL, PySpark, Power BI and other Azure data tools.
  • Implementing automated data validation frameworks such as Great Expectations or Deequ
  • Reconciling large-scale datasets
  • Ensuring data reliability across both batch and streaming processes

The ideal candidate will have hands-on experience with:

  • PySpark, Scala, Delta Lake, and Unity Catalog
  • Devops CI/CD automation
  • Cloud-native data services
  • Azure databricks/Oracle
  • BMC Control-M

Location: Pittsburgh, PA

Job Tags

Part time, Work experience placement,

Similar Jobs

SAVI EHS

Environment, Health and Safety Manager Job at SAVI EHS

 ...Job Description Environmental, Health and Safety Manager SAVI EHS Charleston, South Carolina (On-site) Compensation $45.00 $50.00 per hour $800 weekly per diem Employment Type Full-Time Job Description EHS Manager SAVI EHS is seeking an... 

Michael J Pietrusik DPM

Medical Receptionist Job at Michael J Pietrusik DPM

 ...MEDICAL RECEPTIONIST Michael J. Pietrusik DPM-268 W. Main St., Fredonia NY If you're a person who wants to help patients feel welcome, while supporting a successsful 41 year old podiatry practice, then this position may be for you! Job responsibilities:... 

A Hiring Company

Talent Acquisition Specialist Job at A Hiring Company

 ..., you will be responsible for managing the full life cycle of recruitment for our manufacturing company, leveraging ICIMS as our applicant...  ...qualified candidates through various channels, including online job boards, social media platforms, professional networks, and... 

The Pivot Group Network

HR Manager - Mattawan Area Job at The Pivot Group Network

 ...Job Description Job Description HR Manager | Mattawan, MI Salary Range: $81,000 - $126,000 depending on experience Are you a dynamic leader with a passion for fostering positive workplace environments and driving organizational success? The Pivot Group... 

Solid Solutions Canada Inc

Hybrid Recruiter Job at Solid Solutions Canada Inc

 ...Hybrid Recruiter We are seeking a motivated and dynamic Hybrid Recruiter to join our team. This role is ideal for an enthusiastic...  ...recruitment strategies for various roles. - Source candidates through online platforms, networking events, and job fairs to build a strong...