Data Engineering Tech Lead Hadoop at PA Job at Lorven technologies, Pittsburgh, PA

bU1UbW80ZkNVZVdLUzhqcGZLOFRZVkxT
  • Lorven technologies
  • Pittsburgh, PA

Job Description

Hi

Role: Data Engineering Tech Lead Hadoop

Location: 1 st priority : Pittsburgh, PA

Location: 2 nd : Dallas, TX / Cleveland, OH, NYC Metro & Connecticut


Hybrid Role

Job description:

Must:

  1. Hadoop experience
  2. Led a small team or mentored jr developers(ideally, but not necessary)

We are seeking an accomplished Tech Lead Data Engineer to architect and drive the development of large-scale, high-performance data platforms supporting critical customer and transaction-based systems. The ideal candidate will have a strong background in data pipeline design , Hadoop ecosystem , and real-time data processing , with proven experience building data solutions that power digital products and decisioning platforms in a complex, regulated environment.

As a technical leader, you will guide a team of engineers to deliver scalable, secure, and reliable data solutions enabling advanced analytics, operational efficiency, and intelligent customer experiences.

Key Roles & Responsibilities

  • Lead and oversee the end-to-end design, implementation, and optimization of data pipelines supporting key customer onboarding, transaction, and decisioning workflows.
  • Architect and implement data ingestion, transformation, and storage frameworks leveraging Hadoop , Avro , and distributed data processing technologies.
  • Partner with product, analytics, and technology teams to translate business requirements into scalable data engineering solutions that enhance real-time data accessibility and reliability.
  • Provide technical leadership and mentorship to a team of data engineers, ensuring adherence to coding, performance, and data quality standards.
  • Design and implement robust data frameworks to support next-generation customer and business product launches.
  • Develop best practices for data governance , security , and compliance aligned with enterprise and regulatory requirements.
  • Drive optimization of existing data pipelines and workflows for improved efficiency, scalability, and maintainability.
  • Collaborate closely with analytics and risk modeling teams to ensure data readiness for predictive insights and strategic decision-making.
  • Evaluate and integrate emerging data technologies to future-proof the data platform and enhance performance.

Must-Have Skills

  • 8 10 years of experience in data engineering , with at least 2 3 years in a technical leadership role .
  • Strong expertise in the Hadoop ecosystem (HDFS, Hive, MapReduce, HBase, Pig, etc.).
  • Experience working with Avro , Parquet , or other serialization formats.
  • Proven ability to design and maintain ETL / ELT pipelines using tools such as Spark , Flink , Airflow , or NiFi .
  • Proficiency in Python , Scala for large-scale data processing.
  • Strong understanding of data modeling , data warehousing , and data lake architectures.
  • Hands-on experience with SQL and both relational and NoSQL data stores.
  • Cloud data platform experience with AWS .
  • Deep understanding of data security , compliance , and governance frameworks .
  • Excellent problem-solving, communication, and leadership skills.

Job Tags

Full time,

Similar Jobs

Grammarly

Principal Product Designer, Core (San Francisco) Job at Grammarly

 ...Companies in AI, and one of Inc.s Best Workplaces. The Opportunity To achieve our ambitious goals, were looking for a Principal Product Designer to join our Core Design team. This role will lead the work of critical new product offerings, mentor and inspire the broader... 

Remote Jobs

Senior Project Manager - Recruiting Strategy & Operations : 25-07216 (San Francisco) Job at Remote Jobs

 ...excellence and a positive work environment Engage in meaningful projects that enhance recruitment efficiency and customer experience...  ...is Required (Qualifications) Demonstrated project/program management expertise Strong analytical, organizational, and communication... 

Hamilton Plumbing, Heating, A/C, Rooter

Plumbing Field Supervisor Job at Hamilton Plumbing, Heating, A/C, Rooter

 ...Job Description Company Name: Hamilton Plumbing, Heating, A/C, Rooter Overview: Pay: $75,000 - $100,000/ year based on experience Join Hamilton Plumbing, Heating, A/C, Rooter of ARS, the nations largest provider of residential HVAC, plumbing, and electrical services... 

RIVERSIDE ADVENTURE CO

Coffee Shop Manager at RIVERSIDE ADVENTURE CO Job at RIVERSIDE ADVENTURE CO

 ...Description Assist in all aspects of the day-to-day operations of the coffee shop. Lead by example in regards to giving excellent service to...  ...Adventure Co. operations (Delphina, Wisp, etc.) Assist the manager in providing comprehensive training for all new hires, as well... 

Dropbox

Staff Fullstack Software Engineer, Growth Monetization Job at Dropbox

 ...team within Core Growth Engineering, you will play a crucial role in optimizing how millions of users discover, trial, and purchase Dropbox products. You'll lead technical strategy for our checkout, trial, conversion, and subscription systemsdirectly impacting revenue...