Job Responsibilities:

• Requirement analysis, requirement gathering, design, business impact analysis, gap analysis, estimation, development, testing, coding, code review, unit testing and deployment of the application.
• Design, build, install, configure, and support Hadoop (Big Data). 
• Design and implement big data pipelines to ingest and process data in real-time and monitor for data miss or error in progress to ensure that data reaches the end system.
•   Measure the performance of various APIs and implement optimization on slow services to enhance the responsiveness of the system. 
• Deliver functional enhancements to the existing big data applications. 
• Provide inputs on solution architecture based on evaluation/understanding of solution alternatives, frameworks, and products. 
• Develop Hadoop batch jobs for data extraction from multiple unstructured and structured data sources for populating into various repositories (Hadoop, Neo4J, MongoDB, Apache SOLR). 
• Interact with clients to elicit architectural and non-functional requirements such as performance, scalability, reliability, availability, maintainability. 
• Participate in designing the data model structured data in RDBMS into a graph database solution and NoSQL database solution. 
• Develop near real-time data processing solutions using Kafka and Spark Streaming. 
• Participate in designing Spark architecture with Databricks and Structured Streaming.
• Set up Microsoft Azure with Databricks, and Databricks Workspace for business analytics. 
• Contribute to the automate build process by using Jenkins and Ansible to achieve continuous integration and continuous deployment.

Requirements:

Master’s in Computer Science/Applications, Information Technology/Systems or Electronics/Electrical Engineering + minimum 1 year experience as Big Data Engineer, Data Engineer, Data Engineering Specialist, Data Warehousing Analyst or related occupation.

Must be willing to travel/relocate to anywhere in the US.

Edison, NJ and unanticipated locations throughout US