info@suntechpros.com
+91 919-439-9037
Job Duties:
Perform root cause analysis and resolve production issues related to ETL processes, ensuring high data integrity across all systems. Validate all data processing activities through comprehensive testing, ensuring data accuracy, consistency, and compliance with business requirements.
Design, implement, and maintain ETL pipelines using Informatica PowerCenter, PowerExchange, MDM, IICS/IDMC, and other relevant technologies and processes such as Big Data, Netezza, ServiceNow, DataStage, Data Wrangling and Transformation, Gitlab, Urban Code Deploy, HP Quality Control Center, HP Application Lifecycle Management, Putty, JIRA, Q-Test, to ensure efficient data extraction, transformation, and loading.
Analyze software defects, debug ETL processes, and translate solutions into detailed technical specifications for development teams.
Develop and optimize ETL workflows, mappings, sessions, and transformations, ensuring alignment with Low-Level Design (LLD) and Technical Design Document (TDD).
Establish and manage database connections across diverse systems (Oracle, DB2, Hadoop, Hive, etc.) by obtaining the necessary permissions and configurations from LDAP groups. Configure and execute Informatica mappings, sessions, and workflows to extract, transform, and load data from relational and non-relational sources.
Leverage Unix/Linux scripting to automate ETL jobs and create shell scripts for running Informatica workflows within a secure environment. Utilize scheduling tools (TWS, Autosys, Maestro) to automate and optimize workflow execution, ensuring minimal downtime and maximum efficiency.
Document all technical specifications, including system configurations, metadata management processes, and ETL logic, to facilitate smooth knowledge transfer and operational handover. Conduct unit, integration, and system tests to validate ETL functionality and performance, ensuring conformance to design specifications.
Collaborate closely with business analysts, data analysts, and end-users to gather and validate requirements for data integration and reporting solutions. Design and develop data warehouse models, ensuring the proper integration of metadata and alignment with business needs.
Lead the creation and maintenance of comprehensive reports, dashboards, and data visualizations using Visio and E-R modeler to monitor ETL performance and provide business insights. Perform data reconciliation and analysis on complex data sets, working with data analysts to resolve discrepancies and optimize data flows. Ensure compliance with SDLC best practices throughout the ETL lifecycle, from requirement gathering and design to testing, deployment, and production support. Develop and execute comprehensive test plans (Unit, Integration, Regression) for ETL processes, ensuring that all system changes are thoroughly validated.
Manage migration of ETL objects between development (DEV), quality assurance (QA), and production environments, ensuring smooth transitions and proper validation post-migration. Provide continuous support during QA User Acceptance Testing (UAT), troubleshooting issues and ensuring that test cycles meet the predefined requirements.
Regularly monitor and troubleshoot ETL job failures, analyzing root causes and ensuring that the necessary corrective actions are taken. Optimize and fine-tune Informatica objects and workflows to improve performance, reduce processing time, and maximize resource utilization.
Collaborate with metadata managers to ensure accurate metadata documentation and tracking, ensuring smooth data lineage and integration across systems. Leverage cloud technologies like Azure DataBricks and AWS to build scalable and robust data pipelines, integrating cloud-native capabilities into existing data workflows. Participate in architecture design sessions, providing input on scalability, performance optimization, and data integration strategies.
