Python Developer
Tata Consultancy Services · Pune District, Maharashtra, India · Full Time · Onsite
Posted
Experience-6 to 8 Years Location: Chennai Job Summary: An experienced Python Developer to design, develop, and maintain large-scale data processing solutions using Apache Spark and Python. The ideal candidate should have a strong background in data engineering, data processing, and cloud-based data platforms, big data ecosystems and performance optimization Responsibilities: Data Processing: Design, develop, and maintain data processing solutions using Py Spark, Apache Spark, and Python.Data Pipeline Development: Develop and optimize data pipelines using Py Spark, Apache Spark, and cloud-based data platforms.Data Integration: Integrate data from various sources, including relational databases, No SQL databases, and cloud storage.Data Transformation: Develop and implement data transformation logic using Py Spark, Apache Spark, and Python.Collaboration: Work with cross-functional teams to identify and prioritize project requirements, provide technical guidance, and ensure data quality.Required Skills: Py Spark: In-depth knowledge of Py Spark, Apache Spark, and Python.Data Processing: Strong understanding of data processing concepts, including data ingestion, data transformation, and data storage.Cloud Experience: Experience with cloud-based data platforms, including AWS, Azure, or Google Cloud.Expertise in Data Frames & Spark SQL, Spark Streaming with Apache Kafka real-time data ingestion pipelines Very good conceptual understanding of Multithreading, distributed computing concepts of Pyspark Programming: Proficiency in programming languages, including Python, Java, or Scala.Communication: Excellent communication and collaboration skills.Preferred Skills: Apache Spark Certifications: Relevant certifications, such as Apache Spark Certification or Cloudera Certified Spark Developer.Big Data: Experience with big data technologies, including Hadoop, HBase, or Cassandra.Machine Learning: Experience with machine learning frameworks, including Tensor Flow, Py Torch, or Scikit-learn.Data Science: Experience with data science tools, including Jupyter Notebook, Pandas, or Num Py