Senior Data Engineer
What you'll do...
About the team:
The mission of Walmart Display Ads is to connect brand owners and shoppers with relevant display ads. We bring brand awareness and rich product information to shoppers and help brand owners grow their business with advanced targeting and optimization techniques. This is a fast-growing business never lacking opportunities! We are looking for an experienced tech lead to spearhead advertising data analytics related to demand, supply, and overall marketplace health. You will be responsible for extracting meaningful insights about campaign performance, marketplace efficiency, and gaps in systems and products, surfacing marketplace health metrics via dashboards, and working with cross-functional partners to move the needle in display advertising.
What you will do:
Lead and participate in small to large sized projects by reviewing project requirements; gather requested information; write and develop code; conduct unit testing; communicate status and issues to team members and stakeholders; collaborate with project team and cross functional teams; troubleshoot open issues and bug-fixes; and ensure on-time delivery and hand-offs.
Lead assigned projects by providing pertinent design, documents; identifying short- and long-term solutions and timeline; reviewing and providing feedback for proposed solutions; and performing design and code reviews of changes.
Provide guidance and mentorship to junior data.
Work closely with cross functional teams, including data scientists, product managers, and software engineers, to carry out various experiments across different areas of the AdTech platforms to improve customer experience and ad relevance.
Develop and maintain highly scalable and fault-tolerant data systems that process, store, and serve large volumes of data including online and offline user activities, impressions, and sales data.
Build and maintain data pipelines and ETL processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
Ensure data ingested and processed is accurate and of high quality by implementing data quality checks, data validation, and data cleaning processes.
Implement and enhance machine learning pipelines for model evaluation and promotion.
What you'll bring:
Bachelor’s degree in computer science or a related field.
5+ years of experience in the following areas in a Data Engineer role.
Expertise in one or more programming languages (Python, Java etc.)
Advanced Data Management: Proficiency in managing and manipulating huge datasets in the order of terabytes (TB) is essential. This includes knowledge of distributed file systems, data partitioning, and optimization techniques for handling large-scale data.
Big Data Technologies: Expertise in big data technologies like Hadoop, Apache Spark, Apache Hive, or similar frameworks is crucial for efficiently processing and analyzing massive volumes of data.
Data Integration: Skill in integrating data from multiple data sources is vital. This involves understanding various data formats, APIs, and protocols to effectively bring data together from diverse sources.
Data Modeling: Strong data modeling skills are necessary for designing a schema that can accommodate the evolution of data sources and facilitate seamless data joins across various datasets.
Database Management: Experience in working with large-scale databases is essential to handle the storage and retrieval of vast amounts of data efficiently.
Data Cleansing and Transformation: The ability to clean and transform data from different sources into a standardized format is crucial for accurate data analysis and effective data joins.
Query Optimization: Proficiency in optimizing queries to achieve better performance and reduce processing times is important when dealing with massive datasets.
Distributed Computing: Understanding the principles of distributed computing is vital for effectively utilizing resources across a cluster of machines to process and analyze large volumes of data.
Data Security and Privacy: Knowledge of data security and privacy best practices is essential to protect sensitive information when dealing with massive datasets from multiple sources.
Problem-Solving and Troubleshooting: Strong analytical and problem-solving skills are crucial for identifying and resolving issues that may arise during the data integration and schema evolution process.
Ability to effectively interpret technical and business objectives and challenges and articulate solutions.
Strong analytical skills and programming skills.
Technologies: Spark/Hive, Kafka, Apache Parquet, Avro, GCS, GCP, BigQuery, Python, Airflow
Nice to have:
Experience in the advertising domain
Experience in Machine Learning
About Walmart Global Tech:
Imagine working in an environment where one line of code can make life easier for hundreds of millions of people. That is what we do at Walmart Global Tech. We are a team of software engineers, data scientists, cybersecurity expert's and service professionals within the world’s leading retailer who make an epic impact and are at the forefront of the next retail disruption. People are why we innovate, and people power our innovations. We are people-led and tech-empowered. We train our team in the skillsets of the future and bring in experts like you to help us grow. We have roles for those chasing their first opportunity and those looking for the opportunity to define their career. Here, you can kickstart a distinguished career in tech, gain new skills and experience for virtually every industry, or leverage your expertise to innovate at scale, impact millions and reimagine the future of retail.
Flexible, hybrid work:
We use a hybrid way of working that is primarily in the office coupled with virtual when not onsite. Our campuses serve as a hub to enhance collaboration, bring us together for purpose and deliver on business needs. This approach helps us make quicker decisions, remove location barriers across our global team and be more flexible in our personal lives.
Benefits: Beyond our great compensation package, you can receive incentive awards for your performance. Other great perks include 401(k) match, stock purchase plan, paid maternity and parental leave, PTO, multiple health plans, and much more.
Equal Opportunity Employer:
Walmart, Inc. is an Equal Opportunity Employer – By Choice. We believe we are best equipped to help our associates, customers, and the communities we serve live better when we really know them. That means understanding, respecting and valuing diversity- unique styles, experiences, identities, ideas, and opinions – while being inclusive of all people.
The above information has been designed to indicate the general nature and level of work performed in the role. It is not designed to contain or be interpreted as a comprehensive inventory of all responsibilities and qualifications required of employees assigned to this job. The full Job Description can be made available as part of the hiring process.
Outlined below are the required minimum qualifications for this position. If none are listed, there are no minimum qualifications.Option 1: Bachelor’s degree in Computer Science and 3 years' experience in software engineering or related field. Option 2: 5 years’ experience in
software engineering or related field. Option 3: Master's degree in Computer Science and 1 year’s experience in software engineering or related
2 years' experience in data engineering, database engineering, business intelligence, or business analytics.
Outlined below are the optional preferred qualifications for this position. If none are listed, there are no preferred qualifications.Data engineering, database engineering, business intelligence, or business analytics, ETL tools and working with large data sets in the cloud, Master’s degree in Computer Science or related field and 3 years' experience in software engineering