Web Scraper (Python Data Engineer) Job (Remote)

Key responsibilities:

1. Work on data sourcing
2. Use web scrapers (Beautifulsoup, selenium, etc.)
3. Manage the data normalization and standards validation
4. Parametrize and automate the scrapers
5. Develop and execute the processes for monitoring data sanity and checking for data availability and reliability
6. Understand the business drivers and build insights through data
7. Work with the stakeholders at all levels to establish current and ongoing data support and reporting needs
8. Ensure continuous data accuracy and recognize data discrepancies in systems that require immediate attention/escalation
9. Work and become an expert in the company s data warehouse and other data storage tools, understanding the definition, context, and proper use of all attributes and metrics
10. Create dashboards based on business requirements
11. Work on the distributed systems, scale, cloud, caching, CI/CD (continuous integration and deployment), distributed logging, data pipeline, REST API)

Who can apply:

1. Creativity & complex problem-solving skills
2. Exceptional and scalable web scraping skills
3. Passion and interest in doing ETL jobs
4. Good English speaking and communication skills
5. Ability to work with a global remote culture
6. Initiative and entrepreneurship skills
7. Experience with microservices architecture and writing REST APIs
8. Knowledge of Kubernetes, Docker and Airflow
9. Prior experience with Python, Django and Gunicorn
10. Independent work ethic with an ability to work in a fast-paced environment
11. We are looking for data engineers with Python scripting practices and scalable web scraping skills, including monitoring ingestion of data, adhering to data standards, and solid knowledge of data and cloud workflow orchestration

3