Data Engineer
4 weken geleden
Position: Data Engineer
Job Type: Permanent
Location: Amsterdam, Netherlands
Role: As a Data Engineer you will be responsible for designing, building, and maintaining the infrastructure that enables data collection, storage, and analysis. They work closely with data scientists, analysts, and other engineers to ensure that large datasets are accessible, reliable, and properly structured. Data Engineers play a crucial role in the data pipeline, building systems to acquire, transform, and load (ETL) data for analytics, reporting, and machine learning applications.
Responsibilities:
1. Design and Build Data Pipelines:
- Develop, construct, test, and maintain data pipelines to extract, transform, and load (ETL) data from various sources to data warehouses or data lakes.
- Ensure data pipelines are efficient, scalable, and maintainable, enabling seamless data flow for downstream analysis and modeling.
- Work with stakeholders to identify data requirements and implement effective data processing solutions.
2. Data Integration:
- Integrate data from multiple sources such as internal databases, external APIs, third-party vendors, and flat files.
- Collaborate with business teams to understand data needs and ensure data is structured properly for reporting and analytics.
- Build and optimize data ingestion systems to handle both real-time and batch data processing.
3. Data Storage and Management:
- Design and manage data storage solutions (e.g., relational databases, NoSQL databases, data lakes, cloud storage) that support large-scale data processing.
- Implement best practices for data security, backup, and disaster recovery, ensuring that data is safe, recoverable, and complies with relevant regulations.
- Manage and optimize storage systems for scalability and cost efficiency.
4. Data Transformation:
- Develop data transformation logic to clean, enrich, and standardize raw data, ensuring it is suitable for analysis.
- Implement data transformation frameworks and tools, ensuring they work seamlessly across different data formats and sources.
- Ensure the accuracy and integrity of data as it is processed and stored.
5. Automation and Optimization:
- Automate repetitive tasks such as data extraction, transformation, and loading to improve pipeline efficiency.
- Optimize data processing workflows for performance, reducing processing time and resource consumption.
- Troubleshoot and resolve performance bottlenecks in data pipelines.
6. Collaboration with Data Teams:
- Work closely with Data Scientists, Analysts, and business teams to understand data requirements and ensure the correct data is available and accessible.
- Assist Data Scientists with preparing datasets for model training and deployment.
- Provide technical expertise and support to ensure the integrity and consistency of data across all projects.
7. Data Quality Assurance:
- Implement data validation checks to ensure data accuracy, completeness, and consistency throughout the pipeline.
- Develop and enforce data quality standards to detect and resolve data issues before they affect analysis or reporting.
- Monitor and improve data quality by identifying areas for improvement and implementing solutions.
8. Monitoring and Maintenance:
- Set up monitoring and logging for data pipelines to detect and alert for issues such as failures, data mismatches, or delays.
- Perform regular maintenance of data pipelines and storage systems to ensure optimal performance.
- Update and improve data systems as required, keeping up with evolving technology and business needs.
9. Documentation and Reporting:
- Document data pipeline designs, ETL processes, data schemas, and transformation logic for transparency and future reference.
- Create reports on the performance and status of data pipelines, identifying areas of improvement or potential issues.
- Provide guidance to other teams regarding the usage and structure of data systems.
10. Stay Updated with Technology Trends:
- Continuously evaluate and adopt new tools, technologies, and best practices in data engineering and big data systems.
- Participate in industry conferences, webinars, and training to stay current with emerging trends in data engineering and cloud computing.
-
Data Engineer
6 dagen geleden
Amsterdam, Noord-Holland, Nederland Data Talents VoltijdWil jij zelf de keuze hebben bij wat voor opdrachtgever je gaat werken en wil je uitgedaagd worden door complexe datavraagstukken? Als Data Engineer bij Data Talents heb je zelf de keuze welk project het beste past bij jouw ervaring en ambitie Wat ga je doen? Je gaat jouw expertise voor bepaalde tijd inbrengen bij één van onze opdrachtgevers en je gaat ze...
-
Expert in Data Engineering
6 dagen geleden
Amsterdam, Noord-Holland, Nederland Data Talents VoltijdBij Data Talents kun je terecht als je op zoek bent naar een kansrijke baan in de data technologie. We zijn op zoek naar getalenteerde Data Engineers die ons kunnen helpen om onze opdrachtgevers succesvol te maken.Wat je gaat doenAls Data Engineer bij Data Talents word je onderdeel van een team dat samenwerkt aan interessante projecten. Je bent...
-
Data Engineer
6 dagen geleden
Amsterdam, Noord-Holland, Nederland Data Talents VoltijdBij Data Talents vind je een uitdagende en interessante baan als Data Engineer. Je hebt de mogelijkheid om zelf te kiezen welk project het beste past bij jouw ervaring en ambitie.Wat ga je doen?Je gaat jouw expertise inbrengen bij één van onze opdrachtgevers en hen helpen met complexe data vraagstukken aan te pakken. Onze opdrachtgevers zijn grote...
-
Data Engineer
6 dagen geleden
Amsterdam, Noord-Holland, Nederland JSS ASSOCIATES VoltijdData Access Engineer (Unity Catalog & Databricks): For one of our globally leading tech clients we are urgently looking for an experienced Data Access Engineer (Unity Catalog & Databricks) for a long contract based in Amsterdam. Its paying well and my client can interview asapData Access Layer (Unity Catalog) Engineer: Create and maintain the Data Access...
-
Data Engineer
3 weken geleden
Amsterdam, Noord-Holland, Nederland Foxtek VoltijdWork Alongside Some Of The Most Experienced Data Engineers In The Netherlands This AI solutions and Data-driven business is looking for a Data Engineer to be a part of their elite team. Crafting excellence is ingrained in their processWhen a company has an AI / Data project that is too complex for their internal engineers, they call these guys. This means...
-
Data-Expert in Data Engineering
6 dagen geleden
Amsterdam, Noord-Holland, Nederland Veneficus Voltijd € 3.500 - € 6.000Wat doe je als Data Engineer?Bij Veneficus ga je aan de slag als Data Engineer en speel je een sleutelrol in het verzamelen, structureren en benutten van data. Je werkt samen met diverse sectoren, zoals finance, verzekeringen, retail, transport, overheid, logistiek en energie.Je taken:Opzetten van ETL-processen en bouwen van data pipelines;Onderwerpennen,...
-
Data Engineer
14 uur geleden
Amsterdam, Noord-Holland, Nederland Adyen VoltijdThis is Adyen Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice. At Adyen, everything we do is engineered for ambition. For our teams, we create an environment with opportunities for our people to succeed, backed by the...
-
Data Engineer
3 weken geleden
Amsterdam, Noord-Holland, Nederland Amoria Bond VoltijdData Engineer | Hybrid | Amsterdam | Contract | Finance Data Engineer, SQL, Python, Finance, Databricks Amoria Bond is seeking an experienced Data Engineer for a finance client based in Amsterdam. Location: Amsterdam (Hybrid) Start Date: ASAP Contract: 12 months Must have requirements: Circa 5 years Experience in similar Data Engineer role ...
-
Data Engineer
5 dagen geleden
Amsterdam, Noord-Holland, Nederland brabers VoltijdThe Data Engineering chapter consists of several scrum teams. We work on all kinds of goals in these teams. Keeping a close eye on progress is extremely important. And preventing or removing bottlenecks is part of that. This is how we increase the reliability of the grid. You work from a group of colleagues and together you further professionalize the...
-
Data Engineer
7 dagen geleden
Amsterdam, Noord-Holland, Nederland Oxygen Digital VoltijdData Engineer Salary - €65,000 - €80,000Location - Utrecht - (Hybrid working 2 days in office)Are you a talented Data Engineer with a passion for building scalable, innovative data solutions? Our client, a pioneering organisation based in the Netherlands, is developing a cutting-edge AI assistant service tailored to revolutionise the real estate...
-
Data Engineer
4 weken geleden
Amsterdam, Noord-Holland, Nederland InterEx Group VoltijdAbout: The Data Engineer extracts, combines, transforms, loads, manipulates and cleanses data all whilst effectively handling interfaces to large amounts of data and creates/integrates data structures suitable for analysis. The Data Engineer must understand data architecture and data management. Develop and deploys algorithms in performance-efficient code...
-
Data Engineer
3 weken geleden
Amsterdam, Noord-Holland, Nederland InterEx Group VoltijdAbout: The Data Engineer extracts, combines, transforms, loads, manipulates and cleanses data all whilst effectively handling interfaces to large amounts of data and creates/integrates data structures suitable for analysis. The Data Engineer must understand data architecture and data management. Develop and deploys algorithms in performance-efficient code in...
-
Data Engineer
3 dagen geleden
Amsterdam, Noord-Holland, Nederland InterEx Group VoltijdAbout: The Data Engineer extracts, combines, transforms, loads, manipulates and cleanses data all whilst effectively handling interfaces to large amounts of data and creates/integrates data structures suitable for analysis. The Data Engineer must understand data architecture and data management. Develop and deploys algorithms in performance-efficient code in...
-
Data Engineer
4 dagen geleden
Amsterdam, Noord-Holland, Nederland Maandag VoltijdData engineer in Amsterdam Over de functie Ben je klaar om Maandag verder te helpen naar een datagedreven toekomst en mee te bouwen aan ons nieuwe data platform in Microsoft Fabric? Maandag is een grote detacheergroep die professionals en opdrachtgevers zo goed en snel mogelijk met elkaar koppelt. Dat maakt dat we beschikken over grote hoeveelheden...
-
Data Engineer
3 dagen geleden
Amsterdam, Noord-Holland, Nederland Harnham VoltijdRole: Medior Data EngineerLocation: AmsterdamSalary: Up to 70kAbout the CompanyThis company are transforming education by delivering personalized, impactful learning experiences powered by advanced data systems. The mission is to create technology that helps learners succeed at every stage of their journey. As a growing EdTech start-up, they are looking for...
-
Data Engineer
2 weken geleden
Amsterdam, Noord-Holland, Nederland HCL Technologies B.V. Netherland VoltijdJob Role- Data Engineer Locatoin - Amsterdam, Netherlands. Job Description: Roles and responsibilities Configuration of new data sources. Implementation of data quality rules. Monitoring performance and incidents. Needed skills and experience a good communicator in English (verbal, written & presentation). 8 years or more of experience in data...
-
Data Engineer
3 dagen geleden
Amsterdam, Noord-Holland, Nederland HCL Technologies B.V. Netherland VoltijdJob Role- Data EngineerLocatoin - Amsterdam, Netherlands.Job Description:Roles and responsibilitiesConfiguration of new data sources.Implementation of data quality rules.Monitoring performance and incidents.Needed skills and experiencea good communicator in English (verbal, written & presentation).8 years or more of experience in data engineering, data...
-
Data engineer
14 uur geleden
Amsterdam, Noord-Holland, Nederland TOPdesk Voltijd32-40 Medior / senior functie HBO/WO 2800-6000 Delft Ben jij technisch georiënteerd, zorg je voor betrouwbare data en hou je van een fikse uitdaging? Ben je bovendien pragmatisch en maak je goed het onderscheid tussen het ideale plaatje en de realiteit? Dan ben jij de Data Engineer die we zoeken Wat ga je doen als Data Engineer? Als Data...
-
Data Engineer
1 week geleden
Amsterdam, Noord-Holland, Nederland DataOps House VoltijdJob Only for Dutch Speaker Ben jij een data engineer en ben je klaar om je carrière naar een hoger niveau te tillen? Zoek je een goede balans tussen technisch werk en consulting bij klanten? Krijg je energie van het ontwikkelen van oplossingen die in productie draaien? Je wilt end-to-end verantwoordelijk zijn. Dus van ontwikkeling tot monitoring en...
-
Data Engineer
4 weken geleden
Amsterdam, Noord-Holland, Nederland Tata Consultancy Services VoltijdAbout TCSTata Consultancy Services (TCS) is a global leader in IT services, digital, and business solutions that partners with its clients to simplify, strengthen, and transform their businesses. We ensure the highest levels of certainty and satisfaction through a deep-set commitment to our clients, comprehensive industry expertise, and a global network of...
