Building an End-to-End Open-Source Modern Data Platform for Biomedical Data
- Speaker
- Alberto Labarga
- Event
- PyDay BCN (2022)
- Date
- Type
- Workshop
- Language
- English / Spanish
- Topic
- Data engineering
- Length
- 75 minutes
About this talk
A detailed guide to help you navigate the modern data stack and build your own platform using open-source technologies.
Data engineering has experiences enormous growth in the last years, allowing for rapid progress and innovation as more people than ever are thinking about data resources and how to better leverage them.
In this talk we will explore the related technologies and build from scratch an end-to-end modern data platform for the analysis of medical data. We will be using open-source tools and libraries, including python-based DBT, Apache Airflow and Querybook.
The platform will consist of the following components:
- Data warehouse
- Data integration
- Data transformation
- Data orchestration
- Data visualization
Photos (1)
