How does a datalake look like?¶
It is recommended to structure your tables into the following layers:
- bronze - "staging layer", raw data from source systems
- silver - most business logic, one or multiple tables per use-case
- parsed - data loaded from bronze layer to be stored as datalake tables
- cleansed - data cleaning, source systems bug fixing, ...
- Smart Data Model (SDM) - fully prepared data for further analytical and machine-learning use-cases
- gold - additional filtering/aggregations of silver data (using views or materialized tables) to be served to the final customers.
- Feature Store - central place where customer/product/... features are stored, managed and governed within the organization.
- reporting marts - pre-aggregated views of the data suitable for reporting and visualizations.