SmartCat @ London - Databricks Night!

​Scalable Data Intelligence with Databricks
Nemanja Milićević, Lead Data Engineer @ Smartcat.io
In this session we walk through our migration from an on-prem Spark cluster to a Databricks Data Intelligence Platform that now delivers real-time insight to millions users across South America, Europe, and Africa.  You will learn how we:

• ​Designed a Structured Streaming pipelines that reduces data latency from hours to seconds
• ​Applied Unity Catalog to standardise governance, lineage, and fine-grained access control
• ​Iterated over 10 ML prototypes and shipped two recommendation models using MLflow and Databricks Workflows

‍