Data + AI Summit Reflection

👋 Hi, this is Sarah with the weekly issue of the Dutch Engineer Newsletter. In this newsletter, I cover data and machine learning concepts, code best practices, and career advice that will help you accelerate your career.

---

North Moscone Center Building

This year’s Data and AI Summit was held in San Francisco, CA. I had never been to San Francisco, and here is the overview of how I would describe this week.

The food was delicious.

Panang Lamb Curry at Lao Table (highly recommend!)

The weather was windy and overcast at the beginning but cleared to blue skies for the latter half. A steady 60 degrees for the entire week. But most importantly, the people were amazing.

I want to give a few shoutouts, specifically to Maria and Başak from

Marvelous MLOps Substack

, Zach Wilson from

Zach Wilson

, Adi Polak who is the author of Scaling Machine Learning with Spark: Distributed ML with MLlib, TensorFlow, and PyTorch, Mark Freeman from

Scaling DataOps Newsletter

and

Data Quality Camp

.

During the summit, a ton of announcements were made during the past week that I will cover individually over the next couple of weeks. The main open-source announcements were:

1. Delta Lake 3.0 came out with a new universal table format, dubbed UniForm.

2. In Spark 3.4, Spark Connect introduces a decoupled client-server architecture that enables remote connectivity to Spark clusters from any application, running anywhere.

3. English as the New Programming Language for Apache Spark

See you all next week!

The Ferry Building at Port of San Francisco

Thanks to Delta for sponsoring this newsletter! I am a huge fan of Delta Lake and use it every day both in Data Engineering and Machine Learning.