Democratizing Data and AI: Insights from Databricks' Youssef Mrini

Automatic transcript of the recording (English); speaker labels are automatic. Timestamps open YouTube at that moment.

0:00Sacha: So today I'm joined by Youssef, a senior solutions engineer at Databricks with over three years of experience at company Youssef that's been instrumental in helping customers adopt and derive value from the Databricks platform. He has received multiple MVP awards for his work in customer success and solutions engineering and has expertise in areas such as LLMs Gen AI in AWS. Youssef, it's a pleasure to have you with me today. Welcome.

0:30Guest: Pleased to meet you as well, Sasha.

0:32Sacha: Youssef, Databricks talks a lot about data and AI democratization. Can you explain what data and AI democratization means in practical terms for business today?

0:45Guest: Yeah. So just first of all, like everything I'm going to say, it's my point of view. It doesn't reflect, of course, what the company is saying and does for information. So I think data democratization is a very, very, very interesting or very good topic because like for the last couple of years, we've been, uh, searching several tools like with the modern data stack would be like searching tools. different ones. And it was good for technical force, not that much for the business or non-technical folks. And that was a big problem because the data was scoped only for a subset of people while it would be good if everyone can benefit from it. And I think this part of the, the LLM, I think it did change a lot, uh, this vision because now you've seen that everyone can start, I don't know, uh, running queries or asking just simple questions.

1:47And then those questions are, uh, uh, uh, uh, SQL queries or now like English is the new language. You just need to write your prompt and it will do a lot of magic for you. Of course, we know that LLMs alone won't be able to solve this actually for now. You still need some human interaction over there because, uh, it can hallucinate, can give some, some awkward queries. But yeah, and I think Databricks has invested a lot on it lately. I don't know if you're familiar with the concept of data intelligence platform, but yeah, I think that was a game for me. That was a game changer. And just to give some, uh, like an explanation to the folks who were watching, Databricks was like initially known for the lake house. So lake house, just combination of two concepts, the best of data warehouses and the best of data leaks. And that was good because we were like pioneering on this area.

2:50We created the lake house and then it was adopted by most of 500 fortunes. But then, uh, I think it was two years ago, maybe, or one year and a half, we did the acquisition of a company called Mosaic ML. So they had this, uh, LLM technology. So what, what we did is we combined those two, uh, like the lake house with the LLMs power. And we created a new category called data intelligence platform. And the goal was, as you mentioned, Sasha, to democratize access to data and AI, like to find data easily, uh, having, uh, uh, comments, metadata to help you understand, but also introduce something lately called Genie or AI BI Genie, where you can just ask simple questions. And in, and you will have the answer, like just your CEO, you just open a web app. I don't know. And you ask questions, what are the, I don't know, uh, the, uh, amount of sales we did for in the last 24 hours, which products have been selling, which is the most, I don't know, uh, the most influential, uh, seller in my company.

4:03And you no longer need, I mean, we want to replace, of course, data analysts, but it can answer some questions quickly without having this, um, uh, human feedback, like human interaction being there. Of course, we need, we still have to work to do.

4:19Sacha: Yeah, that's really powerful. Can you give me some examples of the biggest challenges you've seen, um, companies face when they tried to democratize access to big line AI?

4:31Guest: I think the, the, the biggest problem may be around governance. And I think also this is a topic that Databricks tried to address. And I believe it did address successfully with unity catalog. So, and I think this is very, very important because as the company is growing, then you need to make sure that you're not giving access to some, I don't know, PII data to an entity that is not supposed to access this data. So to make sure, and also not only accessing the data, but also making sure that, I don't know, a data analyst may not update some specific tables in production or, and this still was hard to do. With, with, with unity catalog, we were able to do this with simple operations. Like you can do it with Terraform, you can grant permission with the, within the UI, within the SQL stuff, which means we're really democratizing where anyone can give the permissions in a very efficient and easy way.

5:32And the governance is the main blocker. I think that's, that's the main, that's what's, and specifically now with, you know, companies are growing so much, a lot of business units, a lot of PII data. So it's really hard to, to handle those kinds of topics.

5:50Sacha: Why, why was it hard before unity catalogs for companies to govern their data?

5:57Guest: I mean, if I speak in the context of, uh, Databricks. So before we had ACLs by those ACLs were on force through the, the compute, which means you need to, to say, okay, with this compute, you can't access this specific table. But then you would be limited because if someone has the permission to create a new compute, then you can simply bypass those ACLs. So you were obliged to restrict the, or the, uh, cluster creation. And if you have clusters there, then you need to set up ACLs. And it was a little bit technical, required some tweaking. While now it's very easy, just as I mentioned, SQL UI scripting, and you're good to go. So like helping all the customers, like whether they are the, they're familiar with SQL or UI, or if they want more complex stuff with scripting like Terraform, then they are also served.

7:01Sacha: And you, you touched on the topic of compute. And why is compute infrastructure so critical for, um, AI success? Because I know Databricks has a bit of a different approach. They want to move, you know, data from compute and they say, let the best engine win. So how does Databricks approach compute to really enable a scalable and efficient AI workflow?

7:29Guest: I think we did it successfully because first of all, like the engine we're using is spark. And spark is open source. So you don't need to look for some specific, uh, I don't know, skills in the market. I don't know where you're scoped to something where the data is, um, is like written in a specific, uh, I don't know, framework, which is, can be proprietary. So Databricks introduced since the beginning, this, um, open source, uh, frameworks. Like we can talk about Delta Lake, uh, which means you can store your data by an open source. So, um, that's not much, which means if you want to use it in another, I don't know, technology, you can use the same data. You're not locked in same thing for the engine. Again, it's the spark. So whether you're using Databricks, whether you're using EMR or I know previously synapse or any other tool, you are using the same spark. And again, also, if you're running your spark workloads locally. And then like this, they mentioned about this ML flow integration for the AI, which is a framework end to end to make sure that you're developing.

8:38Like to make sure to track everything related to the ML models, like parameters, reproducibility, uh, governance as well later with the, with the, with UC. And then we started like, um, the last thing with last thing, this, I would say this decoupling the client from the server. And this is something that has been introduced with what we call spark connect. Now, which means for example, you can. Use your computer to develop and you can connect to a spark server remotely and run your compute over there. And that was super, super, uh, that was a good move because before it was, let's say they was tightly coupled, which caused some problem. Now it's, uh, much easier. And it was introduced with shared architecture and we've seen a tremendous adoption. If you go just to the real spark Apache spark repo, you've seen a lot of people like spark. First of all, it's successful project and spark connect has been massively, uh, used by users. And again, for the compute. It's it's, it uses the underlying compute, whether you're using, uh, AC to Azure VMS or G or using like, uh, uh, GCP.

9:49Sacha: We've talked a little bit about, um, Jenny. I, and you mentioned in the acquisition of mosaic and I want to go back to that topic a little bit. So, um, how do you view you personally view the rise of, sorry, of Jenny. I, and LMS and what opportunities in China do you see in the present moment, uh, for organizations?

10:15Guest: I think it was, um, it was a good move because first of all, like Navin was, uh, the, uh, founder of, uh, mosaic ML, uh, has the same vision as, uh, data break CEO with Ali Goetze. And then, uh, they shared the same vision, which means like when this acquisition, uh, was completed, it was totally aligned. And we've seen like how fast we did integrate with, uh, mosaic ML. First, we started integrating the MPT, uh, their MPT model. Then few, uh, few months later, we released our first, uh, open source LLM called DbRex. I have the t-shirt, but I, I did forgot to wear it, but you, you can see a Rex there. And now we're working, uh, I think that the, the team, uh, is working on the integration with the LLM models. I've seen like the latest one. So they're appearing together to make sure they are delivering the best experience to, to the customers. But then again, like related to the challenges, I think as all the folks, like everyone is experiencing.

11:24To make, to build the best fine tune model for their, for their use case or to train or to have a rag application. But again, it's all about, for example, the success of the rag application may, may be based, for example, on the checking tech, on the checking strategy, strategy of checking your following. Maybe you need to check by whenever you have, I don't know, return to new line or maybe by after a dot. And sometimes it's hard because there, this is not straightforward or say, okay, if I do this, then it will work. It's all about testing. So we need to keep testing. And so Databricks is providing all the tools. So we have our own vector search. Of course, if you want to use Pinecone, you can, you can use the database vector search. We have the playground for now. Test all your different, the different models. We connect to the open source model, as well as the proprietary ones of like open AI, on Tropic or whatever, whatever model. And then same thing, like putting some like guardrails, make sure that models may not do something it was not supposed to do.

12:29Again, handling everything through Unity Catalog with the governance, like track, for example, okay, Sasha can only use, I don't know, maximum 1000 tokens per day. So you can enforce this, you can track how many tokens Sasha use per day, do this. So we have the technologies. And again, it's on the customers to test. And then I think after a few months, I think everyone will be mature enough to have some specific frameworks, but the tools are here. And then you just need to do your magic to build something awesome over it.

13:05Sacha: And you touched upon it a little bit with basically ethical guardrails. And I want to dive a little bit deeper, and I know Databricks does quite a bit around that as well. How does the Databricks platform address the ethical considerations surrounding AI, like bias, fairness within the platform? So yeah,

13:27Guest: to be honest, I can tell you because I'm not an expert in this area, in the GNI area, I have some knowledge, but how it can be addressed. The only thing I know is we have some technology called guardrails, where, for example, you can specify a list of topics that cannot be, that will be, let's say, filtered, say, okay, we cannot do the answers. Then we have the default one, for example, if I ask a question, how can I rob a bank? Then you have, okay, we can't help you with this specific topic. And then you can specify the topics, you can whitelist. But how I think, regarding the question you ask, it's on the models that will be used, because we're not developing now, we have DBRX. But for the others, like if you are using LAMA, then it's on like the data that LAMA has used. But on your side, with guardrails, you can fix the limit. What are the subjects you can discuss? What are the subjects you can filter?

14:31Sacha: And then, before this question, you were talking about fine-tuning models on database. I think yesterday I read a publication by Matei, which was talking about compound AI systems, which he seems to point that this would be the future. And so to recap a little bit, what it says is that, for example, OpenAI with these huge models, huge global models, won't be like perfect for all use cases. And he sees a future where there is not one model that really wins the competition, but in which you actually go and produce your own model on your own data and have a lot of small AI systems. I want to get your take on this and how do you see it being fascinated by DBRX?

15:30Guest: I think I can like Matei's vision is, I think what Matei says is the right thing. Again, he's more experienced than maybe any of us over here. But the thing I can add like open like compound AI system, I believe it's really, really the future because it tackle as I mentioned tasks by by by combining those multiple interacting components. So it's really different from this monolithic AI models, which will rely, for example, on a single model to perform all the tasks. So you can like have this compound AI system that will use various models and traditional technologies to enhance furthermore the accuracy and the efficiency. Of course, you can use tuned models, you can use some agents, some retrieval mechanism. But the end, like the goal is really to to leverage the strengths of the different components and to make sure that you achieve the better performance while having this flexibility.

16:40And yes, like you can't have one model being perfect at everything. So that's why what I mentioned like previously, like you need to test. Regarding your topic with the different models you're going to have, and that's the that's the goal of play playground. So we can open multiple multiple playgrounds, select different models. You can select that on DBRX, you can select LAMA, LAMA 3.1. You can open, you can select OpenAI on Tropic and then you can ask the same question and it will be sent to different models and you can compare the results like the number of tokens, the velocity and the accuracy of the response and choose which one will mix will make will help you most. So in your opinion, say, OK, maybe for translation LAMA is better than I will go for LAMA. If it's coding, then I will use Cloud because it gives me and then you can have this logic. So by combining multiple models.

17:40Sacha: That's awesome. In your opinion, what are the most exciting trends like in the future and for developments in AI that business should be paying attention to? it.

17:51Guest: I think the first thing, like if I take my example, like I've been using Databricks Assistant since it was released for the first time, I think maybe 14 months ago. It saved me a lot of time because like, you know, sometimes you don't know the syntax very well. So you start coding, you do a mistake and then it automatically say, hey, you need to fix this. that's how you can do it of course it's not In fact, sometimes it may suggest the wrong thing, but most of the time it suggests the good thing. And then it did improve, let's say, my efficiency. So as a developer, like Databricks Assistant did help me go further, avoid avoiding wasting time. And I think the other the other customers can do the same. Like easy thing can be maybe, for example, with just some RAC application. I've seen some some customers and some friends working for companies that build some RAC application. And for example, they just need to upload the document rather than reading a 150 line PDF, 50, 50 page PDF.

19:00They just upload this and then they start asking questions. OK, can you tell me what should I do? Which what are the most important thing I should know on this area? And then it starts answering. And it's really speed in the process because rather than reading those 150 page and maybe you're going to forget some stuff, you need to take some notes, take some time. Just easy. You can, for example, I don't know, maybe your company, you're you're applying to win some some some maybe a market. You can just summarize everything like take the the large offer, upload this, say, OK, what are the things I should focus on? And then you have the list and just go focus on this rather than reading everything. And then say, OK, I have a lot of things that I don't need to consider. You know, it's really increasing. It's going to be increasing the product productivity can be also something. For example, if you are a call center and you you're or maybe not a call center, let's say your company selling, I don't know, a product and then the customer sent tickets.

20:04Then these tickets can be just sent to the application. It can be translated, classified, whether it's a product issue, system issue or it's hardware. Then automatically how angry is this customer? If this customer is angry, then trigger, I don't know, an agent or like a person who's going to interact with this person or just send the right documentation, say, OK, you need to check page number 14. This is how you can reset the system and can increase the customer satisfaction, which means more sales later. Like you can think about like 100,000 of of application. And I think it's just a big game. We are in a new and new era where we're going to see a lot of things changing and made the main examples, as you mentioned, democratizing access to data.

20:55Sacha: You said thank you so much for joining me today. Do you have any last word that you'd like to share with the audience before closing this episode?

21:05Guest: Yeah, I mean, what I would say that the most important thing will be to like always keep learning. I think this is something important. You will always have new technologies, new things to learn. Just keep in mind that you need to learn something that would help you, because if you try to learn everything, that's that's impossibly going to be overwhelmed. So make sure to choose one topic and deep dive and something important. Always make sure to learn new or say open source technologies. I think this is very, very important. So you can be very generic like things like, I don't know, open source like Apache Spark or when you learn like, I don't know, Delta Lake or Apache Iceberg or like open source technologies. You are really the future. And then once you have this good layer of information knowledge or also forgot to mention something very important.

22:12And LinkedIn, if you want to do, you should not build your own, I don't know, RAG application. And then you can learn about like SAS can be data breaks, can be any other, any other company and make sure to practice. I think this is something, something important. Reading blogs is good, but practicing is even better. And one more thing. And I think if you want to learn a lot of things, I think we have in the website data and AI summit, you can, you will see all the deep dive sessions like data break session, but also customer session to learn how data breaks help them. And from the, from the customer and also learn new things. I think this is also a good place to learn for, for free because those sessions are accessible for until the next data and AI summit.

23:05Sacha: Yeah, I fully agree. I've, I've been on that, on that page and I can tell that it's very useful. Yousaf, thank you so much for joining me. It was a, it was truly a pleasure and an honor to everyone today.

23:16Guest: Thank you. Thank you for, for the invitation, Sasha.