Relatively new to the data and analytics market, I was excited and intrigued to meet up with colleagues, partners, and customers at the Data + AI Summit in San Francisco from June 11-13. I work as Director of Product Marketing for Kubit, a disruptor in the product analytics market segment and an ISV technology partner of Databricks. This was my first time at the conference, so I documented my impressions of the experience.
Worth noting off the bat, was Databricks’ ‘cool factor.’ Compared to my last 10 years in enterprise automation software, it was a refreshing change. The expo hall featured an Inclusion Center, pickleball court, and water bottle personalization table. The visual branding was bold, and the soundtrack, edgy and hip.
The physical size of the conference was impressive, not just to me but to any ‘brickster’ with any tenure. According to a Financial Services account rep I met in line at a coffee shop, the ‘data intelligence’ platform provider has grown from 700 employees just 5 years ago to more than 8,000 today. The event boasted over 500 sessions, tens of thousands of attendees, 146 sponsors, and four full days of programming.
Databricks’ co-founder and CEO, Ali Ghodsi, quantified the company’s growth further during the first day keynote, claiming that the Summit is the largest data and AI gathering in the world with 60,000 watching online and 16,000 in attendance. The event expanded into the entire Moscone Center this year, creating a buzz that spawned throughout San Francisco elevated by the company’s first day announcement that it is open sourcing its Unity Catalog, its recent acquisition of Tabular and the integration of last year’s acquisition of Mosaic which has fueled its preparedness for GenAI/LLM – THE topic in virtually every room.
According to Ghodsi, every company wants to be a data and AI company, but companies are challenged with realizing that goal. Here are a few of those top challenges and how I see Kubit is poised with Databricks to help our joint customers move in the right direction with those challenges.
The data state is fragmented
Perhaps the number one challenge acknowledged by Ghodsi is the fragmentation of the data state. Most enterprises maintain so many pieces of software, data and technology leaders don’t even know what they have. CDOs, in fact, often identify data fragmentation as their top challenge. In a fragmented environment, incomplete, overlapping data sets are rampant. Data silos and complexity pose a significant barrier to AI projects. Data integrity is essential for GenAI in transforming businesses. Without it models are inaccurate and potentially harmful to the company. Fragmentation can degrade even further as AI platforms are adopted.
“Stop giving your data to vendors,” advised Ghodsi, “own your own data.” These mantras are synonymous to Kubit’s fundamental approach to analytics. Our Databricks-native platform leverages an organization’s existing Lakehouse investment, so there is no need to move or replicate data. With Kubit, data never needs to leave the Lakehouse so a single source of truth can be realized and gen AI initiatives can be truly effective.
GenAI initiatives are not yet in production
Despite the high interest and budgetary support behind it, a whopping 85% of enterprise GenAI initiatives have not made it to production. Many companies are still learning how to move from excitement to successful implementation while others still struggle with GenAI adoption because they are still learning to integrate traditional AI tools into their operations.
GenAI’s complexity and specific use cases make it difficult to find viable business applications that are worth the investment. Additionally, the long-term effects and costs associated with GenAI, as well as potential regulatory impacts, are still unknown. Deciding which GenAI use cases will provide tangible business benefits is crucial but challenging. Unrealistic expectations regarding timelines, costs, and potential value can further hinder the successful adoption of GenAI projects.
No matter what you do with AI, whether still in testing mode, or in production with actual use cases like chatbots or checkout flows, organizational leaders will need to measure the impact of it and gather data. Analyzing that data is challenging but I’m very excited at the prospect of Kubit leading that charge and in effect, increasing companies’ efficacy of AI investments.
Security and privacy of AI
Ensuring data security is crucial for deploying dependable GenAI solutions and a top obstacle for GenAI project leaders to overcome. Even more, trusted data is vital for instilling confidence in GenAI.
Security and compliance are essential aspects of Kubit’s business. Since our platform seamlessly integrates with the Databricks Lakehouse, it provides users immediate access to real-time data without the need for data duplication. This eliminates data silos and provides customers with security and governance gained by realizing a single source of truth.
Databricks partnership perspective
The following days of the Summit were rewarding in many ways and much of it expanded on how to realize the promise of unprecedented possibilities for data and AI, yet the top highlight for my colleagues and me, was Kubit’s participation on the Startup Forum.
Kubit was asked to share our best practices on how best to scale and work with Databricks. Led by Databricks’ new Global Leader of Startup and Venture Program, Steve Sobel, executives from four leading startups including our own VP of GTM, Shepperd Power, explored the benefits, strategies and lessons learned from building the next generation of data and AI applications with Databricks.
When asked about how Databricks was targeted as a technology partner, panelist Scott Love, CEO of Lovelytics and this year’s Innovation and CME partner of the year, replied “We are hyper-focused on one technology and recognized that Databricks was faster, cheaper and had better functionality than legacy technologies. So, when our customers asked for a forward-looking strategy, we were sold on Databricks as our center of expertise for our services.”
“We are aligned on the value that we bring to the customer – TCO, data integrity, single source of truth, transparency,” added Kubit’s Shepperd Power. “At the end of the day, we have to deliver value to the customers together.”
