Why do 72% of data-driven projects fail? Let’s use a human-centered example. Say a person intent on gaining muscle commits to lifting weights at the gym regularly for a year. If they saw little to no results after that year, would you blame the weights–or would you bet the problem lay in their form?
At Product World this week, our goal was to start a conversation around the importance of using the right methods to turn data into actionable insights–in other words, how product companies are lifting their data weights to build out their products. And I was glad to find that many other product companies in attendance are starting to have this conversation in earnest.
In my 20 years working in product analytics, I’ve seen the advent of commercially accessible machine learning, bigger data funnels, and AI. As a result of these new technologies, most product companies today would describe themselves as “data-driven”–when much of the time and unbeknownst to them, they aren’t fully there yet.
Data-Driven Dubiousness
Any app company will call themselves data-driven; user data is the key to its success, and so they make sure to prioritize tactics like A/B testing. What does that look like when it’s not truly data-driven? Imagine a scenario in which an app company is A/B testing two color schemes–pink and blue–to learn more about what appeals to their users. After spending lots of time and money, the results show that the pink color scheme performs the best. So they do the work to design the app accordingly, only to watch user engagement drop dramatically. Everyone finds themselves scrambling to answer “why?”; departments from business development to engineering will pull charts and run queries, and share them in a group email that eventually becomes a days-long thread cluttered with disparate pieces of information. A meeting is called, and a few people have minutes to make a decision on what to do next.
During this whole process, nobody thought to look further. Turns out, at the same time this app company was running tests, the marketing department had launched a huge campaign to attract female users between the ages of 12 to 18.
This anecdote depicts what’s missing in today’s understanding of “data-driven.” Just like gaining muscle takes so much more than showing up at the gym to lift, being data-driven requires more sophisticated, purposeful effort. At Kubit, we believe the foundations of a truly data-driven culture lie in contextual data, data quality, and wholly collaborative decision-making.
If there’s one thing we took away from Product World this year, it’s that the desire for better data-based moves is there, but there’s a lack of clear direction as to how to make those foundations a reality within a business. (TechCrunch’s Ron Miller touched on this in a recent article.)
I addressed these three foundations at the event this year and wanted to share them here.
Look at Data in Context
In the days before data, business decisions were largely intuitive. Today, data drives the agile race to market dominance–yet many of these agile businesses are driving the data bus with only partial sight.
When data became more readily available, businesses participated in “data-dredging,” which the newspaper headline above represents. This practice presents data as statistically significant in order to create false positives–but correlation does not equal causation. That’s why data in context is so important.
It’s an easy pattern to fall into in today’s ecosystem of fast-paced mobile and consumer product development. Businesses are consumed with looking at all the data they have, making it difficult to accurately navigate product shifts.
To really be “data-driven,” businesses should focus on looking at the right data for their product. That means shifting priority to the KPIs that matter most for each product, and looking at data in broader historical contexts–I call this benchmark analytics.
Additionally, being data-driven requires more than just leveraging data to identify a problem; companies must start applying smart data analytics–which we at Kubit call augmented analytics–to answer the “Now what?”, and push the data to do more, and take teams further into next steps.
Determine Data Quality
Imagine a CEO sends out a furious email saying a recent update has broken the company’s top-performing app, and she or he wants a solution–fast. This triggers a mad dash for data to figure out what’s going on–much like the A/B testing scenario I described earlier. After lots of double-work and siloed analysis, it’s decision-making time.
It’s tempting to cherry-pick the data that justifies the assessment of the highest-paid person in the room (and subsequently ignoring data that undermines it). I call this “data bullying,” and it forces businesses to ignore the very point of using data to drive intelligent business decisions: objective, “truthful” data. Once data quality is compromised, teams are working with half-truths, and that can lead to expensive mistakes.
To become a business that’s actually data-driven, make sure people are asking the right questions to ensure the data was handled correctly. Using only trustworthy data is something we learned in school (Wikipedia is not a reliable source), and it applies to analytics, as well. Data must be clean and make sense to be worthy of basing decisions off of it.
The irony here is that the more you build on a data-driven culture, the bigger and more successful your business can become–making data quality and control maintenance more of a challenge.
That’s why Kubit designed workflow processes around organization and transparency. Instead of wading through several versions of reports using various sources of data, Kubit’s Investigation Board guides the data-driven process in a way that puts everyone on the same page, and everything can be traced to a single source of truth.
Widen the Decision-Making Circle
It’s not entirely accurate to describe a company culture as “data-driven” if the information gathering and decision making processes are in the hands of only a few people. (Outside of data analysis, this concept is growing in popularity as more research ties it to employee turnover and engagement.)
I’ve seen firsthand the benefits that come from providing more employees with access to data so they can proactively discover insights and come up with the next big idea. Crucial to this shift is creating a portal that’s usable for everyone–which is why Kubit has made self-serve data analytics possible through its user-friendly Query Builder (while also focusing on the issue of data quality, addressed previously).
Communication is a major part of building a data-driven culture, so with things like Slack and email, why is that still an issue? Because conversations on those platforms take place outside of the data and are often mixed in with conversations about other issues. Centralizing communications in the same place as the data will help teams avoid the common pitfall of doing parallel or redundant work, and makes searchable, organized input easy. This need for aligned, data-based ideation is why Kubit made its collaborative communication tool a part of its platform–because getting lost in email threads is not collaborative communication.
I’ll end this post with one final observation coming out of Product World: there is no shortage of project management, data, and AI software out there. But just like finding the right data for each product, every company must find the right software for them. That doesn’t mean more bells and whistles–it means finding the software that will help teams close the gaps between each other, between themselves and their product users, and between an ever-growing set of data sources.
Despite the competition, I feel we’re on the right side of history; Kubit’s team has brought their decades of collective experience working in product analytics to address the pain points that so many companies are still experiencing with data–and I look forward to seeing the competitive heights our users will soar to.
