Why is progress still difficult to achieve despite companies continuing to invest in data and AI initiatives, while increasing investment rates? One answer may lie in the way companies see the biggest barriers and obstacles to progress with their data and AI initiatives. This revolves around cultural change and transformation. According to this year’s survey, the company’s executives report with an overwhelming rate – 92.2% – that the biggest challenges to becoming data-oriented and continuing to drive data are due to cultural barriers rather than technological limitations or a lack of viable technology options. This is an example of technology that is ahead of applicability.

Cultural obstacles and barriers to data-oriented transformation efforts primarily arise from challenges in organizing, business processes, change management, communication, human skill sets, and resistance or ignorance of what can make a change. Investing in data governance, understanding data, programs that build awareness of the value and impact of data in an organization, can be a step in the right direction. As demonstrated in Table 2, progress is slow and cultural obstacles continue to be great, with no immediate improvement in companies.

However, as the survey points out, the picture is not entirely bleak. There are notable bright points. For example, the application of Artificial Intelligence (AI) represents a potential bright spot, when companies seem to be making steady progress in adopting AI initiatives. This year, 77.8% of companies reported that AI has now been widely deployed or limited production, up from 65.8% last year, with only 4.1% saying no AI applications were used. This shows steady progress. With the overall investment in data and AI increasing, the application of AI will likely continue.

Another potential brightening point can be observed at the long-term optimism of data executivees. Despite the many challenges faced by leading companies as they pursue their efforts to become data-driven, the majority of executives — 81.0% — hope for the future, expressing optimism about the data/AI prospects in their companies. In these companies, 91.9% reported that the rate of investment in data/AI continued to increase rapidly, another positive index. An even more positive metric is the number of executives – 45.4% – describing their company as at the foremost in making data/AI progress. Even in these most challenging years, 91.6% of companies report that, in the face of the Covid-19 pandemic, their companies will spend the same or more on DATA and AI initiatives. That’s really good news.

A final bright spot is reflected in the commitment that leading companies have made to establish central data functionality, in the form of the Chief Data Office, and to appoint Data Directors to take on these leadership positions. Over the past half decade, more than half of the top companies have taken steps to set up a CDO and appoint a Chief Data Officer. Although the role continues to grow, companies seem to be steadfast in their commitment to the inevitable success of the role, with nearly two-thirds of companies— 65.0% — taking the position of chief executive officer. As illustrated in Table 3, the Role of Data Director is firmly established and is unlikely to be withdrawn.

A decade after the data transformation journey, leading companies are making slow but gradual advances in their efforts to become data-oriented. While investment levels and signs of optimism continue to grow, the results reflect lagging applications and challenges that persist in the organization’s transformation efforts. The current situation is both positive and has bad points.

Over the past decade, Big Data and AI have gradually become the main trend of businesses. Companies that continue to make progress and begin to show results from these investments will transform themselves to compete more effectively in the digital business world of the 21st century. For companies that go after, this will be an important and difficult achievement to win.