Today we will be looking at a BAEF webinar focusing on the types of analytics that companies are using today and the kinds of skills that they require from their analysts. The webinar was live streamed on the 24th June 2024 and presented a study by Dr Christina Phillips and a Master’s student that she supervised.
Zhuoyi Deng started off by giving an account of what the study was about and then onto its main findings. The study started off as a project to investigate what companies are doing in terms of their analytics and the type of skill sets that they would like to see in their analysts. This is from a background of massive amounts of data being generated every day (some 2.5 quintillion bytes a day!) and how some 90% of the worlds current data was generated within the last two years. This has placed a demand from companies for analysts who are able to make sense of all this information (it is estimated that this will translate into some 300K new positions in analytics every year for the remainder of this decade). The objectives of the study being to address skills gaps, inform educational programs, and aid the adoption of various analytics approaches by companies.
The study was conducted in two phases, the initial phase was a pilot study by Jia Woh. The pilot study interviewed 10 senior executives and it informed the development of the questionnaire used in the main study. Some 259 people participated in the main study, answering a questionnaire 0f 64 questions. All the participants had roles within management, IT, or analytics and had an understanding of their companies’ analytics functions.
Of those who responded, some 63% came from large organisations, with 32% from SMEs, and the remainder from micro enterprises. 80% thought that their organisation had a culture of questioning and being inquisitive about using their data, indicating that analytics is being used to question or support management decisions, with nearly 3/4s saying that their company uses data to inform most of their business decisions. Conversely, some 45% of respondents thought that the potential of analytics within their organisation was not fully understood. This suggests that although the use of data is valued, organisations don’t yet fully grasp what the potentials of good analytics can be.
Moving on to some of the challenges faced by analysts today, the study found that the majority of respondents struggled with data being spread across different systems as well as outdated legacy systems impacting on data management. Nearly half the of respondents had mismatches between the company data structure and their data needs. This could imply that many companies are still using legacy systems that might not be able to give full model support, alongside maintenance issues as well as difficulties integrating with modern systems.
Zhuoyi went on to discuss some of the challenges in data analysis. The biggest difficulties in data collection were attributed to the quality of the data and that there seemed to be a significant gap in data analytics skills impacting of the quality of analysis. Unsurprisingly, maybe, the use of spreadsheets like Excel proved to be the most common source of data, followed by cloud based systems. For those using AI/ML models for analytics, just over half said that they used the systems embedded in their analytics provision and only 11% of respondents said that they used Big Data platforms.
In terms of the tools used by companies in their analytics process, the majority of respondents used low code, non-specialised software (such as Excel). Some did use more sophisticated software/programming languages (R or Python) for more specialised analytics.
You might be wondering what it is, in terms of skills, that companies need from their analysts. We are an educator’s forum, and we want to know what we need to teach after all! The next part of the presentation went on to discuss this. The key skill that stood out was presentation and communication skills, this being the most highly rated skill that was required. As important it is to know how to do analytics, getting that information across so that it can inform decision making is equally as important. So, that means challenging our students and getting them to do regular presentations in class. Probably more challenging than teaching the actual statistics! Joking aside, the use of spreadsheets, having a business understanding, and statistics knowledge also came up as key skills required by organisations. This indicates that organisations need analysts to have good technical skills, but that they also need ‘softer’ skills around presenting their analysis in an understandable and actionable way.
Christina Phillips then did a presentation that covered the background to the study through the lens of a story involving a fictional company to highlight the findings of the research and the impact from the perspective of this company. She highlighted the background of the study, why it was conducted, and how. Her account gave context to the findings in terms of the fictional company, highlighting the value of storytelling with data presentations. All agreed that the technique would be stronger if it could use real company feedback; something for the future!
The key take home message is that the analytics
employment environment is very good and prospects are excellent for well
trained analysts. And we need to develop
both the ‘softer’ skills around presentation of findings as well as the more
technical aspects around statistics and the use of software applications.
Dr M C Davies, Lecturer, Bangor University, and BAEF committee member.