According to a recent survey, the vast majority of data engineers are burnt out. Experts are saying the healthcare industry is no exception.
DataKitchen and data.world commissioned the poll, which reached 600 data engineers, 100 of which are managers. Almost all of them said they were burnt out in their everyday jobs, with the majority noting that they were thinking of quitting the industry or their present firm in the next year.
The goal of the study, according to Christopher Bergh, CEO, and co-founder of DataKitchen was to address the poor mental health of data engineers and to focus the study on their emotional states rather than only on commercial principles.
The most common causes of burnout, according to the poll, include too much time spent correcting errors, repeated manual activities linked to data preparation, and a never-ending stream of requests from coworkers, many of which are unrealistic.
Over two-thirds of respondents said they were scapegoated for mistakes, that data governance policies stifle openness and productivity, and that unanticipated workloads disrupted their work-life balance. The majority of respondents (78%) said they would like a therapist to accompany them on the job.
More than half of respondents believed their firms did not adequately handle or test for data quality concerns, which inevitably result in errors and production failures. The companies stated in a survey summary report that dealing with continual disruptions while trying to develop new initiatives is “like trying to play ‘whack a mole’ while simultaneously reading a book.”
Data experts collaborate to assist create data-driven decisions that have the potential to affect operations. Finding and testing molecules for new medications, for example, is a rigorous procedure that necessitates creating ideas and putting them to the test. According to Kurt Zimmer, former head of data engineering for data enablement at AstraZeneca, a DataKitchen client noted that anything that speeds up the development cycle time would tremendously assist a firm.
He lamented the fact that there is “too much data, too many things in process, too many people in the loop.” He was hired by AstraZeneca to help speed up decision-making, but the budget was cut, and with it, the company’s capacity to hire the proper individuals. Burnout is almost always an “obvious” indication of that kind of dissatisfaction. Given their personal experiences in the field, neither he nor Bergh was shocked by the conclusions of the DataKitchen study.
Bergh told Fierce Healthcare that there is a “promise of a data-driven future” in healthcare, such as precision medicine. However, he claims that the labor required to attain it is demanding and crammed with too many daily activities. While technology is continuously developing, the processes for utilizing it are not. According to Zimmer, these outmoded practices foster a “fixing crap” culture rather than systematically addressing fundamental problems and increasing data engineer productivity. According to the survey, only approximately 22% of data engineers’ time is spent on value-creating innovation.
Zimmer and Bergh recommended DataOps, a methodology offered by DataKitchen, to fight the problem. It streamlines routine procedures, ensures the integrity of underlying data flows, increases team collaboration, and frees up data engineers to focus on higher-value work. “The system makes the team,” Bergh stated.
Source: Fierce Healthcare