There is an opinion that a Big Data expert is a superprofessional, a superhuman with a huge number of different skills. To some extent, this is true, because, among other things, he must have a good understanding of the business. Of course, it is difficult for one person to know everything, so they often work in teams – it is much more productive. It’s very important to hire big data engineers with the right big data skillset.
The main stages of the work of a Big Data specialist
They work with a variety of areas: finance, retail, legal industries. One of the most important roles is a strategist:
- At the first stage, most companies simply do not know how to start working with big data. What’s more, sometimes it’s very difficult to understand exactly what the company’s problem is with that data and how to solve it.
- The most important thing is to identify the problem the company is facing. They hold workshops where they talk about the possibilities of Big Data. In the process, they have to transform the way the company works, but our first priority is to solve the problem. They talk to the customer, asking a lot of questions about all areas of the business. In the process of these conversations, there are huge lists of items and tasks that they will consider and work on. The main goal they have when working with Big Data is to be able to better understand the customer, the product, the employees, the suppliers.
- After collecting information, they discuss all the problem points and understand if they are related to Big Data. Some problems may be related to something else – like a lack of employee motivation. So they have to reduce the whole list and leave only the problems that are related to our competence. If you want to know more about your sales, that means you have to be able to keep track of them. Sometimes this is quite difficult. For example, in stores, you have to be able to account for every purchase. But that’s not a Big Data problem. It means you just have to buy a system to keep track of your purchases. Sometimes there have to be a number of significant changes in the company before a Big Data specialist can get started.
- The next step is to make a list of recommendations. After that, we discuss the future strategy of the company, what the managers want it to be. Implementing Big Data is not just about bringing in one specialist, it’s about changing the mindset of all employees. It is very important that everyone understands what the guy who calls himself a Big Data expert is doing. It is very important to dispel the myth that Big Data is just some part of the IT department.
The main skills a Big Data specialist should have
The main thing is the ability to work with large amounts of information and knowledge of technologies: there are hundreds of them already, and new ones appear every month. At the same time:
– he must have scientific thinking and be very inquisitive.
– it’s very important to be able to think in business terms.
– you can be a narrow specialist in something and be a useful member of the team, being responsible for your part of the process.
What industries need Big Data the most
Recently, Big Data has been increasingly used in the banking sector, public administration, and agriculture. Bringing in a Big Data expert is an opportunity to look at existing info from different angles. Sometimes with students, they look at very simple data sets – for example, tables with just three columns:
- date,
- customer number;
- purchase amount.
Although it may seem primitive, I show students how much new information they can get from it. Even if you don’t have a lot of info, you can make predictions and conclusions.
