Member-only story
5 Critical Problems in Data Science and Engineering
What are the common Issues and how can they be solved?
In the field of Data Analytics and related topics like Business Intelligence, Data Science, Data Engineering etc. you often will hear about the same problems when working in a project or on a product. Here, I want to share my experiences and possible solutions.
Problem 1: The Data is Wrong
One of the most unpleasant moments in the life of every project or product manager is when the business department complains about the data quality. The problems can be of different nature. Errors in the source system, ETL process or in the report.
Solution: Here, it is a good idea to set up a monitoring system and to monitor the source systems and the ETL process. In addition, it makes sense to check the data quality, this can be realized by monitoring and data reconciliation between source and target systems. Read more about it here [1].
Problem 2: I will Stick to Excel
No — we’ve always had our VBA Excel tool here, we don’t need a new modern Self Service BI. This is a very normal defensive attitude of a business user. Because new things are scary, people often react with rejection at first.
