What are the common Issues and how can they be solved?

Image for post
Image for post
Photo by Taylor Friehl on Unsplash

In the field of Data Analytics and related topics like BI, Data Science, Data Engineering etc. you often will hear about the same problems when working in a project or on a product. Here, I want to share my experiences and possible solutions.

One of the most unpleasant moments in the life of every project or product manager is when the business department complains about the data quality. The problems can be of different nature. Errors in the source system, ETL process or in the report.

Solution: Here, it is a good idea to set up a monitoring system and…


How to Anonymize and Pseudonymize Data

Image for post
Image for post
Photo by Francisco Suarez on Unsplash

Personal data is the core concept of data protection. Data protection law only applies when data relates to individuals. The GDPR for example increases fines to up to 20 million euros or, in the case of large companies and groups, up to 4% of the global group turnover of the previous year [1]. When working in the field of Big Data, Data Science or related fields it is essential to know about these laws and how anonymization and pseudonymization give the possibility of still using the data for your use cases.

This is any information relating to an identified or…


How to gain Insights on new Visualizing Techniques

Image for post
Image for post
Photo by Nikolay Maslov on Unsplash

In the world of Big Data, data visualization tools and techniques are essential to analyze large amounts of information and make data-driven decisions as data is increasingly used for important management decisions. So there is a trend away from gut feeling and emotional decisions towards rational choices that are made based on numbers. Therefore, reports and visualizations have to be easily understood and meaningful.

It is increasingly beneficial for professionals to be able to use data to make decisions and visuals to tell stories that communicate how data informs the question of person, subject, time, place, and method [1]. In…


Why FTP is still alive and how you can implement it into a modern Cloud Data Platform

Image for post
Image for post
Photo by Egor Myznik on Unsplash

The File Transfer Protocol is for the communication of people and devices over the Internet and other networks works through protocols [1]. Because FTP is an older method of data transfer, such transfers are compatible with many legacy and/or on-premises HR and business systems, making it a useful option if you want to integrate an older system with newer, cloud-based software.

In the past, most digital systems were connected via FTP integration, where one system exports data in a “flat file” format (often a spreadsheet) and another system imports the data. …


Why you don’t need to pay much for Learning and gaining Insights

Image for post
Image for post
Kerensa Pickett on Unsplash

Whether it’s for university, your job, or simply as input for your next story — there are many interesting sources for free whitepapers and educational material in the field of data. With some sources you have to say that there might be a certain intention to sell a product but with the sources I use, scientific thought is mostly in the foreground. Here are my top places to go:

Everyone knows the for Dummies series. You can buy them on Amazon and in good book stores. Snowflake delivers it for free — after you have registered. Top current topics like…


Benefits of using the Google BigQuery IDE

Image for post
Image for post
Photo by Joe Ciciarelli on Unsplash

Work more efficiently with the powerful BigQuery IDE powered by AI that supports Data Engineers, Scientists and BI Developers.

The Chrome Add On features [1]:

- AI engine that optimizes your queries in real-time.

- Adaptive Caching — Never pay twice for the same query.

- Write queries faster with context-aware Smart Compose

- Execute up to 20 queries at the same time.

- Auto-Detect Standard / Legacy SQL.

- Use variables to store values and shorten your workflow.

- Visualize query results with integrated dashboards.

- Download up to 6,000,000 rows to CSV.

You can download the Chrome extension…


Ask the right Questions to succeed in your Data Analytics Projects

Image for post
Image for post
Photo by Kalen Emsley on Unsplash

Business Event Analysis & Modelling (BEAM) is an agile requirement gathering for Data Warehouses, with the goal of aligning requirement analysis with business processes rather than just reports. It has its roots in Agile Data Warehouse Design by Lawrence Corr and Jim Stagnitto [1].

The key principles of this concept are [1][2]:

  • Individuals and Interactions: Business intelligence is driven by what users ask about their business. The technical setting is secondary.
  • Business Driven: Well documented data warehouses that take years to deploy will always be out of date. Business users will look elsewhere. …


A Walkthrough inspired by a Real Use Case using Googles BigQuery connection with Alteryx

Image for post
Image for post
Photo by denise farley on Unsplash

In this article I want to demonstrate how you can use open data for interesting use cases. If you are interested in how to get started with BigQuery and Alteryx, this article might be the one for you. In this walkthrough, I will use OpenStreetMap data from BigQuery. It’s great that Google published it for free in their public data sets. So you can easily query geo information with SQL [1]. At the end of the article I have listed other interesting public data sources that you can use for your projects.

OpenStreetMap

One important side note: Google uploaded their data…


How to get started with Analytics in the Field of Logistics and Warehousing

Image for post
Image for post
Photo by Timelab Pro on Unsplash

In the field of logistics and warehousing, there is of course a lot of software that helps you with the optimal route, warehouse utilization and supply chain management. But what are the basic KPIs in this area? Whether you are a data scientist, analyst or business analyst, it is useful to understand what the individual departments actually do. What are their goals? How do they earn their money directly or indirectly? How can the success be measured? These question with their respective answers helped me to understand the key figures of the individual departments.

The first question is of course…


What you have to know about Artificial Intelligence

Image for post
Image for post
Photo by Victor Garcia on Unsplash

Here is everything you have to know about Artificial Intelligence, Machine Learning and Deep Learning.

Artificial Intelligence is an umbrella term and describes the broad approach of using machines to imitate intelligent human behavior in order to solve problems.

Christianlauer

Big Data Enthusiast based in Hamburg and Kiel.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store