Personal data is the core concept of data protection. Data protection law only applies when data relates to individuals. The GDPR for example increases fines to up to 20 million euros or, in the case of large companies and groups, up to 4% of the global group turnover of the previous year [1]. When working in the field of Big Data, Data Science or related fields it is essential to know about these laws and how anonymization and pseudonymization give the possibility of still using the data for your use cases.

What is Personal Data?

This is any information relating to an identified or…

In the world of Big Data, data visualization tools and techniques are essential to analyze large amounts of information and make data-driven decisions as data is increasingly used for important management decisions. So there is a trend away from gut feeling and emotional decisions towards rational choices that are made based on numbers. Therefore, reports and visualizations have to be easily understood and meaningful.

Impact of Big Data

It is increasingly beneficial for professionals to be able to use data to make decisions and visuals to tell stories that communicate how data informs the question of person, subject, time, place, and method [1]. In…

The File Transfer Protocol is for the communication of people and devices over the Internet and other networks works through protocols [1]. Because FTP is an older method of data transfer, such transfers are compatible with many legacy and/or on-premises HR and business systems, making it a useful option if you want to integrate an older system with newer, cloud-based software.

Why FTP is still Alive

In the past, most digital systems were connected via FTP integration, where one system exports data in a “flat file” format (often a spreadsheet) and another system imports the data. …

Whether it’s for university, your job, or simply as input for your next story — there are many interesting sources for free whitepapers and educational material in the field of data. With some sources you have to say that there might be a certain intention to sell a product but with the sources I use, scientific thought is mostly in the foreground. Here are my top places to go:

THE SNOWFLAKE DUMMIES GUIDE SERIES

Everyone knows the for Dummies series. You can buy them on Amazon and in good book stores. Snowflake delivers it for free — after you have registered. Top current topics like…

Work more efficiently with the powerful BigQuery IDE powered by AI that supports Data Engineers, Scientists and BI Developers.

The Chrome Add On features [1]:

- AI engine that optimizes your queries in real-time.

- Adaptive Caching — Never pay twice for the same query.

- Write queries faster with context-aware Smart Compose

- Execute up to 20 queries at the same time.

- Auto-Detect Standard / Legacy SQL.

- Use variables to store values and shorten your workflow.

- Visualize query results with integrated dashboards.

- Download up to 6,000,000 rows to CSV.

You can download the Chrome extension…

Business Event Analysis & Modelling (BEAM) is an agile requirement gathering for Data Warehouses, with the goal of aligning requirement analysis with business processes rather than just reports. It has its roots in Agile Data Warehouse Design by Lawrence Corr and Jim Stagnitto [1].

Principles

The key principles of this concept are [1][2]:

• Individuals and Interactions: Business intelligence is driven by what users ask about their business. The technical setting is secondary.
• Business Driven: Well documented data warehouses that take years to deploy will always be out of date. Business users will look elsewhere. …

In this article I want to demonstrate how you can use open data for interesting use cases. If you are interested in how to get started with BigQuery and Alteryx, this article might be the one for you. In this walkthrough, I will use OpenStreetMap data from BigQuery. It’s great that Google published it for free in their public data sets. So you can easily query geo information with SQL [1]. At the end of the article I have listed other interesting public data sources that you can use for your projects.

OpenStreetMap

One important side note: Google uploaded their data…

In the field of logistics and warehousing, there is of course a lot of software that helps you with the optimal route, warehouse utilization and supply chain management. But what are the basic KPIs in this area? Whether you are a data scientist, analyst or business analyst, it is useful to understand what the individual departments actually do. What are their goals? How do they earn their money directly or indirectly? How can the success be measured? These question with their respective answers helped me to understand the key figures of the individual departments.

Sources for Data

The first question is of course…