Input preparation or data preparation is the process of preparing data for the next step in your work, which may include detecting and updating unusable data. Output validation is the process of checking that your work actually accomplished what you intended it to accomplish.

Motivation

You’ve probably had the experience of searching for something on the web and didn’t find what you wanted. Perhaps, you updated your search term and then found a relevant link. Your initial data input was unusable for your intended search. So you prepared your data, by modifying your search. Then you validated output from the search engine, by determining it was relevant for your purposes. Of course, you did all of this in your head, so it didn’t seem so procedural, but it was essentially the same processes that we go through more consciously and intentionally when working on technical problems with data.

Software Tools

We’ll use Jupyter Notebook, Excel and a text editor in this lesson.

...