Data preprocessing
Data preprocessing can refer to manipulation, filtration or augmentation of data before it is analyzed, and is often an important step in the data mining process. Data collection methods are often loosely controlled, resulting in out-of-range values, impossible data combinations, and missing values, amongst other issues. Preprocessing is the process by which unstructured data is transformed into intelligible representations suitable for machine-learning models. This phase of model deals with noise in order to arrive at better and improved results from the original data set which was noisy. This dataset also has some level of missing value present in it.
Related work
- Intro to machine learning Dec 2024
Each Jupyter notebook covers one preprocessing technique in Python: filling in missing values (imputation); one-hot encoding and label encoding of categories; normalisation and standardisation of numbers; discretisation, turning numbers into bins.