Data can be referred to as of high quality if it has the following characteristics:
- Accuracy and correctness
- Completeness
- Consistency
- Usability
- Reliability and Relevance
- Privacy etc.
As the quantity of data grows, the quality is likely to decrease due to various reasons. The data quality decreases if data is inaccurate or incorrect, inconsistent, incomplete and not usable. If data quality decreases, then it is not usable for its consumers and enterprises. In such cases, data cleansing or data cleaning is the only solution.
There are various methods and steps involved in data cleaning. Some of those methods and steps are as follows:
- Removing the duplicate data.
- Validating the existing data set with a set of known(accurate) data.
- Fixing Typographical and Structural errors.
- Using Regular Expressions or Fuzzy modelling for validating data.
- Normalization of the data.
- Adding/handling the missing data.
In the process of data cleansing or data cleaning, the inaccurate, incomplete and inconsistent data is either modified(corrected) or removed. This results in an increase in the quality of data. Hence we can conclude that the data cleaning process(operations) always ensures high-quality data.