coding-interview-university
A complete computer science study plan to become a software engineer.
- 365k
- CC-BY-SA-4.0
- 13 d ago
๐ฃ Data Processing interview questions and answers to help you prepare for your next machine learning and data science interview in 2026.
Repo Devinterview-io/data-processing-interview-questions, read from GitHub 52 min ago. Devinterview-io has not joined Take My Repo.
Devinterview-io/data-processing-interview-questions is a public repo on GitHub by Devinterview-io. ๐ฃ Data Processing interview questions and answers to help you prepare for your next machine learning and data science interview in 2026. It declares no license on GitHub. Public does not mean open source: without a license, nobody is granted the right to reuse it. It has 35 stars and last changed on 2026-01-04. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-20.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
Data preprocessing, often known as data cleaning, is a foundational step in the machine learning pipeline. It focuses on transforming and organizing raw data to make it suitable for model training and to improve the performance and accuracy of machine learning algorithms.
Data preprocessing typically involves the following steps:
Data Collection: Obtaining data from various sources such as databases, files, or external APIs.
Data Cleaning: Identifying and handling missing or inconsistent data, outliers, and noise.
Data Transformation: Converting raw data into a form more amenable to ML algorithms. This can include standardization, normalization, encoding, and feature scaling.
Feature Selection: Choosing the most relevant attributes (or features) to be used as input for the ML model.
Dataset Splitting: Separating the data into training and testing sets for model evaluation.
Data Augmentation: Generating additional training examples through techniques such as image or text manipulation.
Text Preprocessing: Specialized tasks for handling unstructured textual data, including tokenization, stemming, and handling stopwords.
Feature Engineering: Creating new features or modifying existing ones to improve model performance.
Here is the Python code:
import pandas as pd from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler, LabelEncoder # Load the data from a CSV file data = pd.read_csv('data.csv') # Handle missing values data.dropna(inplace=True) # Perform label encoding encoder = LabelEncoder() data['category'] = encoder.fit_transform(data['category']) # Split the data into features and labels X = data.drop('target', axis=1) y = data['target'] # Split the data into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Standardize the features scaler = StandardScaler() X_train = scaler.fit_transform(X_train) X_test = scaler.transform(X_test)
Data cleaning is a critical step in the machine learning pipeline, helping to prevent issues that arise from inconsistent or noisy data.
Handling Missing Data: Select the most suitable method, such as imputation, for missing values.
Outlier Detection and Treatment: Identify and address outliers, ensuring they don't unduly influence the model's behavior.
Noise Reduction: Using techniques such as binning or smoothing to reduce the impact of noisy data points.
Addressing Data Skewness: For imbalanced datasets, techniques like oversampling or undersampling can help.
Normalization and Scaling: Ensure data is on a consistent scale to enable accurate model training.
Ensuring Data Consistency: Methods such as data type casting can bring uniformity to data representations.
Feature Engineering and Selection: Constructing or isolating meaningful features can enhance model performance.
Text and Categorical Data Handling: Encoding, vectorizing, and other methods convert non-numeric data to a usable format.
Data Integrity: Data cleaning aids in data validation, ensuring records adhere to predefined standards, such as data ranges or formats.
Here is the Python code:
import pandas as pd # Load data into a DataFrame df = pd.read_csv('your_dataset.csv') # Handling missing values median_age = df['age'].median() df['age'].fillna(median_age, inplace=True) # Outlier treatment using Z-Score (replacing outliers with median) from scipy import stats z_scores = np.abs(stats.zscore(df['income'])) df['income'] = np.where(z_scores > 3, median_income, df['income']) # Normalization and scaling from sklearn.preprocessing import StandardScaler scaler = StandardScaler() df[['feature1', 'feature2']] = scaler.fit_transform(df[['feature1', 'feature2']]) # Data type consistency df['gender'] = df['gender'].astype('category') # Text and categorical data handling (One-Hot-Encoding) df = pd.get_dummies(df, columns=['location']) # Data integrity (example: age cannot be negative) df = df[df['age'] >= 0]
Data quality issues can significantly impact the accuracy and reliability of machine learning models, leading to suboptimal performance.
Attributes lacking data can impede the learning process. Common strategies include data imputation, decrease in model sensitivity to missing data, or special treatment of missing values as a distinct category.
Outliers, though not necessarily incorrect, can unduly skew statistical measures and models. You can choose to remove such anomalous points or transform them to reduce their influence.
Inconsistencies can arise from manual entry or parameter disparities. Aggressive data cleaning and standardization are effective steps in countering this issue.
Redundant information offers no additional value and can lead to overfitting in models. It's wise to detect and eliminate replicas.
Data can be incomplete or outright incorrect due to various reasons including measurement errors, data transmission errors, or bugs in data extraction pipelines. Quality assurance protocols should be implemented throughout the data pipeline.
Skewed distributions, which are either highly asymmetric or include a significant bias, can misrepresent the true data characteristics. Techniques such as log-transformations or bootstrapping can address this.
Visualizations such as histograms, box plots, and scatter plots are invaluable in deducing characteristics about the quality of the dataset, like the presence of outliers.
Machine learning applications rely on two primary forms of data: structured and unstructured data.
Definition: Structured data follows a strict, defined format. It is typically organized into rows and columns and is found in databases and spreadsheets. It also powers the backbone of most business operations and many analytical tools.
Example: A company's sales report containing columns for date, product, salesperson, and revenue.
Usage in machine learning: Structured data straightforwardly maps to supervised learning tasks. Algorithms process specific features to generate precise predictions or classifications.
Definition: Unstructured data is, as the name suggests, devoid of a predefined structure. It doesnโt fit into a tabular format and might contain text, images, audio, or video data.
Example: Customer reviews, social media content, and sensor data are typical sources of unstructured data.
Usage in machine learning: Unstructured data commonly feeds into unsupervised learning platforms. Techniques like clustering help derive patterns from such data, and algorithms like k-means can group similar data points together.
Further, advancements in NLP, computer vision, and speech recognition have empowered machine learning to effectively tackle unstructured inputs, such as textual content, images, and audio streams.
Feature Scaling is a critical step in many machine learning pipelines, especially for algorithms that rely on similarity measures such as Euclidean distance. It ensures that all features contribute equally to the predictive analysis.
Algorithm Performance: Models like K-Means clustering and Support Vector Machines (SVM) are sensitive to feature scales. In their absence, features with higher magnitudes can dominate those with lower magnitudes.
Convergence: Gradient-descent based methods converge more rapidly on scaled features.
Regularization: Algorithms like the LASSO (Least Absolute Shrinkage and Selection Operator) are sensitive to feature magnitudes, meaning unscaled features might be penalized more.
Interpretability: Feature scaling helps models interpret the importance of features in a consistent manner.
$$ X_{\text{new}} = \frac{X - X_{\text{min}}}{X_{\text{max}} - X_{\text{min}}} $$
Feature values are mapped to a common range, typically $[0, 1]$ or $[-1, 1]$.
$$ X_{\text{new}} = \frac{X - \mu}{\sigma} $$
Here, $\mu$ is the mean and $\sigma$ is the standard deviation. Standardization makes features have a mean of zero and a standard deviation of one.
Robust Scaling: This type is similar to standardization, but it uses the median and the interquartile range (IQR) instead of the mean and standard deviation. It is more suited for datasets with outliers.
Unit Vector Scaling: This method scales each feature to have a unit norm (magnitude), making it particularly beneficial for methods that use distances, like K-Nearest Neighbors (KNN).
Gaussian Transformation: Using techniques like the Box-Cox transformation can help stabilize the variance and make the data approximately adhere to the normal distribution, which some algorithms may assume.
Data normalization is essential for ensuring consistent and accurate model training. It minimizes the impact of varying feature scales and supports the performance of many machine learning algorithms.
$$ \text{Scaled Value} = \frac{\text{Value} - \text{Min}}{\text{Max} - \text{Min}} $$
$$ \text{Scaled Value} = \frac{\text{Value} - \text{Mean}}{\text{Standard Deviation}} $$
$$ \text{Scaled Value} = \frac{\text{Value} - \text{Median}}{\text{Interquartile Range}} $$
Data Augmentation involves artificially creating more data from existing datasets, often by applying transformations such as rotation, scaling, or other modifications.
Here is the Python code:
from keras.preprocessing.image import ImageDataGenerator import matplotlib.pyplot as plt import numpy as np # Load sample image img = plt.imread('path_to_image.jpg') # Create an image data generator datagen = ImageDataGenerator( rotation_range=40, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode='nearest') # Reshape the image and visualize the transformations img = img.reshape((1,) + img.shape) i = 0 for batch in datagen.flow(img, batch_size=1): plt.figure(i) imgplot = plt.imshow(np.squeeze(batch, axis=0)) i += 1 if i % 5 == 0: break plt.show()
Data encoding is crucial for preserving information across systems and during storage, especially in the context of Machine Learning applications that sometimes deal with non-traditional data types.
Compatibility: Different systems and software might have varied requirements on how data is represented. Encoding ensures data is interpreted as intended.
Interoperability: Complex applications, especially in Machine Learning, often involve multiple disparate components. A common encoding scheme ensures they can interact effectively.
Text Representation: Not all data is numerical. Text, categorical values, and even images and audio require appropriate representation for computational processes.
Error Detection and Correction: Certain encoding schemes offer mechanisms for detecting and correcting errors during transmission or storage.
Efficient Storage: Some encodings are more space-efficient, which is valuable when dealing with large datasets.
Security: Certain encoding methods, such as encryption, are crucial for safeguarding sensitive data.
Versioning: In systems where data structures might evolve, encoding can ease transitions and ensure compatibility across versions.
Internationalization and Localization: In the case of text data, encoding schemes are necessary for managing multiple languages and character sets.
Data Compression: This method, often used in multimedia contexts, reduces the size of the data for efficient storage or transmission.
Data Integrity: By encoding information in a specific way, we ensure it remains intact and interpretable during its lifecycle.
One-Hot Encoding: converting categorical variables into a set of binary vectors (0/1, true/false) โ useful for algorithms that can process only numeric data.
Label Encoding: converting categorical variables into numerical labels โ especially useful in algorithms that can work with unordered categorical data.
Binary Encoding: representing integers with binary digits.
Gray Code: Optimized version of binary code where consecutive values differ by only a single bit.
Base64 Encoding: A technique used for safe data transfer in web protocols and APIs, particularly when data might contain special, non-printable, or multi-byte characters.
Unicode: A global standard to interpret and represent different characters and symbols across diverse languages.
JSON and XML: Standard ways to structure and encode complex data, often used in web services and data interchange. While both JSON and XML supply data in a clear, human-readable format, XML has a mechanism for data validity in the form of a schema definition.
CSV ("Comma Separated Values"): Itโs simple, text-based, and serves as a cross-platform data exchange format for spreadsheets and databases.
Encryption Algorithms such as Advanced Encryption Standard (AES) and RivestโShamirโAdleman (RSA).
Missing data presents challenges for statistical analysis and machine learning models. Here are several strategies to handle it effectively.
Eliminate: Remove data entries with missing values. While this simplifies the dataset, it reduces the sample size and can introduce bias.
Fill with Measures of Central Tendency: Impute missing values with statistical measures such as mean, median, or mode. This approach preserves the data structure but can affect statistical estimates.
Predictive Techniques: Use machine learning models or algorithms to predict missing values based on other features in the dataset.
Here is the Python code:
# Import pandas import pandas as pd # Create a sample DataFrame data = {'A': [1, 2, 3, None, 5], 'B': ['a', 'b', None, 'c', 'd']} df = pd.DataFrame(data) # Print original DataFrame print(df) # Drop rows with any missing values dropped_df = df.dropna() print(dropped_df) # Fill missing values with mean filled_df = df.fillna(df.mean()) print(filled_df) # Predict missing values in 'B' based on 'A' using simple imputation from sklearn.impute import SimpleImputer imputer = SimpleImputer(strategy='most_frequent') df['B'] = imputer.fit_transform(df[['B']]) print(df)
When dealing with missing data, two common strategies are imputation and deletion.
Deletion methods remove instances with missing values. This can be done in multiple fashions:
Imputation involves substituting missing values with either an estimated value or a placeholder, often following a statistical or data-driven approach.
Some common imputation methods include:
Deletion:
Imputation:
The choice between these methods should consider the unique characteristics of the dataset, the nature of the missingness, and the specific domain needs.
Imputation techniques serve to handle missing data, each with its trade-offs.
Pros:
Cons:
Pros:
Cons:
Pros:
Cons:
K-nearest neighbors (KNN) imputation leverages $k$ closest data points to replace missing values. This method is frequently employed in exploratory data analysis.
Data Setup:
Distance Calculation:
K-Neighbor Selection:
Imputation:
Sensitivity to k:
Here is the Python code:
from sklearn.impute import KNNImputer import numpy as np # Example feature matrix with missing values X = np.array([[1, 2, np.nan], [4, 5, 6], [7, 8, 9]]) # Initialize KNN imputer with 2 nearest neighbors imputer = KNNImputer(n_neighbors=2) # Impute and display result X_imputed = imputer.fit_transform(X) print(X_imputed)
Regression imputation can be helpful when dealing with missing data. By leveraging the relationships among variables in your dataset through regression, it imputes missing values more accurately.
Here is the Python code:
import pandas as pd from sklearn.linear_model import LinearRegression from sklearn.model_selection import train_test_split # Read data data = pd.read_csv('data.csv') # Split into missing and non-missing data missing_data = data[data['target_variable'].isnull()] complete_data = data.dropna(subset=['target_variable']) # Split the complete data into train and test sets X_train, X_test, y_train, y_test = train_test_split( complete_data[['predictor1', 'predictor2']], complete_data['target_variable'], test_size=0.2, random_state=42 ) # Train the regression model regressor = LinearRegression() regressor.fit(X_train, y_train) # Predict missing values missing_data['target_variable'] = regressor.predict(missing_data[['predictor1', 'predictor2']])
Missing values can heavily compromise the predictive power of machine learning models, as most algorithms struggle to work with incomplete data.
Data Avoidance: Eliminate records or features with missing values. Though it's a quick fix, it reduces the dataset size and can introduce bias.
Single-value Imputation: Replace missing values using the attribute's mode, median, or mean. While easy, it can introduce bias.
Hot Deck Imputation: Replace a missing value with a randomly selected observed value within the same dataset. Can be more effective, especially for non-linear relationships.
Model-based Imputation: Use an ML algorithm to predict missing values based on available data. This method can be effective if there are patterns in the missing data.
Advanced Techniques: K-nearest neighbor (KNN), Expectation-Maximization (EM), and data-driven methods like Pandas' .fillna() all have different degrees of complexity and potential accuracy.
Here is the Python code:
import pandas as pd from sklearn.impute import SimpleImputer # Load data data = pd.read_csv("data.csv") # Initialize the imputer imputer = SimpleImputer(strategy='mean') # Fit the imputer to the data imputer.fit(data) # Apply the imputer to the dataset imputed_data = imputer.transform(data)
Mean Absolute Error (MAE): Measure the absolute difference between imputed and true values, then find the average.
Root Mean Squared Error (RMSE): Calculate the square root of the mean of the squared differences between imputed and true values.
Predictive Accuracy: Apply different imputation strategies and compare the impact on model performance.
Visual Analysis: Observe patterns in the data and see how different imputation strategies capture these patterns.
One-Hot Encoding (OHE) is a preprocessing technique for transforming categorical features into a form that is interpretable for machine learning algorithms.
Each categorical variable with $n$ unique categories is transformed into $n$ new binary variables. For a given data point, only one of these binary variables takes on the value 1 (indicating the presence of that category), with all others being 0, which is why it is called One-Hot Encoding.
Algorithm Suitability: Certain algorithms (like regression models) require numeric input, making OHE a prerequisite for categorical data.
Algorithm Performance: OHE can lead to improved model performance by preventing the model from misinterpreting ordinal or nominal categorical data as having a specific order or hierarchy.
Visualization: Transparency of one-hot encoded features is an added benefit for model interpretation and understanding.
Here is a Python code:
import pandas as pd # Sample data data = pd.DataFrame({'Size': ['S', 'M', 'M', 'L', 'S', 'L']}) # One-hot encoding one_hot_encoded = pd.get_dummies(data, columns=['Size']) print(one_hot_encoded)
Output:
| Size_L | Size_M | Size_S | |
|---|---|---|---|
| 0 | 0 | 0 | 1 |
| 1 | 0 | 1 | 0 |
| 2 | 0 | 1 | 0 |
| 3 | 1 | 0 | 0 |
| 4 | 0 | 0 | 1 |
| 5 | 1 | 0 | 0 |
For $n$ categories, one-hot encoding generates $n$ binary features, potentially leading to the curse of dimensionality. This can affect model performance with sparse or high-dimensional data.
One-hot encoding is undistorted, with distances (like Hamming distance) reflecting the true dissimilarities or similarities between categories.
The variance of one-hot encoded features can become a pitfall in some model algorithms.
Devinterview-io/data-processing-interview-questions is a public repo on GitHub by Devinterview-io. ๐ฃ Data Processing interview questions and answers to help you prepare for your next machine learning and data science interview in 2026.
It declares no license on GitHub. Public does not mean open source: without a license, nobody is granted the right to reuse it.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
A complete computer science study plan to become a software engineer.
Its last change on GitHub was on 2026-01-04. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
12 weeks, 26 lessons, 52 quizzes, classic Machine Learning for all
Bring data to life with SVG, Canvas and HTML. :bar_chart::chart_with_upwards_trend::tada:

๐ฃ LLMs interview questions and answers to help you prepare for your next machine learning and data science interview in 2026.

๐ฃ Python interview questions and answers to help you prepare for your next technical interview in 2026.

๐ฃ SQL interview questions and answers to help you prepare for your next machine learning and data science interview in 2026.

๐ฃ Pytorch interview questions and answers to help you prepare for your next machine learning and data science interview in 2026.

๐ฃ React interview questions and answers to help you prepare for your next technical interview in 2026.

๐ฃ API Design interview questions and answers to help you prepare for your next software architecture and design patterns interview in 2026.
This page is also available as structured data, with the same sources and dates.
/api/v1/resources/gh/Devinterview-io/data-processing-interview-questionsTake My Repo has not tested this on your machine. Requirements come from the project's own documentation.