PayloadsAllTheThings
A list of useful payloads and bypass for Web Application Security and Pentest/CTF
- 83k
- Python
- MIT
- 3 wk ago
SciKit Learn Machine Learning Cheat Sheet
Repo mpolinowski/python-scikitlearn-cheatsheet, read from GitHub 1 min ago. mpolinowski has not joined Take My Repo.
mpolinowski/python-scikitlearn-cheatsheet is a public repo on GitHub by mpolinowski. SciKit Learn Machine Learning Cheat Sheet. It declares no license on GitHub. Public does not mean open source: without a license, nobody is granted the right to reuse it. It is written mainly in Jupyter Notebook, has 24 stars and last changed on 2023-06-17. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-19.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
Regressions ++ Classifications ++ Clustering ++ Dimensionality Reduction ++ Model Selection ++ Pre-processing
import matplotlib.pyplot as plt import matplotlib.image as mpimg import numpy as np import pandas as pd import plotly.express as px import seaborn as sns from sklearn import svm from sklearn.cluster import KMeans from sklearn.datasets import load_iris, load_wine, fetch_20newsgroups, fetch_openml from sklearn.impute import MissingIndicator, SimpleImputer from sklearn.ensemble import ( RandomForestClassifier, RandomForestRegressor, GradientBoostingRegressor, AdaBoostRegressor, GradientBoostingClassifier, AdaBoostClassifier ) from sklearn.feature_extraction.text import ( CountVectorizer, TfidfTransformer, TfidfVectorizer ) from sklearn.linear_model import ( LinearRegression, LogisticRegression, Ridge, ElasticNet ) from sklearn.metrics import ( mean_absolute_error, mean_squared_error, classification_report, confusion_matrix, ConfusionMatrixDisplay, accuracy_score ) from sklearn.model_selection import ( train_test_split, GridSearchCV, cross_val_score, cross_validate ) from sklearn.naive_bayes import MultinomialNB from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor from sklearn.pipeline import Pipeline, make_pipeline from sklearn.preprocessing import ( MinMaxScaler, StandardScaler, OrdinalEncoder, LabelEncoder, OneHotEncoder, PolynomialFeatures ) from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
X_missing = pd.DataFrame( np.array([5,2,3,np.NaN,np.NaN,4,-3,2,1,8,np.NaN,4,10,np.NaN,5]).reshape(5,3) ) X_missing.columns = ['f1','f2','f3'] X_missing
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | 5.0 | 2.0 | 3.0 |
| 1 | NaN | NaN | 4.0 |
| 2 | -3.0 | 2.0 | 1.0 |
| 3 | 8.0 | NaN | 4.0 |
| 4 | 10.0 | NaN | 5.0 |
X_missing.isnull().sum() # f1 1 # f2 3 # f3 0 # dtype: int64
indicator = MissingIndicator(missing_values=np.NaN) indicator = indicator.fit_transform(X_missing) indicator = pd.DataFrame(indicator, columns=['a1', 'a2']) indicator
| a1 | a2 | |
|---|---|---|
| 0 | False | False |
| 1 | True | True |
| 2 | False | False |
| 3 | False | True |
| 4 | False | True |
imputer_mean = SimpleImputer(missing_values=np.NaN, strategy='mean') X_filled_mean = pd.DataFrame(imputer_mean.fit_transform(X_missing)) X_filled_mean.columns = ['f1','f2','f3'] X_filled_mean
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | 5.0 | 2.0 | 3.0 |
| 1 | 5.0 | 2.0 | 4.0 |
| 2 | -3.0 | 2.0 | 1.0 |
| 3 | 8.0 | 2.0 | 4.0 |
| 4 | 10.0 | 2.0 | 5.0 |
imputer_median = SimpleImputer(missing_values=np.NaN, strategy='median') X_filled_median = pd.DataFrame(imputer_median.fit_transform(X_missing)) X_filled_median.columns = ['f1','f2','f3'] X_filled_median
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | 5.0 | 2.0 | 3.0 |
| 1 | 6.5 | 2.0 | 4.0 |
| 2 | -3.0 | 2.0 | 1.0 |
| 3 | 8.0 | 2.0 | 4.0 |
| 4 | 10.0 | 2.0 | 5.0 |
imputer_median = SimpleImputer(missing_values=np.NaN, strategy='most_frequent') X_filled_median = pd.DataFrame(imputer_median.fit_transform(X_missing)) X_filled_median.columns = ['f1','f2','f3'] X_filled_median
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | 5.0 | 2.0 | 3.0 |
| 1 | -3.0 | 2.0 | 4.0 |
| 2 | -3.0 | 2.0 | 1.0 |
| 3 | 8.0 | 2.0 | 4.0 |
| 4 | 10.0 | 2.0 | 5.0 |
X_missing_dropped = X_missing.dropna(axis=1) X_missing_dropped
| f3 | |
|---|---|
| 0 | 3.0 |
| 1 | 4.0 |
| 2 | 1.0 |
| 3 | 4.0 |
| 4 | 5.0 |
X_missing_dropped = X_missing.dropna(axis=0).reset_index() X_missing_dropped
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | 5.0 | 2.0 | 3.0 |
| 1 | -3.0 | 2.0 | 1.0 |
X_cat_df = pd.DataFrame( np.array([ ['M', 'O-', 'medium'], ['M', 'O-', 'high'], ['F', 'O+', 'high'], ['F', 'AB', 'low'], ['F', 'B+', 'medium'] ]) ) X_cat_df.columns = ['f1','f2','f3'] X_cat_df
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | M | O- | medium |
| 1 | M | O- | high |
| 2 | F | O+ | high |
| 3 | F | AB | low |
| 4 | F | B+ | medium |
encoder_ord = OrdinalEncoder(dtype='int') X_cat_df.f3 = encoder_ord.fit_transform(X_cat_df.f3.values.reshape(-1, 1)) X_cat_df
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | M | O- | 2 |
| 1 | M | O- | 0 |
| 2 | F | O+ | 0 |
| 3 | F | AB | 1 |
| 4 | F | B+ | 2 |
encoder_lab = LabelEncoder() X_cat_df['f2'] = encoder_lab.fit_transform(X_cat_df['f2']) X_cat_df
| f1 | f2 | f3 | |
|---|---|---|---|
| 0 | M | 3 | 2 |
| 1 | M | 3 | 0 |
| 2 | F | 2 | 0 |
| 3 | F | 0 | 1 |
| 4 | F | 1 | 2 |
encoder_oh = OneHotEncoder(dtype='int') onehot_df = pd.DataFrame( encoder_oh.fit_transform(X_cat_df[['f1']]) .toarray(), columns=['F', 'M'] ) onehot_df['f2'] = X_cat_df.f2 onehot_df['f3'] = X_cat_df.f3 onehot_df
| F | M | f2 | f3 | |
|---|---|---|---|---|
| 0 | 0 | 1 | 3 | 2 |
| 1 | 0 | 1 | 3 | 0 |
| 2 | 1 | 0 | 2 | 0 |
| 3 | 1 | 0 | 0 | 1 |
| 4 | 1 | 0 | 1 | 2 |
| load_iris(*[, return_X_y, as_frame]) | classification | Load and return the iris dataset. |
| load_diabetes(*[, return_X_y, as_frame, scaled]) | regression | Load and return the diabetes dataset. |
| load_digits(*[, n_class, return_X_y, as_frame]) | classification | Load and return the digits dataset. |
| load_linnerud(*[, return_X_y, as_frame]) | multi-output regression | Load and return the physical exercise Linnerud dataset. |
| load_wine(*[, return_X_y, as_frame]) | classification | Load and return the wine dataset. |
| load_breast_cancer(*[, return_X_y, as_frame]) | classification | Load and return the breast cancer wisconsin dataset. |
iris_ds = load_iris() iris_data = iris_ds.data col_names = iris_ds.feature_names target_names = iris_ds.target_names print( 'Iris Dataset', '\n * Data array: ', iris_data.shape, '\n * Column names: ', col_names, '\n * Target names: ', target_names ) # Iris Dataset # * Data array: (150, 4) # * Column names: ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'] # * Target names: ['setosa' 'versicolor' 'virginica']
iris_df = pd.DataFrame(data=iris_data, columns=col_names) iris_df.head()
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | |
|---|---|---|---|---|
| 0 | 5.1 | 3.5 | 1.4 | 0.2 |
| 1 | 4.9 | 3.0 | 1.4 | 0.2 |
| 2 | 4.7 | 3.2 | 1.3 | 0.2 |
| 3 | 4.6 | 3.1 | 1.5 | 0.2 |
| 4 | 5.0 | 3.6 | 1.4 | 0.2 |
| fetch_olivetti_faces(*[, data_home, ...]) | classification | Load the Olivetti faces data-set from AT&T. |
| fetch_20newsgroups(*[, data_home, subset, ...]) | classification | Load the filenames and data from the 20 newsgroups dataset. |
| fetch_20newsgroups_vectorized(*[, subset, ...]) | classification | Load and vectorize the 20 newsgroups dataset. |
| fetch_lfw_people(*[, data_home, funneled, ...]) | classification | Load the Labeled Faces in the Wild (LFW) people dataset. |
| fetch_lfw_pairs(*[, subset, data_home, ...]) | classification | Load the Labeled Faces in the Wild (LFW) pairs dataset. |
| fetch_covtype(*[, data_home, ...]) | classification | Load the covertype dataset. |
| fetch_rcv1(*[, data_home, subset, ...]) | classification | Load the RCV1 multilabel dataset. |
| fetch_kddcup99(*[, subset, data_home, ...]) | classification | Load the kddcup99 dataset. |
| fetch_california_housing(*[, data_home, ...]) | regression | Load the California housing dataset. |
newsgroups_train = fetch_20newsgroups(subset='train') train_data = newsgroups_train.data col_names = newsgroups_train.filenames.shape target_names = newsgroups_train.target.shape print( 'Newsgroup - Train Subset', '\n * Data array: ', len(train_data), '\n * Column names: ', col_names, '\n * Target names: ', target_names ) # Newsgroup - Train Subset # * Data array: 11314 # * Column names: (11314,) # * Target names: (11314,)
print('Target Names: ', newsgroups_train.target_names) # Target Names: ['alt.atheism', 'comp.graphics', 'comp.os.ms-windows.misc', 'comp.sys.ibm.pc.hardware', 'comp.sys.mac.hardware', 'comp.windows.x', 'misc.forsale', 'rec.autos', 'rec.motorcycles', 'rec.sport.baseball', 'rec.sport.hockey', 'sci.crypt', 'sci.electronics', 'sci.med', 'sci.space', 'soc.religion.christian', 'talk.politics.guns', 'talk.politics.mideast', 'talk.politics.misc', 'talk.religion.misc']
mice_ds = fetch_openml(name='miceprotein', version=4, parser="auto")
print( 'Mice Protein Dataset', '\n * Data Shape: ', mice_ds.data.shape, '\n * Target Shape: ', mice_ds.target.shape, '\n * Target Names: ', np.unique(mice_ds.target) ) # Mice Protein Dataset # * Data Shape: (1080, 77) # * Target Shape: (1080,) # * Target Names: ['c-CS-m' 'c-CS-s' 'c-SC-m' 'c-SC-s' 't-CS-m' 't-CS-s' 't-SC-m' 't-SC-s']
print(mice_ds.DESCR)
iris_df.plot( figsize=(12,5), kind='scatter', x='sepal length (cm)', y='sepal width (cm)', title='Iris Dataset :: Sepal Width&Height' ) print(iris_df.corr())
The Sepal Width has very little correlation to all other metrics but itself. While the other three correlate nicely:
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | |
|---|---|---|---|---|
| sepal length (cm) | 1.000000 | -0.117570 | 0.871754 | 0.817941 |
| sepal width (cm) | -0.117570 | 1.000000 | -0.428440 | -0.366126 |
| petal length (cm) | 0.871754 | -0.428440 | 1.000000 | 0.962865 |
| petal width (cm) | 0.817941 | -0.366126 | 0.962865 | 1.000000 |
iris_df['petal length (cm)'][:1] # 0 1.4 # Name: petal length (cm), dtype: float64
iris_df['petal length (cm)'].values.reshape(-1,1)[:1] # array([[1.4]])
# scikit expects a 2s imput => remove index X = iris_df['petal length (cm)'].values.reshape(-1,1) y = iris_df['petal width (cm)'].values.reshape(-1,1)
# train/test split X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2) print(X_train.shape, X_test.shape) # (120, 1) (30, 1) 80:20 split
regressor = LinearRegression() regressor.fit(X_train,y_train) intercept = regressor.intercept_ slope = regressor.coef_ print(' Intercept: ', intercept, '\n Slope: ', slope) # Intercept: [-0.35135666] # Correlation Coeficient: [[0.41310505]]
y_pred = regressor.predict([X_test[0]]) print(' Prediction: ', y_pred, '\n True Value: ', y_test[0]) # Prediction: [[0.22699041]] # True Value: [0.2]
def predict(value): return (slope*value + intercept)[0][0]
print('Prediction: ', predict(X_test[0])) # Prediction: [[0.22699041]]
iris_df['petal width (cm) prediction'] = iris_df['petal length (cm)'].apply(predict) print(' Prediction: ', iris_df['petal width (cm) prediction'][0], '\n True Value: ', iris_df['petal width (cm)'][0]) # Prediction: 0.22699041280334376 # True Value: 0.2
iris_df.head(10)
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | petal width (cm) prediction | |
|---|---|---|---|---|---|
| 0 | 5.1 | 3.5 | 1.4 | 0.2 | 0.226990 |
| 1 | 4.9 | 3.0 | 1.4 | 0.2 | 0.226990 |
| 2 | 4.7 | 3.2 | 1.3 | 0.2 | 0.185680 |
| 3 | 4.6 | 3.1 | 1.5 | 0.2 | 0.268301 |
| 4 | 5.0 | 3.6 | 1.4 | 0.2 | 0.226990 |
| 5 | 5.4 | 3.9 | 1.7 | 0.4 | 0.350922 |
| 6 | 4.6 | 3.4 | 1.4 | 0.3 | 0.226990 |
| 7 | 5.0 | 3.4 | 1.5 | 0.2 | 0.268301 |
| 8 | 4.4 | 2.9 | 1.4 | 0.2 | 0.226990 |
| 9 | 4.9 | 3.1 | 1.5 | 0.1 | 0.268301 |
iris_df.plot( figsize=(12,5), kind='scatter', x='petal width (cm)', y='petal width (cm) prediction', # no value in colorizing..just looks pretty c='petal width (cm) prediction', colormap='summer', title='Iris Dataset - Sepal Width True vs Prediction' )
mae = mean_absolute_error( iris_df['petal width (cm)'], iris_df['petal width (cm) prediction'] ) mse = mean_squared_error( iris_df['petal width (cm)'], iris_df['petal width (cm) prediction'] ) rmse = np.sqrt(mse) print(' MAE: ', mae, '\n MSE: ', mse, '\n RMSE: ', rmse) # MAE: 0.1569441318761155 # MSE: 0.04209214667485277 # RMSE: 0.2051637070118708
!wget https://raw.githubusercontent.com/Satish-Vennapu/DataScience/main/AMES_Final_DF.csv -P datasets
ames_df = pd.read_csv('datasets/AMES_Final_DF.csv') ames_df.head(5).transpose()
| 0 | 1 | 2 | 3 | 4 | |
|---|---|---|---|---|---|
| Lot Frontage | 141.0 | 80.0 | 81.0 | 93.0 | 74.0 |
| Lot Area | 31770.0 | 11622.0 | 14267.0 | 11160.0 | 13830.0 |
| Overall Qual | 6.0 | 5.0 | 6.0 | 7.0 | 5.0 |
| Overall Cond | 5.0 | 6.0 | 6.0 | 5.0 | 5.0 |
| Year Built | 1960.0 | 1961.0 | 1958.0 | 1968.0 | 1997.0 |
| ... | |||||
| Sale Condition_AdjLand | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Sale Condition_Alloca | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Sale Condition_Family | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Sale Condition_Normal | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| Sale Condition_Partial | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 274 rows × 5 columns |
# the target value is: ames_df['SalePrice']
| 0 | 215000 |
| 1 | 105000 |
| 2 | 172000 |
| 3 | 244000 |
| 4 | 189900 |
| ... | |
| 2920 | 142500 |
| 2921 | 131000 |
| 2922 | 132000 |
| 2923 | 170000 |
| 2924 | 188000 |
| Name: SalePrice, Length: 2925, dtype: int64 |
# remove target column from training dataset X_ames = ames_df.drop('SalePrice', axis=1) y_ames = ames_df['SalePrice'] print(X_ames.shape, y_ames.shape) # (2925, 273) (2925,)
# train/test split X_ames_train, X_ames_test, y_ames_train, y_ames_test = train_test_split( X_ames, y_ames, test_size=0.1, random_state=101 ) print(X_ames_train.shape, X_ames_test.shape) # (2632, 273) (293, 273)
# normalize feature set scaler = StandardScaler() X_ames_train_scaled = scaler.fit_transform(X_ames_train) X_ames_test_scaled = scaler.transform(X_ames_test)
base_ames_elastic_net_model = ElasticNet(max_iter=int(1e4))
param_grid = { 'alpha': [50, 75, 100, 125, 150], 'l1_ratio':[0.2, 0.4, 0.6, 0.8, 1.0] }
grid_ames_model = GridSearchCV( estimator=base_ames_elastic_net_model, param_grid=param_grid, scoring='neg_mean_squared_error', cv=5, verbose=1 ) grid_ames_model.fit(X_ames_train_scaled, y_ames_train) print( 'Results:\nBest Estimator: ', grid_ames_model.best_estimator_, '\nBest Hyperparameter: ', grid_ames_model.best_params_ )
Results:
ElasticNet(alpha=125, l1_ratio=1.0, max_iter=10000){'alpha': 125, 'l1_ratio': 1.0}y_ames_pred = grid_ames_model.predict(X_ames_test_scaled) print( 'MAE: ', mean_absolute_error(y_ames_test, y_ames_pred), 'MSE: ', mean_squared_error(y_ames_test, y_ames_pred), 'RMSE: ', np.sqrt(mean_squared_error(y_ames_test, y_ames_pred)) ) # MAE: 14185.506207185055 MSE: 422714457.5190704 RMSE: 20560.020854052418
# average SalePrize np.mean(ames_df['SalePrice']) # 180815.53743589742 rel_error_avg = mean_absolute_error(y_ames_test, y_ames_pred) * 100 / np.mean(ames_df['SalePrice']) print('Pridictions are on average off by: ', rel_error_avg.round(2), '%') # Pridictions are on average off by: 7.85 %
plt.figure(figsize=(10,4)) plt.scatter(y_ames_test,y_ames_pred, c='mediumspringgreen', s=3) plt.axline((0, 0), slope=1, color='dodgerblue', linestyle=(':')) plt.title('Prediction Accuracy :: MAE:'+ str(mean_absolute_error(y_ames_test, y_ames_pred).round(2)) + 'US$') plt.xlabel('True Sales Price') plt.ylabel('Predicted Sales Price') plt.savefig('assets/Scikit_Learn_11.webp', bbox_inches='tight')
Above I used the petal width and length to create a linear regression model. But as explored earlier we can also use the sepal length (only the sepal width does not show a linear correlation):
print(iris_df.corr())
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | |
|---|---|---|---|---|
| sepal length (cm) | 1.000000 | -0.117570 | 0.871754 | 0.817941 |
| sepal width (cm) | -0.117570 | 1.000000 | -0.428440 | -0.366126 |
| petal length (cm) | 0.871754 | -0.428440 | 1.000000 | 0.962865 |
| petal width (cm) | 0.817941 | -0.366126 | 0.962865 | 1.000000 |
X_multi = iris_df[['petal length (cm)', 'sepal length (cm)']] y = iris_df['petal width (cm)']
regressor_multi = LinearRegression() regressor_multi.fit(X_multi, y) intercept_multi = regressor_multi.intercept_ slope_multi = regressor_multi.coef_ print(' Intercept: ', intercept_multi, '\n Slope: ', slope_multi) # Intercept: -0.00899597269816943 # Slope: [ 0.44937611 -0.08221782]
def predict_multi(petal_length, sepal_length): return (slope_multi[0]*petal_length + slope_multi[1]*sepal_length + intercept_multi)
y_pred = predict_multi( iris_df['petal length (cm)'][0], iris_df['sepal length (cm)'][0] ) print(' Prediction: ', y_pred, '\n True value: ', iris_df['petal width (cm)'][0]) # Prediction: 0.20081970121763193 # True value: 0.2
iris_df['petal width (cm) prediction (multi)'] = ( ( slope_multi[0] * iris_df['petal length (cm)'] ) + ( slope_multi[1] * iris_df['sepal length (cm)'] ) + ( intercept_multi ) )
iris_df.head(10)
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) | petal width (cm) prediction | petal width (cm) prediction (multi) | |
|---|---|---|---|---|---|---|
| 0 | 5.1 | 3.5 | 1.4 | 0.2 | 0.226990 | 0.200820 |
| 1 | 4.9 | 3.0 | 1.4 | 0.2 | 0.226990 | 0.217263 |
| 2 | 4.7 | 3.2 | 1.3 | 0.2 | 0.185680 | 0.188769 |
| 3 | 4.6 | 3.1 | 1.5 | 0.2 | 0.268301 | 0.286866 |
| 4 | 5.0 | 3.6 | 1.4 | 0.2 | 0.226990 | 0.209041 |
| 5 | 5.4 | 3.9 | 1.7 | 0.4 | 0.350922 | 0.310967 |
| 6 | 4.6 | 3.4 | 1.4 | 0.3 | 0.226990 | 0.241929 |
| 7 | 5.0 | 3.4 | 1.5 | 0.2 | 0.268301 | 0.253979 |
| 8 | 4.4 | 2.9 | 1.4 | 0.2 | 0.226990 | 0.258372 |
| 9 | 4.9 | 3.1 | 1.5 | 0.1 | 0.268301 | 0.262201 |
iris_df.plot( figsize=(12,5), kind='scatter', x='petal width (cm)', y='petal width (cm) prediction (multi)', c='petal width (cm) prediction', colormap='summer', title='Iris Dataset - Sepal Width True vs Prediction (multi)' )
mae_multi = mean_absolute_error( iris_df['petal width (cm)'], iris_df['petal width (cm) prediction (multi)'] ) mse_multi = mean_squared_error( iris_df['petal width (cm)'], iris_df['petal width (cm) prediction (multi)'] ) rmse_multi = np.sqrt(mse_multi) print(' MAE_Multi: ', mae_multi,' MAE: ', mae, '\n MSE_Multi: ', mse_multi, ' MSE: ', mse, '\n RMSE_Multi: ', rmse_multi, ' RMSE: ', rmse)
The accuracy of the model was improved by adding an additional, correlating value:
| Multi Regression | Single Regression | |
|---|---|---|
| Mean Absolute Error | 0.15562108079300102 | 0.1569441318761155 |
| Mean Squared Error | 0.04096208526408982 | 0.04209214667485277 |
| Root Mean Squared Error | 0.20239092189149646 | 0.2051637070118708 |
np.random.seed(666) # generate 10 index values between 0-10 x_data_logistic_binary = np.random.randint(10, size=(10)).reshape(-1, 1) # generate binary category for values above y_data_logistic_binary = np.random.randint(2, size=10)
logistic_binary_model = LogisticRegression( solver='liblinear', C=10.0, random_state=0 ) logistic_binary_model.fit(x_data_logistic_binary, y_data_logistic_binary) intercept_logistic_binary = logistic_binary_model.intercept_ slope_logistic_binary = logistic_binary_model.coef_ print(' Intercept: ', intercept_logistic_binary, '\n Slope: ', slope_logistic_binary) # Intercept: [-0.4832956] # Slope: [[0.11180522]]
prob_pred_logistic_binary = logistic_binary_model.predict_proba(x_data_logistic_binary) y_pred_logistic_binary = logistic_binary_model.predict(x_data_logistic_binary) print('Prediction Probabilities: ', prob_pred[:1]) unique, counts = np.unique(y_pred_logistic_binary, return_counts=True) print('Classes: ', unique, '| Number of Class Instances: ', counts) # probabilities e.g. below -> 58% certainty that the first element is class 0 # Prediction Probabilities: [[0.58097284 0.41902716]] # Classes: [0 1] | Number of Class Instances: [5 5]
conf_mtx = confusion_matrix(y_data_logistic_binary, y_pred_logistic_binary) conf_mtx # [2, 3] [TP, FP] # [3, 2] [FN, TN]
report = classification_report(y_data_logistic_binary, y_pred_logistic_binary) print(report)
| precision | recall | f1-score | support | |
|---|---|---|---|---|
| 0 | 0.40 | 0.40 | 0.40 | 5 |
| 1 | 0.40 | 0.40 | 0.40 | 5 |
| accuracy | 0.40 | 10 | ||
| macro avg | 0.40 | 0.40 | 0.40 | 10 |
| weighted avg | 0.40 | 0.40 | 0.40 | 10 |
iris_ds = load_iris() # train/test split X_train_iris, X_test_iris, y_train_iris, y_test_iris = train_test_split( iris_ds.data, iris_ds.target, test_size=0.2, random_state=42 ) print(X_train_iris.shape, X_test_iris.shape) # (120, 4) (30, 4)
pipe_iris = Pipeline([ ('minmax', MinMaxScaler()), ('log_reg', LogisticRegression()), ]) pipe_iris.fit(X_train_iris, y_train_iris)
iris_score = pipe_iris.score(X_test_iris, y_test_iris) print('Prediction Accuracy: ', iris_score.round(4)*100, '%') # Prediction Accuracy: 96.67 %
!wget https://raw.githubusercontent.com/reisanar/datasets/master/Advertising.csv -P datasets
adv_df = pd.read_csv('datasets/Advertising.csv') adv_df.head(5)
| TV | Radio | Newspaper | Sales | |
|---|---|---|---|---|
| 0 | 230.1 | 37.8 | 69.2 | 22.1 |
| 1 | 44.5 | 39.3 | 45.1 | 10.4 |
| 2 | 17.2 | 45.9 | 69.3 | 9.3 |
| 3 | 151.5 | 41.3 | 58.5 | 18.5 |
| 4 | 180.8 | 10.8 | 58.4 | 12.9 |
# Split ds into features and targets X_adv = adv_df.drop('Sales', axis=1) y_adv = adv_df['Sales']
# 70:30 train/test split X_adv_train, X_adv_test, y_adv_train, y_adv_test = train_test_split( X_adv, y_adv, test_size=0.3, random_state=666 ) print(X_adv_train.shape, y_adv_train.shape) # (140, 3) (140,)
# normalize features scaler_adv = StandardScaler() scaler_adv.fit(X_adv_train) X_adv_train = scaler_adv.transform(X_adv_train) X_adv_test = scaler_adv.transform(X_adv_test)
model_adv1 = Ridge( alpha=100.0 ) model_adv1.fit(X_adv_train, y_adv_train)
y_adv_pred = model_adv1.predict(X_adv_test) mean_squared_error(y_adv_test, y_adv_pred) # 6.528575771818745
model_adv2 = Ridge( alpha=1.0 ) model_adv2.fit(X_adv_train, y_adv_train)
y_adv_pred2 = model_adv2.predict(X_adv_test) mean_squared_error(y_adv_test, y_adv_pred2) # 2.3319016551123535
# 70:30 train/temp split X_adv_train, X_adv_temp, y_adv_train, y_adv_temp = train_test_split( X_adv, y_adv, test_size=0.3, random_state=666 ) # 50:50 test/val split X_adv_test, X_adv_val, y_adv_test, y_adv_val = train_test_split( X_adv_temp, y_adv_temp, test_size=0.5, random_state=666 ) print(X_adv_train.shape, X_adv_test.shape, X_adv_val.shape) # (140, 3) (30, 3) (30, 3)
# normalize features scaler_adv = StandardScaler() scaler_adv.fit(X_adv_train) X_adv_train = scaler_adv.transform(X_adv_train) X_adv_test = scaler_adv.transform(X_adv_test) X_adv_val = scaler_adv.transform(X_adv_val)
model_adv3 = Ridge( alpha=100.0 ) model_adv3.fit(X_adv_train, y_adv_train)
# do evaluation with the validation set y_adv_pred3 = model_adv3.predict(X_adv_val) mean_squared_error(y_adv_val, y_adv_pred3) # 7.136230975501291
model_adv4 = Ridge( alpha=1.0 ) model_adv4.fit(X_adv_train, y_adv_train) y_adv_pred4 = model_adv4.predict(X_adv_val) mean_squared_error(y_adv_val, y_adv_pred4) # 2.6393803874124435
# only once you are certain that you have the best performance # do a final evaluation with the test set y_adv4_final_pred = model_adv4.predict(X_adv_test) mean_squared_error(y_adv_test, y_adv4_final_pred) # 2.024422922812264
Do a train/test split and segment the training set by k-folds (e.g. 5-10) and use each of those segments once to validate a training step. The resulting error is the average of all k errors.
# 70:30 train/temp split X_adv_train, X_adv_test, y_adv_train, y_adv_test = train_test_split( X_adv, y_adv, test_size=0.3, random_state=666 )
# normalize features scaler_adv = StandardScaler() scaler_adv.fit(X_adv_train) X_adv_train = scaler_adv.transform(X_adv_train) X_adv_test = scaler_adv.transform(X_adv_test)
model_adv5 = Ridge( alpha=100.0 )
# do a 5-fold cross-eval scores = cross_val_score( estimator=model_adv5, X=X_adv_train, y=y_adv_train, scoring='neg_mean_squared_error', cv=5 ) # take the mean of all five neg. error values abs(scores.mean()) # 8.688107513529168
model_adv6 = Ridge( alpha=1.0 )
# do a 5-fold cross-eval scores = cross_val_score( estimator=model_adv6, X=X_adv_train, y=y_adv_train, scoring='neg_mean_squared_error', cv=5 ) # take the mean of all five neg. error values abs(scores.mean()) # 3.3419582340688576
model_adv6.fit(X_adv_train, y_adv_train) y_adv6_final_pred = model_adv6.predict(X_adv_test) mean_squared_error(y_adv_test, y_adv6_final_pred) # 2.3319016551123535
adv_df = pd.read_csv('datasets/Advertising.csv') X_adv = adv_df.drop('Sales', axis=1) y_adv = adv_df['Sales']
# 70:30 train/test split X_adv_train, X_adv_test, y_adv_train, y_adv_test = train_test_split( X_adv, y_adv, test_size=0.3, random_state=666 )
# normalize features scaler_adv = StandardScaler() scaler_adv.fit(X_adv_train) X_adv_train = scaler_adv.transform(X_adv_train) X_adv_test = scaler_adv.transform(X_adv_test)
model_adv7 = Ridge( alpha=100.0 )
scores = cross_validate( model_adv7, X_adv_train, y_adv_train, scoring=[ 'neg_mean_squared_error', 'neg_mean_absolute_error' ], cv=10 )
scores_df = pd.DataFrame(scores) scores_df
| fit_time | score_time | test_neg_mean_squared_error | test_neg_mean_absolute_error | |
|---|---|---|---|---|
| 0 | 0.016399 | 0.000749 | -12.539147 | -2.851864 |
| 1 | 0.000684 | 0.000452 | -2.806466 | -1.423516 |
| 2 | 0.000937 | 0.000782 | -11.142227 | -2.740332 |
| 3 | 0.001060 | 0.000633 | -7.237347 | -2.196963 |
| 4 | 0.001045 | 0.000738 | -11.313985 | -2.690813 |
| 5 | 0.000650 | 0.000510 | -3.169169 | -1.526568 |
| 6 | 0.000698 | 0.000429 | -6.578249 | -1.727616 |
| 7 | 0.000600 | 0.000423 | -5.740245 | -1.640964 |
| 8 | 0.000565 | 0.000463 | -10.268075 | -2.415688 |
| 9 | 0.000562 | 0.000487 | -10.641669 | -1.974407 |
abs(scores_df.mean())
| fit_time | 0.002320 |
| score_time | 0.000566 |
| test_neg_mean_squared_error | 8.143658 |
| test_neg_mean_absolute_error | 2.118873 |
| dtype: float64 |
model_adv8 = Ridge( alpha=1.0 )
scores = cross_validate( model_adv8, X_adv_train, y_adv_train, scoring=[ 'neg_mean_squared_error', 'neg_mean_absolute_error' ], cv=10 ) abs(pd.DataFrame(scores).mean())
| fit_time | 0.001141 |
| score_time | 0.000777 |
| test_neg_mean_squared_error | 3.272673 |
| test_neg_mean_absolute_error | 1.345709 |
| dtype: float64 |
model_adv8.fit(X_adv_train, y_adv_train) y_adv8_final_pred = model_adv8.predict(X_adv_test) mean_squared_error(y_adv_test, y_adv8_final_pred) # 2.3319016551123535
Loop through a set of hyperparameters to find an optimum.
base_elastic_net_model = ElasticNet()
param_grid = { 'alpha': [0.1, 1, 5, 10, 50, 100], 'l1_ratio':[0.1, 0.3, 0.5, 0.7, 0.9, 1.0] }
grid_model = GridSearchCV( estimator=base_elastic_net_model, param_grid=param_grid, scoring='neg_mean_squared_error', cv=5, verbose=2 ) grid_model.fit(X_adv_train, y_adv_train) print( 'Results:\nBest Estimator: ', grid_model.best_estimator_, '\nBest Hyperparameter: ', grid_model.best_params_ )
Results:
ElasticNet(alpha=0.1, l1_ratio=1.0){'alpha': 0.1, 'l1_ratio': 1.0}gridcv_results = pd.DataFrame(grid_model.cv_results_)
| mean_fit_time | std_fit_time | mean_score_time | std_score_time | param_alpha | param_l1_ratio | params | split0_test_score | split1_test_score | split2_test_score | split3_test_score | split4_test_score | mean_test_score | std_test_score | rank_test_score | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.001156 | 0.000160 | 0.000449 | 0.000038 | 0.1 | 0.1 | {'alpha': 0.1, 'l1_ratio': 0.1} | -1.924119 | -3.384152 | -3.588444 | -3.703040 | -5.091974 | -3.538346 | 1.007264 | 6 |
| 1 | 0.001144 | 0.000181 | 0.000407 | 0.000091 | 0.1 | 0.3 | {'alpha': 0.1, 'l1_ratio': 0.3} | -1.867117 | -3.304382 | -3.561106 | -3.623188 | -5.061781 | -3.483515 | 1.016000 | 5 |
| 2 | 0.000623 | 0.000026 | 0.000272 | 0.000052 | 0.1 | 0.5 | {'alpha': 0.1, 'l1_ratio': 0.5} | -1.812633 | -3.220727 | -3.539711 | -3.547572 | -5.043259 | -3.432780 | 1.028406 | 4 |
| 3 | 0.000932 | 0.000165 | 0.000321 | 0.000060 | 0.1 | 0.7 | {'alpha': 0.1, 'l1_ratio': 0.7} | -1.750153 | -3.144120 | -3.525226 | -3.477228 | -5.034008 | -3.386147 | 1.046722 | 3 |
| 4 | 0.000725 | 0.000106 | 0.000259 | 0.000024 | 0.1 | 0.9 | {'alpha': 0.1, 'l1_ratio': 0.9} | -1.693440 | -3.075686 | -3.518777 | -3.413393 | -5.029683 | -3.346196 | 1.065195 | 2 |
| 5 | 0.000654 | 0.000053 | 0.000274 | 0.000026 | 0.1 | 1.0 | {'alpha': 0.1, 'l1_ratio': 1.0} | -1.667506 | -3.044928 | -3.518866 | -3.384363 | -5.031297 | -3.329392 | 1.075006 | 1 |
| 6 | 0.000595 | 0.000016 | 0.000244 | 0.000002 | 1 | 0.1 | {'alpha': 1, 'l1_ratio': 0.1} | -8.575470 | -11.021534 | -8.212152 | -6.808719 | -10.792072 | -9.081990 | 1.604192 | 12 |
| 7 | 0.000591 | 0.000018 | 0.000244 | 0.000002 | 1 | 0.3 | {'alpha': 1, 'l1_ratio': 0.3} | -8.131855 | -10.448423 | -7.774620 | -6.179358 | -10.071728 | -8.521197 | 1.569173 | 11 |
| 8 | 0.000628 | 0.000049 | 0.000266 | 0.000023 | 1 | 0.5 | {'alpha': 1, 'l1_ratio': 0.5} | -7.519809 | -9.562473 | -7.261824 | -5.453399 | -9.213320 | -7.802165 | 1.481785 | 10 |
| 9 | 0.000594 | 0.000015 | 0.000243 | 0.000002 | 1 | 0.7 | {'alpha': 1, 'l1_ratio': 0.7} | -6.614835 | -8.351711 | -6.702104 | -4.698977 | -8.230616 | -6.919649 | 1.329741 | 9 |
| 10 | 0.000714 | 0.000108 | 0.000268 | 0.000033 | 1 | 0.9 | {'alpha': 1, 'l1_ratio': 0.9} | -5.537250 | -6.887828 | -6.148400 | -4.106124 | -7.101573 | -5.956235 | 1.078430 | 8 |
| 11 | 0.000649 | 0.000067 | 0.000263 | 0.000028 | 1 | 1.0 | {'alpha': 1, 'l1_ratio': 1.0} | -4.932027 | -6.058207 | -5.892529 | -3.798441 | -6.472871 | -5.430815 | 0.959804 | 7 |
| 12 | 0.000645 | 0.000042 | 0.000264 | 0.000040 | 5 | 0.1 | {'alpha': 5, 'l1_ratio': 0.1} | -21.863798 | -25.767488 | -18.768865 | -12.608680 | -23.207907 | -20.443347 | 4.520904 | 13 |
| 13 | 0.000617 | 0.000030 | 0.000281 | 0.000038 | 5 | 0.3 | {'alpha': 5, 'l1_ratio': 0.3} | -23.626694 | -27.439028 | -20.266203 | -12.788078 | -24.609195 | -21.745840 | 5.031493 | 14 |
| 14 | 0.000599 | 0.000011 | 0.000249 | 0.000013 | 5 | 0.5 | {'alpha': 5, 'l1_ratio': 0.5} | -26.202964 | -29.867138 | -22.527913 | -13.423857 | -26.835934 | -23.771561 | 5.675911 | 15 |
| 15 | 0.000588 | 0.000013 | 0.000276 | 0.000035 | 5 | 0.7 | {'alpha': 5, 'l1_ratio': 0.7} | -27.768946 | -33.428462 | -23.506474 | -14.599984 | -29.112276 | -25.683228 | 6.382379 | 17 |
| 16 | 0.000580 | 0.000003 | 0.000271 | 0.000001 | 5 | 0.9 | {'alpha': 5, 'l1_ratio': 0.9} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 17 | 0.000591 | 0.000011 | 0.000259 | 0.000021 | 5 | 1.0 | {'alpha': 5, 'l1_ratio': 1.0} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 18 | 0.000632 | 0.000028 | 0.000250 | 0.000012 | 10 | 0.1 | {'alpha': 10, 'l1_ratio': 0.1} | -26.179546 | -30.396420 | -22.386698 | -14.596498 | -27.292337 | -24.170300 | 5.429322 | 16 |
| 19 | 0.000593 | 0.000020 | 0.000239 | 0.000001 | 10 | 0.3 | {'alpha': 10, 'l1_ratio': 0.3} | -28.704426 | -33.379967 | -24.561645 | -15.634153 | -29.883725 | -26.432783 | 6.090062 | 18 |
| 20 | 0.000595 | 0.000036 | 0.000245 | 0.000013 | 10 | 0.5 | {'alpha': 10, 'l1_ratio': 0.5} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 21 | 0.000610 | 0.000053 | 0.000258 | 0.000015 | 10 | 0.7 | {'alpha': 10, 'l1_ratio': 0.7} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 22 | 0.000597 | 0.000022 | 0.000248 | 0.000015 | 10 | 0.9 | {'alpha': 10, 'l1_ratio': 0.9} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 23 | 0.000623 | 0.000057 | 0.000305 | 0.000076 | 10 | 1.0 | {'alpha': 10, 'l1_ratio': 1.0} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 24 | 0.000602 | 0.000016 | 0.000252 | 0.000013 | 50 | 0.1 | {'alpha': 50, 'l1_ratio': 0.1} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 25 | 0.000577 | 0.000009 | 0.000238 | 0.000001 | 50 | 0.3 | {'alpha': 50, 'l1_ratio': 0.3} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 26 | 0.000607 | 0.000046 | 0.000245 | 0.000010 | 50 | 0.5 | {'alpha': 50, 'l1_ratio': 0.5} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 27 | 0.000569 | 0.000004 | 0.000259 | 0.000012 | 50 | 0.7 | {'alpha': 50, 'l1_ratio': 0.7} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 28 | 0.000582 | 0.000022 | 0.000244 | 0.000011 | 50 | 0.9 | {'alpha': 50, 'l1_ratio': 0.9} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 29 | 0.000603 | 0.000041 | 0.000251 | 0.000015 | 50 | 1.0 | {'alpha': 50, 'l1_ratio': 1.0} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 30 | 0.000670 | 0.000106 | 0.000251 | 0.000013 | 100 | 0.1 | {'alpha': 100, 'l1_ratio': 0.1} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 31 | 0.000764 | 0.000179 | 0.000343 | 0.000054 | 100 | 0.3 | {'alpha': 100, 'l1_ratio': 0.3} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 32 | 0.000623 | 0.000077 | 0.000244 | 0.000007 | 100 | 0.5 | {'alpha': 100, 'l1_ratio': 0.5} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 33 | 0.000817 | 0.000156 | 0.000329 | 0.000076 | 100 | 0.7 | {'alpha': 100, 'l1_ratio': 0.7} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 34 | 0.000590 | 0.000017 | 0.000242 | 0.000004 | 100 | 0.9 | {'alpha': 100, 'l1_ratio': 0.9} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
| 35 | 0.000595 | 0.000027 | 0.000242 | 0.000007 | 100 | 1.0 | {'alpha': 100, 'l1_ratio': 1.0} | -29.868949 | -34.423737 | -25.623955 | -16.750237 | -31.056181 | -27.544612 | 6.087093 | 19 |
gridcv_results[ [ 'param_alpha', 'param_l1_ratio' ] ].plot(title='Grid Search Hyperparameter :: Parameter', figsize=(12,8))
gridcv_results[ [ 'mean_fit_time', 'std_fit_time', 'mean_score_time' ] ].plot(title='Grid Search Hyperparameter :: Timing', figsize=(12,8))
gridcv_results[ [ 'split0_test_score', 'split1_test_score', 'split2_test_score', 'split3_test_score', 'split4_test_score', 'mean_test_score', 'std_test_score', 'rank_test_score' ] ].plot(title='Grid Search Hyperparameter :: Parameter', figsize=(12,8))
y_grid_pred = grid_model.predict(X_adv_test) mean_squared_error(y_adv_test, y_grid_pred) # 2.380865536033581
wine = load_wine() print(wine.data.shape) print(wine.feature_names) print(wine.data[:1]) # (178, 13) # ['alcohol', 'malic_acid', 'ash', 'alcalinity_of_ash', 'magnesium', 'total_phenols', 'flavanoids', 'nonflavanoid_phenols', 'proanthocyanins', 'color_intensity', 'hue', 'od280/od315_of_diluted_wines', 'proline'] # [[1.423e+01 1.710e+00 2.430e+00 1.560e+01 1.270e+02 2.800e+00 3.060e+00 # 2.800e-01 2.290e+00 5.640e+00 1.040e+00 3.920e+00 1.065e+03]]
wine_df = pd.DataFrame(data=wine.data, columns=wine.feature_names) wine_df.head(2).T
| 0 | 1 | |
|---|---|---|
| alcohol | 14.23 | 13.20 |
| malic_acid | 1.71 | 1.78 |
| ash | 2.43 | 2.14 |
| alcalinity_of_ash | 15.60 | 11.20 |
| magnesium | 127.00 | 100.00 |
| total_phenols | 2.80 | 2.65 |
| flavanoids | 3.06 | 2.76 |
| nonflavanoid_phenols | 0.28 | 0.26 |
| proanthocyanins | 2.29 | 1.28 |
| color_intensity | 5.64 | 4.38 |
| hue | 1.04 | 1.05 |
| od280/od315_of_diluted_wines | 3.92 | 3.40 |
| proline | 1065.00 | 1050.00 |
# normalization scaler = MinMaxScaler() scaler.fit(wine.data) wine_norm = scaler.fit_transform(wine.data)
# train/test split X_train_wine, X_test_wine, y_train_wine, y_test_wine = train_test_split( wine_norm, wine.target, test_size=0.3 ) print(X_train_wine.shape, X_test_wine.shape) # (124, 13) (54, 13)
# model for k=3 knn = KNeighborsClassifier(n_neighbors=3) knn.fit(X_train_wine, y_train_wine) y_pred_wine_knn3 = knn.predict(X_test_wine) print('Accuracy Score: ', (accuracy_score(y_test_wine, y_pred_wine_knn3)*100).round(2), '%') # Accuracy Score: 98.15 %
# model for k=5 knn = KNeighborsClassifier(n_neighbors=5) knn.fit(X_train_wine, y_train_wine) y_pred_wine_knn5 = knn.predict(X_test_wine) print('Accuracy Score: ', (accuracy_score(y_test_wine, y_pred_wine_knn5)*100).round(2), '%') # Accuracy Score: 98.15 %
# model for k=7 knn = KNeighborsClassifier(n_neighbors=7) knn.fit(X_train_wine, y_train_wine) y_pred_wine_knn7 = knn.predict(X_test_wine) print('Accuracy Score: ', (accuracy_score(y_test_wine, y_pred_wine_knn7)*100).round(2), '%') # Accuracy Score: 96.3 %
# model for k=9 knn = KNeighborsClassifier(n_neighbors=7) knn.fit(X_train_wine, y_train_wine) y_pred_wine_knn7 = knn.predict(X_test_wine) print('Accuracy Score: ', (accuracy_score(y_test_wine, y_pred_wine_knn7)*100).round(2), '%') # Accuracy Score: 96.3 %
!wget https://gist.githubusercontent.com/Dviejopomata/ea5869ba4dcff84f8c294dc7402cd4a9/raw/4671f90b8b04ba4db9d67acafaa4c0827cd233c2/bill_authentication.csv -P datasets
bill_auth_df = pd.read_csv('datasets/bill_authentication.csv') bill_auth_df.head(3)
| Variance | Skewness | Curtosis | Entropy | Class | |
|---|---|---|---|---|---|
| 0 | 3.6216 | 8.6661 | -2.8073 | -0.44699 | 0 |
| 1 | 4.5459 | 8.1674 | -2.4586 | -1.46210 | 0 |
| 2 | 3.8660 | -2.6383 | 1.9242 | 0.10645 | 0 |
# remove target feature from training set X_bill = bill_auth_df.drop('Class', axis=1) y_bill = bill_auth_df['Class']
mpolinowski/python-scikitlearn-cheatsheet is a public repo on GitHub by mpolinowski. SciKit Learn Machine Learning Cheat Sheet.
It declares no license on GitHub. Public does not mean open source: without a license, nobody is granted the right to reuse it.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2023-06-17. Take My Repo does not judge from that whether it is maintained.
A list of useful payloads and bypass for Web Application Security and Pentest/CTF
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
PyGWalker: Turn your dataframe into an interactive UI for visual analysis

A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.
a delightful machine learning tool that allows you to train, test, and use models without writing code

1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.

RabbitMQ Docker-Compose for MQTT, WS and AMQP


Using YOLOv8 to build a Object Classifier/Tracker for RBG/Thermal Cameras

Paho MQTT Python client script for Home Assistant MQTT Auto-Discovery

Use an image classifier to predict audio file labels.
This page is also available as structured data, with the same sources and dates.
/api/v1/resources/gh/mpolinowski/python-scikitlearn-cheatsheetTake My Repo has not tested this on your machine. Requirements come from the project's own documentation.
