Clasificación, Softmax, Iris
Contents
Clasificación, Softmax, Iris#
Clasificación con múltiples categorías
Introducción#
Esta lección está dedicada a un modelo de clasificación con múltiples categorías, que corresponde a la generalización natural del modelo logístico.
Practicaremos la codificación one-hot para los datos de salida.
También usaremos la API funcional de tf.keras, que es una forma de programación maś flexible y poderosa que el modelo Sequential
Usaremos las funciones relu para capas intermedias y entrada y la función de activación softmax para la salida, debido a que se tienen varias clases.
Importa módulos#
Usaremos las bibliotecas
seaborn para gráficas un poco más elegantes
sklearn para utilidades de estandarización de datos y matriz de confusión
Puede usar las siguientes instrucciones para instalar desde la consola.
# !conda install -c anaconda seaborn
# !conda install -c intel scikit-learn
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import tensorflow as tf
#
from tensorflow.keras.models import Model
#
from tensorflow.keras.layers import Dense, Input, Activation, Dropout
#
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.utils import plot_model
#
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import confusion_matrix
#
#from sklearn import KFold
print("Versión de Tensorflow:", tf.__version__)
Versión de Tensorflow: 2.9.1
Funciones de activación#
Relu#
Dada la salida del sumador digamos \(y=\mathbf{w}'\mathbf{x} +b\), la función de activación relu esta definida por
\[\begin{split}
\text{relu}(y) = \begin{cases} &0, \text{ si } y\le 0,\\
&y, \text{ en otro caso } \end{cases}
\end{split}\]
Softmax#
Dados los valores \(x_1,\ldots, x_n\) la función *softmax * es definida por
\[
\text{softmax}(x_i) = \frac{e^{x_i}}{\sum_{j=1}^{n} e^{x_j}}
\]
Es decir, softmax transforma los valores en un función de probabilidad.
El conjunto de datos Iris#
Este conjunto de datos fue introducido por sir Ronald Fisher
Lectura de datos y primera vista de los datos#
Bajamos los datos de Internet usando tf.keras.utils y luego los cargamos en dataframes de Python.
# nombres de las columnas de los datos
col_names = ['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth', 'Species']
target_dimensions = ['Setosa', 'Versicolor', 'Virginica']
# lee los datos
training_data_path = tf.keras.utils.get_file("iris_training.csv", "https://storage.googleapis.com/download.tensorflow.org/data/iris_training.csv")
test_data_path = tf.keras.utils.get_file("iris_test.csv", "https://storage.googleapis.com/download.tensorflow.org/data/iris_test.csv")
training = pd.read_csv(training_data_path, names=col_names, header=0)
test = pd.read_csv(test_data_path, names=col_names, header=0)
test.head()
| SepalLength | SepalWidth | PetalLength | PetalWidth | Species | |
|---|---|---|---|---|---|
| 0 | 5.9 | 3.0 | 4.2 | 1.5 | 1 |
| 1 | 6.9 | 3.1 | 5.4 | 2.1 | 2 |
| 2 | 5.1 | 3.3 | 1.7 | 0.5 | 0 |
| 3 | 6.0 | 3.4 | 4.5 | 1.6 | 1 |
| 4 | 5.5 | 2.5 | 4.0 | 1.3 | 1 |
Pre-procesamiento#
La variable objetivo (target) tiene tres categorías. Usaremos la codificación one-hot para transformar las codificaciones en vectores binarios.
Codificación one-hot#
y_train= pd.DataFrame(to_categorical(training.Species))
y_train.columns = target_dimensions
y_test = pd.DataFrame(to_categorical(test.Species))
y_test.columns = target_dimensions
y_test
| Setosa | Versicolor | Virginica | |
|---|---|---|---|
| 0 | 0.0 | 1.0 | 0.0 |
| 1 | 0.0 | 0.0 | 1.0 |
| 2 | 1.0 | 0.0 | 0.0 |
| 3 | 0.0 | 1.0 | 0.0 |
| 4 | 0.0 | 1.0 | 0.0 |
| 5 | 0.0 | 1.0 | 0.0 |
| 6 | 1.0 | 0.0 | 0.0 |
| 7 | 0.0 | 0.0 | 1.0 |
| 8 | 0.0 | 1.0 | 0.0 |
| 9 | 0.0 | 0.0 | 1.0 |
| 10 | 0.0 | 0.0 | 1.0 |
| 11 | 1.0 | 0.0 | 0.0 |
| 12 | 0.0 | 0.0 | 1.0 |
| 13 | 0.0 | 1.0 | 0.0 |
| 14 | 0.0 | 1.0 | 0.0 |
| 15 | 1.0 | 0.0 | 0.0 |
| 16 | 0.0 | 1.0 | 0.0 |
| 17 | 1.0 | 0.0 | 0.0 |
| 18 | 1.0 | 0.0 | 0.0 |
| 19 | 0.0 | 0.0 | 1.0 |
| 20 | 1.0 | 0.0 | 0.0 |
| 21 | 0.0 | 1.0 | 0.0 |
| 22 | 0.0 | 0.0 | 1.0 |
| 23 | 0.0 | 1.0 | 0.0 |
| 24 | 0.0 | 1.0 | 0.0 |
| 25 | 0.0 | 1.0 | 0.0 |
| 26 | 1.0 | 0.0 | 0.0 |
| 27 | 0.0 | 1.0 | 0.0 |
| 28 | 0.0 | 0.0 | 1.0 |
| 29 | 0.0 | 1.0 | 0.0 |
Elimina la columna Species del dataframe#
y_train_species = training.pop('Species')
#test.drop(['Species'], axis=1, inplace=True)
y_test_species = test.pop('Species') # extrae la columna y la coloca en y_test_species
#
#Si necesita subir al dataframe la recodificación use estas líneas
#training = training.join(y_train )
#test = test.join(y_test)
Normaliza los features#
StandardScaler#
# crea el objeto StandardScaler
scaler = StandardScaler()
# Ajusta los parámetros del scaler
scaler.fit(training)
print (scaler.mean_)
# escala training y test
x_train = scaler.transform(training)
x_test = scaler.transform(test)
# labels ( no requieren escalación)
[5.845 3.065 3.73916667 1.19666667]
Crea el modelo usando la API funcional#
La API funcional de Keras es bastante más flexible y poderosa que el modelo Sequential
# Con la API funcion se requiere la capa Input que transforma la entrada
# en un tensor de tensorflow directamente
#
inputs = Input(shape=(4,),name='capa_entrada')
#
# vamos construyendo capa por capa
x = Activation('relu')(inputs)
x = Dense(8, activation='relu',name='primera_capa_oculta')(x)
x = Dropout(0.2)(x)
x = Dense(16, activation='relu', name='segunda_capa_oculta')(x)
x = Dropout(0.2)(x)
outputs = Dense(3, activation='softmax', name='capa_salida')(x)
# Creamos ahora el modelo
model_iris = Model(inputs=inputs, outputs=outputs)
model_iris.summary()
plot_model(model_iris, to_file='../Imagenes/iris_model.png',
show_shapes=True)
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
capa_entrada (InputLayer) [(None, 4)] 0
activation (Activation) (None, 4) 0
primera_capa_oculta (Dense) (None, 8) 40
dropout (Dropout) (None, 8) 0
segunda_capa_oculta (Dense) (None, 16) 144
dropout_1 (Dropout) (None, 16) 0
capa_salida (Dense) (None, 3) 51
=================================================================
Total params: 235
Trainable params: 235
Non-trainable params: 0
_________________________________________________________________
You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) for plot_model/model_to_dot to work.
Compila#
model_iris.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy']
)
Entrena#
class PrintDot(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs):
print('.', end='')
epochs = 200
history = model_iris.fit(x_train, y_train,
batch_size= 16,
epochs= epochs,
validation_split=0.1, verbose=0,
callbacks=[PrintDot()])
print('\nHecho')
print('Resultados finales de pérdida y exactitud\n')
# presenta la última parte de la historia
hist = pd.DataFrame(history.history)
hist['epoch'] = history.epoch
hist.tail()
....
....
...
...
....
....
...
...
....
....
...
...
...
....
...
...
....
....
...
....
....
...
...
....
....
....
...
...
....
...
...
...
...
....
...
...
....
...
...
...
...
...
....
....
....
....
....
...
...
...
....
...
...
...
....
...
...
...
..
Hecho
Resultados finales de pérdida y exactitud
| loss | accuracy | val_loss | val_accuracy | epoch | |
|---|---|---|---|---|---|
| 195 | 0.356077 | 0.851852 | 0.307717 | 0.916667 | 195 |
| 196 | 0.318904 | 0.879630 | 0.307724 | 0.916667 | 196 |
| 197 | 0.277614 | 0.888889 | 0.305931 | 0.916667 | 197 |
| 198 | 0.261583 | 0.888889 | 0.303785 | 0.916667 | 198 |
| 199 | 0.285054 | 0.870370 | 0.302396 | 0.916667 | 199 |
Evaluación del modelo#
def plot_history(history):
hist = pd.DataFrame(history.history)
hist['epoch'] = history.epoch
plt.figure()
plt.xlabel('Epoch')
plt.ylabel('Pérdida')
plt.plot(hist['epoch'], hist['loss'],
label='Pérdida entrenamiento')
plt.plot(hist['epoch'], hist['val_loss'],
label = 'Pérdida validación')
plt.ylim([0,2])
plt.legend()
plt.figure()
plt.xlabel('Epoch')
plt.ylabel('Exactitud')
plt.plot(hist['epoch'], hist['accuracy'],
label='Exactitud entrenamiento')
plt.plot(hist['epoch'], hist['val_accuracy'],
label = 'Exactitud Validación')
plt.ylim([0,1])
plt.legend()
plt.show()
plot_history(history)
Predicciones#
# Predicting the Test set results
y_pred = model_iris.predict(x_test)
y_pred_c = np.argmax(y_pred, axis=1)
1/1 [==============================] - ETA: 0s
1/1 [==============================] - 0s 240ms/step
Matriz de confusión#
cm = confusion_matrix(y_test_species, y_pred_c)
print("Our accuracy is {}%".format(((cm[0][0] + cm[1][1]+ cm[2][2])/y_test_species.shape[0])*100))
Our accuracy is 93.33333333333333%
sns.heatmap(cm,annot=True)
plt.savefig('h.png')
Exploración interna de la red#
Cálculo de la salida de los datos de entrenamiento#
inputs = x_train
outputs = model_iris(inputs)
outputs.numpy().round(2)[:10]
array([[0. , 0. , 1. ],
[0.59, 0.39, 0.02],
[0. , 0.32, 0.68],
[0.81, 0.18, 0.01],
[0.91, 0.08, 0. ],
[0.91, 0.08, 0. ],
[0.91, 0.08, 0. ],
[0. , 0. , 1. ],
[0. , 0.88, 0.12],
[0.91, 0.08, 0. ]], dtype=float32)
Extrae la segunda capa oculta para estos datos#
# modelo Sequential
#layer_2 = tf.keras.models.Model(
# inputs=model_iris.inputs,
# outputs=model_iris.get_layer(name='segunda_capa_oculta').output,
#)
# API funcional
# 1. crea un nuevo modelo
# 2. Compila
# 3. Predice
inputs = x_train
model = Model(model_iris.input, model_iris.get_layer(name='segunda_capa_oculta').output)
model.compile()
output = model.predict(inputs)
1/4 [======>.......................] - ETA: 0s
4/4 [==============================] - 0s 5ms/step
output.shape
(120, 16)
Crea tabla de datos para hacer un gráfico tsne#
plot_data = np.hstack([output, np.array(y_train_species).reshape(y_train_species.shape[0],1)])
plot_data = pd.DataFrame(plot_data)
plot_data
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.595804 | 0.000000 | 0.000000 | 0.339116 | 0.076414 | 5.210826 | 1.301694 | 0.000000 | 1.812386 | 0.000000 | 4.008091 | 0.464635 | 7.058684 | 3.973451 | 0.000000 | 5.071805 | 2.0 |
| 1 | 0.000000 | 0.242695 | 0.166379 | 0.000000 | 0.667097 | 0.212840 | 0.000000 | 0.683829 | 0.000000 | 0.667696 | 0.204219 | 0.000000 | 0.390383 | 0.128503 | 0.118723 | 0.317416 | 1.0 |
| 2 | 0.173337 | 0.069638 | 0.000000 | 0.001982 | 0.410923 | 2.344948 | 0.513256 | 0.067906 | 0.624839 | 0.253278 | 1.831339 | 0.198072 | 3.317308 | 1.766144 | 0.000000 | 2.347900 | 2.0 |
| 3 | 0.000000 | 0.247351 | 0.255774 | 0.000000 | 0.667080 | 0.101341 | 0.000000 | 0.707954 | 0.000000 | 0.680460 | 0.105888 | 0.000000 | 0.193972 | 0.024393 | 0.166143 | 0.145486 | 0.0 |
| 4 | 0.000000 | 0.245993 | 0.347222 | 0.000000 | 0.639631 | 0.000000 | 0.000000 | 0.730347 | 0.000000 | 0.686948 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.179519 | 0.000000 | 0.0 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 115 | 0.000000 | 0.228051 | 0.000000 | 0.000000 | 0.712288 | 0.570997 | 0.000000 | 0.569060 | 0.000000 | 0.632626 | 0.654038 | 0.000000 | 1.127714 | 0.398548 | 0.125362 | 0.783992 | 1.0 |
| 116 | 0.000000 | 0.232330 | 0.031854 | 0.000000 | 0.698358 | 0.464140 | 0.000000 | 0.603313 | 0.000000 | 0.643005 | 0.519526 | 0.000000 | 0.907442 | 0.318200 | 0.122872 | 0.644950 | 1.0 |
| 117 | 0.000000 | 0.242695 | 0.166379 | 0.000000 | 0.667097 | 0.212840 | 0.000000 | 0.683829 | 0.000000 | 0.667696 | 0.204219 | 0.000000 | 0.390383 | 0.128503 | 0.118723 | 0.317416 | 0.0 |
| 118 | 0.000000 | 0.242695 | 0.166379 | 0.000000 | 0.667097 | 0.212840 | 0.000000 | 0.683829 | 0.000000 | 0.667696 | 0.204219 | 0.000000 | 0.390383 | 0.128503 | 0.118723 | 0.317416 | 0.0 |
| 119 | 0.000000 | 0.242695 | 0.166379 | 0.000000 | 0.667097 | 0.212840 | 0.000000 | 0.683829 | 0.000000 | 0.667696 | 0.204219 | 0.000000 | 0.390383 | 0.128503 | 0.118723 | 0.317416 | 1.0 |
120 rows × 17 columns
Crea gráfico tsne#
from sklearn.manifold import TSNE
# reduce dimensionalidad con t-sne
tsne = TSNE(n_components=2, verbose=1, perplexity=50, n_iter=1000, learning_rate=200)
tsne_results = tsne.fit_transform(output)
C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\sklearn\manifold\_t_sne.py:795: FutureWarning: The default initialization in TSNE will change from 'random' to 'pca' in 1.2.
warnings.warn(
[t-SNE] Computing 119 nearest neighbors...
[t-SNE] Indexed 120 samples in 0.000s...
[t-SNE] Computed neighbors for 120 samples in 0.016s...
[t-SNE] Computed conditional probabilities for sample 120 / 120
[t-SNE] Mean sigma: 1.164720
[t-SNE] KL divergence after 250 iterations with early exaggeration: 48.293922
[t-SNE] KL divergence after 1000 iterations: 0.030992
labels = [target_dimensions[i] for i in y_train_species]
#['Setosa', 'Versicolor', 'Virginica']
# visualiza con seaborn
df_tsne = pd.DataFrame(tsne_results, columns=['x', 'y'])
df_tsne['label'] = labels
sns.lmplot(x='x', y='y', data=df_tsne, hue='label', fit_reg=False)
<seaborn.axisgrid.FacetGrid at 0x168a44dce20>
Ejercicio#
Reescriba y reentren la red en Pytorch.
Investigue como extraer la capa oculta en Pytorch
Haga un gráfico TSNE para los datos originales
Haga un reducción ACP y haga el correspondiente gráfico
¿Cuáles son sus conclusiones?