python - 为什么我在 tensorboard t-sne RANDOM 中对 cnn 图像特征的可视化？

问题描述

我有一个卷积神经网络 (VGG16)，它在 26 个图像类的分类任务中表现良好。现在我想在张量板上用 t-SNE 可视化数据分布。我删除了 CNN 的最后一层，因此输出是 4096 个特征。因为分类工作正常（~90% val_accuracy）我希望在 t-SNE 中看到类似模式的东西。但无论我做什么，分布都是随机的（-> 数据以圆形/球体对齐，并且类杂乱无章）。我做错什么了吗？我是否误解了 t-SNE 或 tensorboard？这是我第一次使用它。

这是我获取功能的代码：

import tensorflow as tf
from tensorflow.keras import Model
import os
import numpy as np
from tensorboard.plugins import projector


def get_image_features(data_dir, model, layername='fc2'):  # fc2 = last vgg dense layer

    out = model.get_layer(layername).output
    feature_model = Model(model.input, out)   # the model, see summary() below

    dataGen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1. / model.input.shape[2])
    test_batches = dataGen.flow_from_directory(data_dir, target_size=model.input.shape[1:3], class_mode='categorical',
                                               batch_size=50, shuffle=False)
    features = feature_model.predict(test_batches)

    return features


def create_projector(self, features, metadata_path):
    import tensorflow.compat.v1 as tf   # tf v1 needed
    tf.disable_v2_behavior()

    if type(features) is str:
        features = np.loadtxt(features)

    data_dir_list = os.listdir(self.test_data_dir)
    data_dir_list.sort()
    img_data = []
    for dataset in data_dir_list:
        img_list = os.listdir(os.path.join(self.test_data_dir, dataset))
        img_list.sort()
        for img in img_list:
            input_img = cv2.imread(os.path.join(self.test_data_dir, dataset, img))
            input_img_resize = cv2.resize(input_img, (112, 112))
            img_data.append(input_img_resize)

    img_data = np.array(img_data)
    features_var = tf.Variable(features, name='features')

    with tf.Session() as sess:
        saver = tf.train.Saver([features_var])

        sess.run(features_var.initializer)
        saver.save(sess, os.path.join(self.log_dir, 'images_4_classes.ckpt'))

        config = projector.ProjectorConfig()
        embedding = config.embeddings.add()
        embedding.tensor_name = features_var.name
        embedding.metadata_path = metadata_path

        projector.visualize_embeddings(tf.summary.FileWriter(self.log_dir), config)

编辑：当我取消选中“球化数据”时，它不再是球体，但类的分布仍然看起来完全随机。顺便说一句：PCA 工作正常并显示出某种模式

这是 feature_model.summary()：

Layer (type)                 Output Shape              Param #
=================================================================
block1_conv1_input (InputLay [(None, 224, 224, 3)]     0
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 224, 224, 64)      1792
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 224, 224, 64)      36928
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 112, 112, 64)      0
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 112, 112, 128)     73856
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 112, 112, 128)     147584
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 56, 56, 128)       0
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 56, 56, 256)       295168
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 56, 56, 256)       590080
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 56, 56, 256)       590080
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 28, 28, 256)       0
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 28, 28, 512)       1180160
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 28, 28, 512)       2359808
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 28, 28, 512)       2359808
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 14, 14, 512)       0
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 14, 14, 512)       2359808
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 14, 14, 512)       2359808
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 14, 14, 512)       2359808
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 7, 7, 512)         0
_________________________________________________________________
flatten (Flatten)            (None, 25088)             0
_________________________________________________________________
fc1 (Dense)                  (None, 4096)              102764544
_________________________________________________________________
fc2 (Dense)                  (None, 4096)              16781312
=================================================================

标签： pythontensorflowkerasconv-neural-networktensorboard

python - 为什么我在 tensorboard t-sne RANDOM 中对 cnn 图像特征的可视化？

问题描述

解决方案

推荐阅读