Keras通用技能

发表于 | 分类于

Keras通用技能归纳

  • 迁移学习
  • GPU，Session
  • 训练：metrics
  • 训练：callback
  • 训练：loss
  • 模型融合

迁移学习

常见的是预训练模型+全局池化的操作，如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
from keras.applications import Xception
from keras import models

#以Xception作示例
conv_base = Xception(weights='imagenet',
                  include_top=False,
                  input_shape=(299, 299, 3)
                 )
print(conv_base.summary())
for layer in conv_base.layers[:FREEZE_LAYERS]:
    layer.trainable = False
for layer in conv_base.layers[FREEZE_LAYERS:]:
    layer.trainable = True

model = models.Sequential()
model.add(conv_base)
model.add(layers.GlobalMaxPooling2D())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(1, activation='sigmoid'))

如果是提取训练完成的模型几层来生成特征向量用于后续步骤，拆模型常见方法：

1
2
3
4
5
6
#加载预训练模型，这里build_classifier()是自己的方法
model_classifier = build_classifier()
model_classifier.load_weights('keras_weights_bin-03-0.02.h5')

#‘activation_3'是我们需要输出向量的层名，通过model.summary()方法获得
model_feature = Model(inputs=[model_classifier.input], outputs=[model_classifier.get_layer('activation_3').output])

预训练权重也可以在最后，

以model_feature.local_weights('keras_weights_bin-03-0.02.h5', by_name=True)方式导入权重。

GPU，Session

这里给出比较全的资料：博客1, 博客2

说一下常用的3个操作~

  • 指定GPU卡号

    1
    2
    import os
    os.environ["CUDA_VISIBLE_DEVICES"] = "2"

  • Session对GPU用量动态调整

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    import keras.backend.tensorflow_backend as KTF
    import tensorflow as tf
    import os


    os.environ["CUDA_VISIBLE_DEVICES"] = "1"

    config = tf.ConfigProto()
    config.gpu_options.allow_growth=True   #不全部占满显存, 按需分配
    sess = tf.Session(config=config)

    KTF.set_session(sess)

  • 多GPU训练

    1
    2
    3
    parallel_model = multi_gpu_model(model, gpus=8)
    parallel_model.compile(loss='categorical_crossentropy',
                           optimizer='rmsprop')

训练：metrics

通常我们在model.compile()内metrices会如下写：

metrics=[‘mae’, ‘acc’]

或者

from keras import metrics && metrics=[metrics.mae, metrics.categorical_accuracy]

但比赛等场景评估标准为f1 score，以下为添加f1 score进metrics

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from keras import backend as K
 
def f1(y_true, y_pred):
    def recall(y_true, y_pred):
        """Recall metric.
        Only computes a batch-wise average of recall.
        Computes the recall, a metric for multi-label classification of
        how many relevant items are selected.
        """
        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
        recall = true_positives / (possible_positives + K.epsilon())
        return recall
 
    def precision(y_true, y_pred):
        """Precision metric.
        Only computes a batch-wise average of precision.
        Computes the precision, a metric for multi-label classification of
        how many selected items are relevant.
        """
        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
        precision = true_positives / (predicted_positives + K.epsilon())
        return precision
    precision = precision(y_true, y_pred)
    recall = recall(y_true, y_pred)
    return 2*((precision*recall)/(precision+recall+K.epsilon()))

使用：

1
2
3
model.compile(loss='binary_crossentropy',
             optimizer=optimizers.rmsprop(lr=2e-4),
             metrics=['acc', f1])

还有就是解决数据不均衡问题时采用的Metrics：sensitivity和specificity，下面会提到

训练：callback

callback传入model.fit_generator()内在每轮结束时调用，这里给出个例子，具体解释可见中文文档

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import keras
from keras.callbacks import ModelCheckpoint, EarlyStopping
from keras.callbacks import ReduceLROnPlateau
# Callbacks
tensor_board = keras.callbacks.TensorBoard(log_dir='./kouzhao_half_shan_xception_logs', histogram_freq=0, write_graph=True, write_images=True)

early_stop = EarlyStopping('val_acc', patience=int(200))

reduce_lr = ReduceLROnPlateau('val_acc', factor=0.5, patience=int(200), verbose=1)

trained_models_path = 'kouzhao_half_shan_xception_models/model'
model_names = trained_models_path + '.{epoch:02d}-{val_acc:.4f}.hdf5'
model_checkpoint = ModelCheckpoint(model_names, monitor='val_acc', verbose=1, save_best_only=False)

callbacks = [tensor_board, model_checkpoint, early_stop, reduce_lr]

注意：谨慎使用ReduceLROnPlateau，使用不当结果会倒退许多

附model_checkpoint的自构方法：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class MyCbk(keras.callbacks.Callback):
    def __init__(self, model):
        keras.callbacks.Callback.__init__(self)
        self.model_to_save = model

    def on_epoch_end(self, epoch, logs=None):
        fmt = 'models/model.%02d-%.4f.hdf5'
        highest_acc = get_highest_acc()
        if float(logs['val_acc']) > highest_acc:
            self.model_to_save.save(fmt % (epoch, logs['val_acc']))

...
...
#使用
model_checkpoint = MyCbk(model)

训练：Loss

通常的就不说了，说一下关于解决数据样本不均衡问题的

参考1，参考2

首先说明，sample_weights,class_weights用于model.fit()；

  • class_weight 针对类

    主要针对的上数据不均衡问题，比如：异常检测的二项分类问题，异常数据仅占1%，正常数据占99%; 此时就要设置不同类对loss的影响。

  • sample_weight 针对样本

    主要解决的是样本质量不同的问题，比如前1000个样本的可信度，那么它的权重就要高，后1000个样本可能有错、不可信，那么权重就要调低。

    loss_weights用于model.compile()时候，是多loss时各loss权重比

使用1

1
2
3
4
5
6
7
8
9
10
11
12
# class weights
cw = {0: 1, 1: 50}
model.fit(x_train, y_train,batch_size=batch_size,epochs=epochs,verbose=1,callbacks=cbks,validation_data=(x_test, y_test), shuffle=True,class_weight=cw)

# sample weights
from sklearn.utils import class_weight
 
list_classes = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
y = train[list_classes].values
sample_weights = class_weight.compute_sample_weight('balanced', y)
 
model.fit(X_t, y, batch_size=batch_size, epochs=epochs,validation_split=0.1,sample_weight=sample_weights, callbacks=callbacks_list)

使用2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
class_weight={
    1: n_non_cancer_samples / n_cancer_samples * t
}

def sensitivity(y_true, y_pred):
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
    return true_positives / (possible_positives + K.epsilon())

def specificity(y_true, y_pred):
    true_negatives = K.sum(K.round(K.clip((1-y_true) * (1-y_pred), 0, 1)))
    possible_negatives = K.sum(K.round(K.clip(1-y_true, 0, 1)))
    return true_negatives / (possible_negatives + K.epsilon())
model.compile(
    loss='binary_crossentropy',
    optimizer=RMSprop(0.001),
    metrics=[sensitivity, specificity]
)

有关解决数据不均衡的还有些小姿势,这里

模型融合

请见之前俺滴博客~

TODO