variables - 在tensorflow中，如何初始化部分变量

Question

我想初始化一个变量列表，并将它们定义为一个名为“block_var”的列表。我想使用截断的正常方法来初始化它们。

block_var = [v for v in tf.global_variables() if 'block' in v.name]
init_block = tf.variables_initializer(var_list = block_var)

那我该怎么办？我努力了

for v in block_var:
    v.initializer = tf.truncated_normal_initializer()

我也试过

init_block = tf.truncated_normal_initializer()

两个领域。

标签： variablestensorflowinitialization

Answer 1

解决方案 1

您需要为get_variablelike传递初始化程序参数

import tensorflow as tf
import numpy as np

ref0 = tf.get_variable('block0', [2], initializer=tf.truncated_normal_initializer(mean=40))
ref1 = tf.get_variable('block1', [2], initializer=tf.truncated_normal_initializer(mean=40))
ref2 = tf.get_variable('block2', [2], initializer=tf.truncated_normal_initializer(mean=40))

ref4 = tf.get_variable('foo0', [2], initializer=tf.truncated_normal_initializer(mean=10))
ref5 = tf.get_variable('foo1', [2], initializer=tf.truncated_normal_initializer(mean=10))

block_vars = [v for v in tf.global_variables() if 'block' in v.name]
block_vars_complement = [v for v in tf.global_variables() if 'block' not in v.name]

with tf.Session() as sess:
    sess.run(tf.variables_initializer(var_list=block_vars))
    print(np.mean(sess.run([ref0, ref1, ref2])), 'should be ~ 40')

    try:
        print(np.mean(sess.run([ref4])))
    except Exception as e:
        print('[INFO] failed as expected with message %s' % e)

    sess.run(tf.variables_initializer(var_list=block_vars_complement))
    print(np.mean(sess.run([ref4, ref5])), 'should be ~ 10')

解决方案 2

如果您不想将 传递initializer给每个，则get_variable可以使用自定义 getter，例如

import tensorflow as tf
import numpy as np


def my_getter(getter, name, shape, *args, **kwargs):
    if 'block' not in name:
        return getter(name=name, shape=shape, *args, **kwargs)
    else:
        kwargs['initializer'] = tf.truncated_normal_initializer(mean=40)
        return getter(name=name, shape=shape, *args, **kwargs)

with tf.variable_scope("some_scopename", custom_getter=my_getter):
    ref0 = tf.get_variable('block0', [2], initializer=tf.truncated_normal_initializer(mean=10))
    ref1 = tf.get_variable('block1', [2], initializer=tf.truncated_normal_initializer(mean=10))
    ref2 = tf.get_variable('block2', [2], initializer=tf.truncated_normal_initializer(mean=10))

    ref4 = tf.get_variable('foo0', [2], initializer=tf.truncated_normal_initializer(mean=10))
    ref5 = tf.get_variable('foo1', [2], initializer=tf.truncated_normal_initializer(mean=10))

block_vars = [v for v in tf.global_variables() if 'block' in v.name]
block_vars_complement = [v for v in tf.global_variables() if 'block' not in v.name]

with tf.Session() as sess:
    sess.run(tf.variables_initializer(var_list=block_vars))
    print(np.mean(sess.run([ref0, ref1, ref2])), 'should be ~ 40')

    try:
        print(np.mean(sess.run([ref4])))
    except Exception as e:
        print('[INFO] failed as expected with message %s' % e)

    sess.run(tf.variables_initializer(var_list=block_vars_complement))
    print(np.mean(sess.run([ref4, ref5])), 'should be ~ 10')

解决方案 3

tf.truncated_normal_initializer或其他初始化程序本身只是操作。因此，这些可以循环应用于集合中的所有变量，并且最终可以应用这样的分组更新（请参阅 参考资料initialize_collection）：

import tensorflow as tf
import numpy as np

ref0 = tf.get_variable('block0', [2], initializer=tf.truncated_normal_initializer(mean=40))
ref1 = tf.get_variable('block1', [2], initializer=tf.truncated_normal_initializer(mean=40))
ref2 = tf.get_variable('block2', [2], initializer=tf.truncated_normal_initializer(mean=40))

ref4 = tf.get_variable('foo0', [2], initializer=tf.truncated_normal_initializer(mean=10))
ref5 = tf.get_variable('foo1', [2], initializer=tf.truncated_normal_initializer(mean=10))

block_vars = [v for v in tf.global_variables() if 'block' in v.name]
block_vars_complement = [v for v in tf.global_variables() if 'block' not in v.name]


def initialize_collection(collection, initializer):
    ops = []
    for v in collection:
        ops.append(v.assign(initializer(shape=v.shape)))
    return tf.group(ops)


with tf.Session() as sess:
    sess.run(tf.variables_initializer(var_list=block_vars))
    print(np.mean(sess.run([ref0, ref1, ref2])), 'should be ~ 40')

    sess.run(initialize_collection(block_vars, tf.truncated_normal_initializer(mean=-40, stddev=0.01)))
    print(np.mean(sess.run([ref0, ref1, ref2])), 'should be ~ -40')