machine-learning - Can someone explain why this scalar layer non-trainable?

Question

  query_word_ids = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="query_word_ids")
  query_mask = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="query_mask")
  query_segment_ids = tf.keras.Input(shape=(max_len,), dtype=tf.int32, name="query_segment_ids")

  pooled_output, sequence_output = bert_layer([query_word_ids, query_mask, query_segment_ids])
  query_dense_embeddings = sequence_output[:, 0, :] # [None, 768]


  candidates_dense_embeddings = tf.keras.Input(shape=(topk,768,), dtype=tf.float32, name="candidates_dense_embeddings")


  candidates_dense_score = keras.layers.Dot(axes=(2,1),name="dense_score")([candidates_dense_embeddings,query_dense_embeddings])

  batch_size = candidates_dense_score.shape[0] 
  candidate_sparse_score = tf.keras.Input(shape=(batch_size,), dtype=tf.float32, name="candidate_sparse_score")

  scaling_sparse_score = Scalar()(candidate_sparse_score)

  score = scaling_sparse_score + candidates_dense_score

  model =  tf.keras.models.Model(inputs = [[query_word_ids, query_mask, query_segment_ids],candidates_dense_embeddings, candidate_sparse_score], outputs=score )
  model.compile(optimizer=tf.keras.optimizers.Adam(lr=1e-5),loss=marginal_loss)

  return model  

  class Scalar(Layer):
    def __init__(self):
      super(Scalar, self).__init__()
    def build(self,input_shape) :
      self.W = K.variable(2)
      self._trainable_weights=[self.W]
    def call(self,inputs):
      return self.W*inputs

I don't know why if I put Scalar model outside. It's still trainable

The original model can be found here: https://github.com/dmis-lab/BioSyn/blob/master/src/biosyn/rerankNet.py I try to implement using Keras. I know that Scalar has its own learning rate, but using same loss function with the score model how can I do that. Please show me the easiest way, I'm newbie

标签： machine-learningkeraskeras-layer