时间戳

首页 > 解决方案 > Pytorch RNN HTML 生成

python - Pytorch RNN HTML 生成

问题描述

我被困了几天，试图制作和RNN 网络来学习基本的HTML 模板。我尝试了不同的方法，甚至对以下数据进行了过度拟合：

<!DOCTYPE html>
<html>
<head>
<title>Page Title</title>
</head>
<body>

<h1>This is a Heading</h1>
<p>This is a paragraph.</p>

</body>
</html>

使用Adam Optimizer 和CrossEntropyLoss获得100% 的训练和验证准确率。

问题是当我尝试从网络中采样时，结果是完全随机的，我不知道问题出在哪里：

..<a<a<a<a<aa<ttp11111b11111b11111111b11b1bbbb<btttn111

我的采样功能如下：

def sample_sentence():
    words = list()
    count = 0
    modelOne.eval()
    with torch.no_grad():
        # Setup initial input state, and input word (we use "the").
        previousWord = torch.LongTensor(1, 1).fill_(trainData.vocabulary['letter2id']['[START]'])
        hidden =  Variable(torch.zeros(6, 1, 100).to(device))


        while True:
            # Predict the next word based on the previous hidden state and previous word.
            inputWord = torch.autograd.Variable(previousWord.to(device))


            predictions, newHidden = modelOne(inputWord, hidden)

            hidden = newHidden


            pred = torch.nn.functional.softmax(predictions.squeeze()).data.cpu().numpy().astype('float64')

            pred = pred/np.sum(pred)


            nextWordId = np.random.multinomial(1, pred, 1).argmax()


            if nextWordId == 0:
                continue

            words.append(trainData.vocabulary['id2letter'][nextWordId])
            # Setup the inputs for the next round.
            previousWord.fill_(nextWordId)


            # Keep adding words until the [END] token is generated.
            if nextWordId == trainData.vocabulary['letter2id']['[END]']:
                break

            if count>20000:
                break
            count += 1
        words.insert(0, '[START]')


        return words

我的网络架构在这里：

class ModelOne(Model) :
    def __init__(self,
                vocabulary_size,
                hidden_size,
                num_layers,
                rnn_dropout,
                embedding_size,
                dropout,
                num_directions):
        super(Model, self).__init__()

        self.vocabulary_size = vocabulary_size
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.rnn_dropout = rnn_dropout
        self.dropout = dropout
        self.num_directions = num_directions
        self.embedding_size = embedding_size


        self.embeddings = nn.Embedding(self.vocabulary_size, self.embedding_size)
        self.rnn = nn.GRU(self.embedding_size,
                          self.hidden_size,
                          num_layers=self.num_layers,
                          bidirectional=True if self.num_directions==2 else False,
                          dropout=self.rnn_dropout,
                          batch_first=True)
        self.linear = nn.Linear(self.hidden_size*self.num_directions, self.vocabulary_size)





    def forward(self, paddedSeqs, hidden):

        batchSequenceLength = paddedSeqs.size(1)

        batchSize = paddedSeqs.size(0)

        lengths = paddedSeqs.ne(0).sum(dim=1)

        embeddingVectors = self.embeddings(paddedSeqs)

        x = torch.nn.utils.rnn.pack_padded_sequence(embeddingVectors, lengths, batch_first=True)

        self.rnn.flatten_parameters()

        x,hid = self.rnn(x, hidden)

        output, _ = torch.nn.utils.rnn.pad_packed_sequence(x, batch_first=True, padding_value=0, total_length=batchSequenceLength)

        predictions = self.linear(output)


        return predictions.view(batchSize, self.vocabulary_size, batchSequenceLength), hid

    def init_hidden(self, paddedSeqs):
        hidden = Variable(torch.zeros(self.num_layers*self.num_directions,
                                    1,
                                    self.hidden_size).to(device))
        return hidden



modelOne =ModelOne(vocabulary_size=vocabularySize,
                 hidden_size=100,
                 embedding_size=50,
                 num_layers=3,
                 rnn_dropout=0.0,
                 dropout=0,
                 num_directions=2).to(device)

如果您对需要更改的内容有任何想法，请告诉我。我在这里将所有代码添加到 github 存储库：https ://github.com/OverclockRo/HTMLGeneration/blob/SamplingTestTemplate/Untitled.ipynb

标签： pythonhtmlnlppytorchrecurrent-neural-network

解决方案

首先，要使 GRU (RNN) 高效，您可能需要更多数据来训练。

其次，您的嵌入似乎有问题。看起来，映射词汇['id2letter'] 不起作用，否则您将获得标签序列，例如<head><title><title><title>, 而不是p111.

编辑

我已经在这个页面的 html 源代码上训练了这个字符级 GRU 网络1700 个 epoch。这里是它生成的 2000 个字符的示例摘录：

A+Implexementation--nope bande that shoos</td></tr><tr><td class="line-number" value="296"></td><td class="line-content">              </td></tr><tr><td class="line-number" value="1437"></td><td class="line-content">                    <span class="html-tag">&lt;/a&gt;</span></td></tr><tr><td class="line-number" value="755"></td><td class="line-content">                </td></tr><tr><td class="line-number" value="584"></td><td class="line-content">    <span class="html-tag">&lt;a <span class="html-attribute-name">data-controller</span>="<span class="html-attribute-value">footer__menu__link</span>"&gt;</span><span class="html-tag">&lt;div <span class="html-attribute-name">data-target</span>="<span class="html-attribute-value">release__line</span>"&gt;</span><span class="html-tag">&lt;a <span class="html-attribute-name">class</span>="<span class="html-attribute-value">/hase__version-date</span>"&gt;</span></td></tr><tr><td class="line-number" value="174"></td><td class="line-content"><br></td></tr><tr><td class="line-number" value="1315"></td><td class="line-content">Bule and the use the twith a hoas suiecode excess ardates</td></tr><tr><td class="line-number" value="1003"></td><td class="line-content"><span class="html-tag">&lt;/a&gt;</span></td></tr><tr><td class="line-number" value="129"></td><td class="line-content">              </td></tr><tr><td class="line-number" value="269"></td><td class="line-content">              <span class="html-tag">&lt;/ul&gt;</span></td></tr><tr><td class="line-number" value="591"></td><td class="line-content">                <span class="html-tag">&lt;/div&gt;</span></td></tr><tr><td class="line-number" value="553"></td><td class="line-content">    <span class="html-tag">&lt;div <span class="html-attribute-name">href</span>="<a class="html-attribute-value html-external-link" target__link</td><td class="line-content">              </td></tr><tr><td class="line-number" value="103"></td><td class="line-content">    </td></tr><tr><td cla

我希望这有帮助。

推荐阅读