Multivariate Time Series Transformer #130066
Unanswered
sergiofdez02
asked this question in
Programming Help
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Body
The transformer class I've implemented for multivariate time series forecasting is giving me weird results when training, leading to volatile valid loss and increasing training loss. Could it be due to the structure designed? I am showing you the structure of the transformer, without including the Decoder and Encoder layers:
class Transformer(tf.keras.Model):
def init(self, num_blocks_enc, num_blocks_dec, d_model, d_ff, num_heads, output_size, rate=0.1, **kwargs):
super(Transformer, self).init(**kwargs)
self.time2vec = Time2Vec(d_model)
self.encoder = [EncoderLayer(d_model, d_ff, num_heads, rate) for _ in range(num_blocks_enc)]
self.decoder = [DecoderLayer(d_model, d_ff, num_heads, rate) for _ in range(num_blocks_dec)]
self.final_layer = tf.keras.layers.Dense(output_size, activation='relu')
self.num_blocks_enc = num_blocks_enc
self.num_blocks_dec = num_blocks_dec
self.d_model = d_model
self.d_ff = d_ff
self.num_heads = num_heads
self.output_size = output_size
self.rate = rate
Guidelines
Beta Was this translation helpful? Give feedback.
All reactions