Skip to content

KangOxford/Fourier-Transformer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

48 Commits
 
 
 
 
 
 

Repository files navigation

Fourier-Transformer for solving Fokker-Planck PDE

Transformer for solving the problem in Stochastic Local Volatility

Week4

2022.May.09, 11:00AM~12:15Noon, with Dr. Shuhao Cao, Ziheng Wang, and Deqing Jiang at Mathematical Institute.

Week3

Process

  • Problems remian in this week
    • Only a set of PDEs are suitable for Fourier Transform.
    • Why sample is 256*256 and downsample is 64*64, what has downsampling done here?
    • How MCMC using FNO?
    • To solve different PDEs, do we need to change the code in the FNO?
      • Or the info has been included in the dataset by sampling.
    • We need to solve a two dimensional difussion equation.
      • """
        @author: Zongyi Li
        This file is the Fourier Neural Operator for 3D problem
        such as the Navier-Stokes equation discussed in Section 5.3
        in the [paper](https://arxiv.org/pdf/2010.08895.pdf),
        which takes the 2D spatial + 1D temporal equation directly as a 3D problem
        """
      • In the class SpectralConv3d(nn.Module):
        #Compute Fourier coeffcients up to factor of e^(- something constant)
        x_ft = torch.fft.rfftn(x, dim=[-3,-2,-1])
        # Multiply relevant Fourier modes
        out_ft = torch.zeros(batchsize, self.out_channels, x.size(-3), x.size(-2), x.size(-1)//2 + 1, dtype=torch.cfloat, device=x.device)
        out_ft[:, :, :self.modes1, :self.modes2, :self.modes3] = \
        self.compl_mul3d(x_ft[:, :, :self.modes1, :self.modes2, :self.modes3], self.weights1)
        out_ft[:, :, -self.modes1:, :self.modes2, :self.modes3] = \
        self.compl_mul3d(x_ft[:, :, -self.modes1:, :self.modes2, :self.modes3], self.weights2)
        out_ft[:, :, :self.modes1, -self.modes2:, :self.modes3] = \
        self.compl_mul3d(x_ft[:, :, :self.modes1, -self.modes2:, :self.modes3], self.weights3)
        out_ft[:, :, -self.modes1:, -self.modes2:, :self.modes3] = \
        self.compl_mul3d(x_ft[:, :, -self.modes1:, -self.modes2:, :self.modes3], self.weights4)
        #Return to physical space
        x = torch.fft.irfftn(out_ft, s=(x.size(-3), x.size(-2), x.size(-1)))
    • How does the advantage of operator learning functions in this lab?
      • Does it not need to train again with different intial condition or boundrary condition.
      • Perhaps. As the Operator is learnt. So one of the three condtion has been solved, with the other two to be the intial condition and boundrary condition.
  • Try with FNO for basic set up of the experiment.
  • The FNO model for solving entire family of PDEs.
  • By now there is no need to focus on the Galerkin Transformer, as it only provides a better precision on results.
  • We will focus more on the FNO in the next stages.
  • The FNO is more focused on solving a family of PDEs without try to train the neural network from the scratch, such as when the initial condition or the boundary condition change.

About

Transformer and Neural Operator for solving Stochastic PDE

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published