SMWLevelGenerator is a level generation framework for Super Mario
World based on deep neural networks.
By combining techniques from natural language processing, generative
methods and image processing, we obtain a level generation pipeline.
We provide setup scripts, database generation, multiple models and their corresponding training loops. When you actually want to generate levels, we have functions handling everything in the background.
For fundamental questions, check out the thesis. The presentation may prove useful as a less detailed introduction.
Three different tasks:
- Sequence prediction: Predict the rest of a level from one or more inputs.
- First screen generation: Generate the first screen of a level from random noise.
- Metadata prediction: Predict a level's metadata from its first screen.
For each task, we implement multiple models and a training loop. The
training loop works on any given model if it implements our
LearningModel interface.
Models work on data in different dimensionalities:
- 1D: a line of the level
- 2D: one layer of the level
- 3D: multiple layers of the level
3-dimensional data can have different sizes in the third dimension (meaning any combination of layers).
All models handle all dimensionalities correctly if setup correctly. For some dimensions, we provide the correct setups. Any others may easily be specified manually; we focus on easy extension.
- Bash (not required if you download pre-computed databases; see below)
- Any decompression program (we recommend 7-Zip)
- Julia (version 1.2 or 1.3)
- TensorBoard (will be made optional)
- Wine (if not on Windows; required to run Lunar Magic)
- Private build of Lunar Magic
- Floating IPS (also included in the above link with Lunar Magic)
- American Super Mario World ROM (CRC32 checksum
a31bead4; others may work too but this one is tested) - (optional) LMSW (internal emulator for Lunar Magic; newer versions exist but are not tested)
Execute the following in your command line of choice in this directory:
julia --project -e 'using Pkg; Pkg.instantiate()'Or in the Julia REPL, type ] to enter pkg mode. Then, execute the
second line:
julia> ]
pkg> activate .; instantiateJulia is now correctly setup for this project with all dependencies
and their correct versions. Whenever you execute Julia for this
project (in this repository), start it as julia --project
(optionally, to set optimization to high, append -O3 to the
previous command).
With the dependencies setup, you can just skip all the boring manual setup and get right to downloading pre-computed databases.
First, get the dependencies above.
Download and unzip
this,
then see Training or Generation.
Currently, we only provide shell scripts (Bash compatibility) for
the basic setup. If you do not have access to a shell script
interpreter, either wait for the Julia implementation or use the
pre-computed databases.
The most convenient setup command is the following:
./scripts/setup.shTo download from the original source (not recommended to save SMW Central's servers from stress; also may not work due to DDoS protection and/or download limits):
./scripts/setup_smwcentral.shAlternatively, check out the scripts folder to find out how to set everything up manually. We have documentation!
Some patches, roms and levels had to be removed for various reasons (ROMs mostly due to encryption).
It is also very much recommended to remove duplicate and test levels and remove custom sprites:
julia --project scripts/check_checksums.jl
julia --project scripts/remove_custom_sprites.jlWhenever you remove or add hacks, always execute
SMWLevelGenerator.Sprites.generateall() and
SMWLevelGenerator.SecondaryLevelStats.generateall()!
This will pre-calculate statistics to speed up loading and – more
importantly – update the statistics according to your dataset. If
you are setting up manually, you must not skip this step to get the
correct statistics. We do this to minizime the amount of layers we
supply to the model.
You may either generate the databases yourself or download them.
After the previous setup, execute the following:
julia --project -e 'using SMWLevelGenerator; generate_default_databases()'
... or execute it in the Julia REPL if Julia is properly activated in the project.
Or checkout the documentation to generatedb
and related functions to generate databases individually.
If there were issues with one of the previous steps, you can still
generate the databases yourself by downloading the raw dumped level
files as a 8 MB 7z from
here
or as a 23 MB tar.gz
here.
You still need to decompress these in the main folder, then the above
line will work.
Alternatively, download a 34 MB 7z file from
here
or a 61 MB tar.gz from
here.
You will need to decompress these.
For training, you can give parameters to the model as well as to the training loop. Your first decision should be which dimensionality of data to train on. Other parameters can be found in one of the three individual training loops (sequence prediction, generative methods and image processing).
Now, let's look at some example configurations for 2-dimensional data
(these do not include all models). To find all supplied convenience
constructors for the models, take a look at the exported functions
in SMWLevelGenerator.jl.
For an LSTM:
julia --project -O3 -e '
using SMWLevelGenerator;
const model = lstm2d();
const dbpath = "levels_2d_flags_t.jdb";
const res = trainingloop!(model, dbpath, TPs(
epochs=1000,
logdir=joinpath("exps", "meta_2d_low_learning_rate"),
dataiter_threads=3,
use_soft_criterion=false,
));
'For a GPT-2-based transformer using the soft criterion:
julia --project -O3 -e '
using SMWLevelGenerator;
const model = transformer2d();
const dbpath = "levels_2d_flags_t.jdb";
const res = trainingloop!(model, dbpath, TPs(
epochs=1000,
logdir=joinpath("exps", "meta_2d_low_learning_rate"),
dataiter_threads=3,
use_soft_criterion=false,
));
'For a standard DCGAN:
julia --project -O3 -e '
using SMWLevelGenerator;
const d_model = discriminator2d(64);
const g_model = generator2d(64, 96);
const dbpath = "levels_2d_flags_t.jdb";
const res = gan_trainingloop!(d_model, g_model, dbpath, GTPs(
epochs=1000,
lr=5e-5,
logdir=joinpath("exps", "gan_2d"),
dataiter_threads=3,
));
'For a fully connected Wasserstein GAN using RMSProp instead of Adam:
julia --project -O3 -e '
using SMWLevelGenerator;
import Flux;
const d_model = densewsdiscriminator2d(64, 4);
const g_model = densewsgenerator2d(64, 4, 96);
const dbpath = "levels_2d_flags_t.jdb";
const res = gan_trainingloop!(d_model, g_model, dbpath, GTPs(
epochs=1000,
lr=5e-5,
optimizer=Flux.RMSProp,
d_warmup_steps=0,
d_steps_per_g_step=1,
logdir=joinpath("exps", "densewsgan_2d_rmsprop"),
dataiter_threads=3,
));
'For a convolutional metadata predictor:
julia --project -O3 -e '
using SMWLevelGenerator;
const model = metapredictor2d(64);
const dbpath = "levels_2d_flags_t.jdb";
const res = meta_trainingloop!(model, dbpath, MTPs(
epochs=1000,
lr=5e-5,
logdir=joinpath("exps", "meta_2d_low_learning_rate"),
dataiter_threads=3,
));
'For a fully connected metadata predictor:
julia --project -O3 -e '
using SMWLevelGenerator;
const model = densemetapredictor2d(64, 3);
const dbpath = "levels_2d_flags_t.jdb";
const res = meta_trainingloop!(model, dbpath, MTPs(
epochs=1000,
logdir=joinpath("exps", "densemeta_2d_64_3"),
dataiter_threads=3,
));
'We will only list some examples. Once again, please take a look at the
exported functions in
SMWLevelGenerator.jl to get a better idea
of what is possible.
The following writes 16 levels to a ROM (not necessarily all differently numbered, meaning the same level may be overwritten by a later generation).
julia> rompath = joinpath("path", "to", "My Modifiable ROM.smc");
julia> writelevels(predictor, generator, metapredictor,
inputs=16, write_rom=rompath);Predict a whole hack based on the first columns of each level in the vanilla game:
julia> rompath = joinpath("path", "to", "My Modifiable ROM.smc");
julia> predict_vanilla(predictor, db, first_screen=false, write_rom=rompath);Generate a single screen for levels 0x105 and 0x106:
julia> rompath = joinpath("path", "to", "My Modifiable ROM.smc");
julia> writescreen(generator, write_rom=rompath);
julia> writescreen(generator, write_rom=rompath, number=0x106);To load a model, you need to import or be using all the packages
the model uses as well (good candidates are Flux and/or
Transformers).
Due to a bug in JLD2, you will additionally need to import the
module containing any GANs you may want to load. For example, for a
stored WassersteinGAN, do the following:
julia> using SMWLevelGenerator, Flux, JLD2
julia> import SMWLevelGenerator.WassersteinGAN # Change this line if necessary.
julia> generator = jldopen(joinpath("exps",
"my_best_wsgan_checkpoint.jld2")) do cp
# This _must_ not give a warning like "Warning: type ... does
# not exist in workspace; reconstructing"!
generator = cp["g_model"] # Note that the keys are `String`s.
end;Other models do not need this extra line.
These checkpoints are easier to use but are less stable than the JLD2 checkpoints, meaning you may experience unrecoverable checkpoints (in other words, data loss). For these, you do something like this:
julia> using SMWLevelGenerator, Flux, BSON
julia> cp = BSON.load(joinpath("exps", "my_best_wsgan_checkpoint.jld2"));
Dict{Symbol,Any} [...]
julia> generator = cp[:g_model]; # Note that the keys are `Symbol`s.See stats/authors.txt for a list of authors or
stats/hack_stats.csv for detailed
information about each hack.
SMWLevelGenerator - Generating Super Mario World Levels Using Deep Neural Networks
Copyright (C) 2019 Jan Ebert
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; only version 2 of the License (GPL-2.0-only).
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
Contact:
domain.de | local-part
----------+---------------
posteo | janpublicebert