Instructions for Adding New Games/States

[MaxStrange]

Issue summary

There is no documentation that I can find that explains how to add new games or new states.

By copying the site-packages/retro/data/SonicTheHedgehog-Genesis folder structure with a different ROM, I can get that ROM to show up as a game choice in retro.

The README file in this repo explains that states are just emulator states, but gzipped. But when I use Genesis Plus GX to save state and then gzip and copy that state into the folder I created above, the state can get loaded but it does not seem to work.

For example, when using SonicTheHedgehog-Genesis and saving a state in the GreenHillZone-Act1 stage, and then gzipping and putting that state into the SonicTheHedgehog-Genesis folder, I can load the state, but an agent cannot seem to control it.

Details on the emulator core used (and version) and steps taken to produce a working state file would be much appreciated for those trying to add new games or states to retro.

System information

• OS: Windows 10
• Python version: 3.6
• Gym Retro version: 0.5.4

[endrift]

First, as savestate formats are volatile, you need to use the right version of the Genesis Plus GX libretro core: https://github.com/libretro/Genesis-Plus-GX/tree/f66bcb1e780acdcced95a05e04550bb8a91c1850 -- you can just copy the .dll from retro/cores to wherever your libretro frontend (e.g. RetroArch) looks for cores.

Second, we are working on providing instructions on doing all of this in depth, but it won't be ready for a few weeks.

[MaxStrange]

This works. Thanks so much!

[Scitator]

hi @endrift!
Is there any updates about your states adding?
By the way, @MaxStrange ,I tried to load retro/cores/ to other simulators (like RetroArch or BizHawk), but all they gave an core loading error. Or state loading error with their default core.

UPD: I finally get it work, but with strange artefacts:

@endrift Have you see something like this before?

UPD: finally, it works! I started all over again and it worked. Looks like I had a corrupted rom.

[MaxStrange]

How to install your own Game and States in Retro:

Note that these instructions are written for version 0.5.4 of retro and are thus likely to break as the repo changes and becomes more stable

• Install retro: https://github.com/openai/retro
• Obtain the ROM you are interested in; right now only Genesis and Atari 2600 ROMs work
• Make a game folder in <python_install_path>/Lib/site-packages/retro/data
  E.g. If we want to make a Street Fighter II instance: name the folder StreetFighterII-Genesis
• Add the ROM to this folder and rename it to rom.md (for Genesis) or rom.a26 (for Atari 2600)
• Download and install the RetroArch emulator: http://www.retroarch.com/?page=platforms
• Take the .dll or .so or whatever from <python_install_path>/Lib/site-packages/retro/cores and put them wherever RetroArch has its cores directory. On Windows, this was: C:\Users<yourname>\AppData\Roaming\RetroArch\cores. Good luck on other platforms.
• Open RetroArch
  • Navigate to load core->Sega MS/GG/MD/CD (Genesis Plus GX) (or whatever you put in the folder)
    For Atari: load core->Atari 2600 (Stella) (or whatever you put in the folder)
  • Now go back to the main menu and scroll to Load Content
  • Find your ROM and load it
  • Play to wherever you want your Agent to be able to start at
  • Save the state using the menu: Command->Save State Options->Save State
  • Take note of which State slot it saved it to - the name of the file will reflect this
  • Do this for all the states you want
  • Close RetroArch
• Find where RetroArch saved your state - on Windows, this will likely be in C:\Users<yourname>\AppData\Roaming\RetroArch\states - I haven’t checked on other platforms where these files would be located, but my guess is that it is equally obscure
• Move the states (they should end in .state) to the folder you made in Python’s site-packages (with your ROM) and rename them to whatever you want them to be called (but they must end in .state)
• Gzip each state - don’t change its name as part of this - they have to end in .state
• Create a file called “data.json” and put this in it:

{
  “info”: {
  }
}

• Create a file called “scenario.json” and put this in it:

{
}

• Create a file called “metadata.json” and put this in it:

{
  “default_state”: “Name of the state you want to set to default, minus the .state ending”
}

You should now be able to run the example Agent script: https://github.com/openai/retro/blob/master/examples/random_agent.py by passing in the name of your game (E.g. “StreetFighterII-Genesis” and (potentially) the state name “MyState”).

Some of these steps are likely not needed, but I put them here because I don’t want to bother trying out what is and what isn’t important.

[arthurolga]

Hi @MaxStrange I tried to do that but failed at saving the states. Even tried to save a new state to an already playable game(Streets of Rage). How do ou save the state?

Tried:
Going to the level I want(start of SF2 battle) in Retroarch
Opening Quick menu
Selecting Save New State
Close
Changing name and gzip
Took off the .gzip

Also tried to change name after gzip. Did i make any noticable mistake? Random_agent runs for about 8 frames
Anyways, thanks for posting this instructions

[MaxStrange]

Odd that it works for 8 frames and not just not at all. You are certain you are using the right .dll or .so? If the state can be loaded (before gzipping) into RetroArch using the same shared lib that this repo is using, it should work fine. Are you confident that the .dll/.so you are using in RetroArch is the same one that your installed retro is using?

Another thing to worry about is that you may need to define a 'done' in the scenario.json, which will likely involve digging around in the RAM of your target game (I recommend BizHawk for this).

[arthurolga]

I copied the .dll from Gym-Retro to retro/cores and managed to get the game running in Retro. It runs well, but I guess I will have to dig through the RAM to define "done".

Any idea how can I learn to do that with BizHawk?

[MaxStrange]

This link has some pointers, but honestly the BizHawk toolset for this purpose is top-notch. Just download the BizHawk emulator, run the ROM in it, then tools->Ram Search. Play around with it to get the hang of it.

Things to note:

• Genesis is a 16-bit platform, so most things will not be more than 16-bit, and many values will be a single byte.
• If you are looking for a value that is likely to be represented in the RAM as true/false, it will probably be 0 for false, but could be anything else for true.
• Once you've narrowed down a value of interest to one or a few addresses, try to disprove them by pausing, unpausing, dying, game-overing, restarting the console, etc.
• Most interesting values I have found are unsigned
• Scores are often multiplied by a constant factor (usually 10 or 100) before displaying them to save on memory, so if you can't find your score in the RAM, try dividing it by 10 or 100
• I have found a few scores in games are represented in binary-coded decimal, which is something of a pain if you don't realize it. To check if a score (or whatever) is represented in the RAM in BCD, find a hex calculator and convert the displayed score (let's say 4521) to decimal (0x4521 -> 17697). Search for that value instead of what you see on screen.
• NOTE Remember that the base address is decimal 16711680 (0xFF0000); BizHawk subtracts this off for you, but retro's JSON files do not want it subtracted off, so whatever addresses you find in BizHawk will need to have this value added to them to map them into the appropriate address range.

[arthurolga]

It worked! You explained very well, going to put a "Special thanks to MaxStrange" on my project. Thanks for helping a newbie!

[endrift]

Official instructions are now on the https://github.com/openai/retro/tree/develop branch. Check https://github.com/openai/retro/blob/develop/IntegratorsGuide.md

[kalashnikovisme]

#33 (comment)

Great instruction!!! Thanks!!

• Default directory for cores in Ubuntu with retroarch installed via snap is ~/snap/retroarch/current/.config/retroarch/cores/
• Default directory with save states in Ubuntu with retroarch installed via snap is ~/snap/retroarch/current/.config/retroarch/states

UPD: These directories become available after first run on RetroArch.

[kalashnikovisme]

Unfortunately, I have issue with loading saved state.

I've tested it and it works in RetroArch. But when I start learning process it runs game from the beginning and ignoring my new state.

[Traviktox]

I have the same problem as the previous post

[kalashnikovisme]

Still nobody can help?)

[thegamenicorus]

Thanks for the instruction.

btw, same as all you guys, I also have an issue with loading state via retro.make. I've tried many old versions of RetroArch both x86 and x64 but nothing works. And finally, I find the way out (at least for my NES ROM) by using the Gym retro integration UI instead. you can download it here: https://github.com/openai/retro/releases/tag/f347d7e

1. After extracted, copy all files in retro/cores/ to the cores folder and run Gym Retro Integration.exe
2. Load your ROM by menu Game > Load game..., hope it running.
3. Save game state by menu Game > Save state..., you'll get .state file that works with retro.make, no need to gzip before use.