Can't read game files of 1.50.

[jorensanbar]

In new version of the games, the _hashHeader.Version change to 2, so the software can't access to SCS Files.

HashArchiveFile CLASS

if (_hashHeader.Version != SupportedHashVersion)
{
Logger.Instance.Error("Unsupported Hash Version");
return false;
}

[jorensanbar]

SCS Build another SCS Extractor.

https://modding.scssoft.com/wiki/Documentation/Tools/Game_Archive_Extractor#Download

[meatlayer]

It looks like new DLC Guards have also appeared in 1.50 beta ATS

	new DlcGuard("dlc_ne", 32, false),
	new DlcGuard("dlc_ne_and_co", 33, false),
	new DlcGuard("dlc_ne_and_ks", 34, false),
	new DlcGuard("dlc_ne_and_wy", 35, false),

[dowmeister]

Someone has a working version?

[VTLog-Legacy]

Lets get together on this, hash has changed. Is there a way to avoid using the .scs files and start using the extracted ones for now? (bypass)

[jorensanbar]

Yes, this tool can read ZIP files.

Then extract all and compress in ZIP files (Normal method), then change the GetFile Method of AddFolder in TSMapper from: GetFiles(.scs) to GetFiles(.zip)

[dowmeister]

thanks @jorensanbar, that is brilliant idea!

[dowmeister]

@jorensanbar as you suggested, i've unpacked all .scs files in a single directory then packed the whole content in a .zip file with 7zip and compression Normal - i've tried also with compression = None
So now i have a big 20gb zip file containing all raw files inside.

But throw an exception at ZipArchiveFile.cs line 97, seems can't understand the internal zip content.

var parentDirPath = Path.GetDirectoryName(name).Replace('\\', '/');

Have also this warning:
| Debug] [ZipArchiveFile.cs::Parse] Local name length is different than CD one for zip entry 0 '' in '<zip file>'

What i'm missing? :D

[jorensanbar]

SCS_Extractor_1.50 cannot access some bytes of files due to HASH V2. So the extraction is incomplete.

Compress only the paths that your application really requires.

[dowmeister]

thanks @jorensanbar but still cannot get it to work.
I've unpacked every single file and then repacked in a zip from 7zip with normal compression base, base_map, def, base_vehicle and base_share

Strange thing: altought i have the base.scs (assuming is ETS2 1.50 that one in your screenshot) at same size, when repacked the size is not like yours, is over 6gb.

Evidently i'm messing with something...

[jorensanbar]

I've unpacked every single file

No, you didn't because the SCS_Extractor does not decompress 100% of the SCS file, HASH V2 does not allow it.

For this reason if you recreate a ZIP by selecting the entire folder, it will be corrupted.

You only have to keep the paths used by TS_MAP, that's why my ETS2_1.50 Base.scs file is only 1.5GB in size, even though it was originally 8GB.

[meatlayer]

It worked for me. Using some *.scs files
scs_extractor.exe decrypt this list of files:

!!! base.scs !!!
base_map.scs
def.scs
dlc_balkan_e.scs
dlc_balkan_w.scs
dlc_balt.scs
dlc_east.scs
dlc_feldbinder.scs
dlc_fr.scs
dlc_iberia.scs
dlc_it.scs
dlc_north.scs

The contents of each decrypted file must be packed in .zip without compression, then rename to .scs
But, from the !!! base.scs !!! need only folders (prefab, prefab2, material/ui) after adding them to base.zip and renamed to base.scs, turned out to be ~2.8 GB in size for the base.scs file. It may be necessary to add other folders besides prefab and prefab2, but for me it was enough for the map to be rendered...

[dowmeister]

I went a bit in deep than unpacking and repacking without going to select single files.

The code below is really initial, wrote quickly yesterday but it works: i've simulated the UberFilesystem logic, instead of reading from .scs files or .zip, are read directly from the filesystem

In UberFilesyste.AddSourceDirectory replace this code and other relevant following:

 foreach (var scsFilePath in scsFilesPaths)
 {
     var fileResult = AddSourceFile(scsFilePath);
     if (!fileResult) result = false;
 }

with

var directories = Directory.GetDirectories(path);

UberDirectory parentDir;

foreach (string dir in directories)
{
    string directoryName = new DirectoryInfo(dir).Name;

    var parentDirHash = CityHash.CityHash64(directoryName);

    if (this.Directories.ContainsKey(parentDirHash))
    {
        parentDir = this.Directories[parentDirHash];
    }
    else
    {
        parentDir = new UberDirectory();
        parentDir.VirtualPath = new DirectoryInfo(dir).Name;
        this.AddLocalFilesRecursive(parentDir, new DirectoryInfo(dir).Name, dir);
        this.Directories.Add(parentDirHash, parentDir);
    }
}

and add

private void AddLocalFilesRecursive(UberDirectory uberDirectory, string virtualPath, string realPath)
{
    var files = Directory.GetFiles(realPath, "*.*");

    foreach (string file in files)
    {
        string fileVirtualPath = virtualPath + "/" + new FileInfo(file).Name;
        this.Files.Add(CityHash.CityHash64(fileVirtualPath), new UberFile(new LocalFileEntry(file)));
        uberDirectory.AddSubFileName(new FileInfo(file).Name);
    }

    var dirs = Directory.GetDirectories(realPath);

    foreach (string dir in dirs)
    {
        uberDirectory.AddSubDirName(new DirectoryInfo(dir).Name);

        var subUberDir = new UberDirectory();                
        subUberDir.VirtualPath = virtualPath + "/" + new DirectoryInfo(dir).Name;
        this.AddLocalFilesRecursive(subUberDir, virtualPath + "/" + new DirectoryInfo(dir).Name, dir);
        this.Directories.Add(CityHash.CityHash64(virtualPath + "/" + new DirectoryInfo(dir).Name), subUberDir);
    }
}

creates a new class called LocalFileEntry under TsMap.FileSystem

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using TsMap.FileSystem.Zip;
using TsMap.Helpers;

namespace TsMap.FileSystem
{
    public class LocalFileEntry : Entry
    {
        public LocalFileEntry(string path) : base(path)
        {
        }

        public override bool IsCompressed()
        {
            return false;
        }

        public override bool IsDirectory()
        {
            throw new NotImplementedException();
        }

        public override byte[] Read()
        {
            return File.ReadAllBytes(this._path);
        }

        protected override byte[] Inflate(byte[] buff)
        {
            throw new NotImplementedException();
        }
    }
}

modify Entry.cs adding

 internal string _path;

 public Entry(ArchiveFile fsFile)
 {
     _archiveFile = fsFile;
 }

Be aware also you must replace all references with code similar to

Logger.Instance.Error($"Unknown base file version ({Sector.Version}) for item {Type} " +
                      $"in file '{Path.GetFileName(Sector.FilePath)}' @ {startOffset} from '{Sector.GetUberFile().Entry.GetArchiveFile().GetPath()}'");

with Sector.GetUberFile().Entry.GetArchiveFile()?.GetPath() otherwise will throw an error when a reference is not found.

With this is not necessary repack all files but can work on all extracted files.

About locale.scs and others files without a root directory, i've discovered this: https://github.com/sk-zk/Extractor

[meatlayer]

Did any of you guys manage to solve the problem related to the red background of the POI icons of gas stations and recreation centers? Before version 1.50, they were the correct blue and green colors.

[dowmeister]

maybe we can replace them with custom images? they would be better than previous ones :D

[meatlayer]

I went a bit in deep than unpacking and repacking without going to select single files.

The code below is really initial, wrote quickly yesterday but it works: i've simulated the UberFilesystem logic, instead of reading from .scs files or .zip, are read directly from the filesystem

In UberFilesyste.AddSourceDirectory replace this code and other relevant following:

 foreach (var scsFilePath in scsFilesPaths)
 {
     var fileResult = AddSourceFile(scsFilePath);
     if (!fileResult) result = false;
 }

with

var directories = Directory.GetDirectories(path);

UberDirectory parentDir;

foreach (string dir in directories)
{
    string directoryName = new DirectoryInfo(dir).Name;

    var parentDirHash = CityHash.CityHash64(directoryName);

    if (this.Directories.ContainsKey(parentDirHash))
    {
        parentDir = this.Directories[parentDirHash];
    }
    else
    {
        parentDir = new UberDirectory();
        parentDir.VirtualPath = new DirectoryInfo(dir).Name;
        this.AddLocalFilesRecursive(parentDir, new DirectoryInfo(dir).Name, dir);
        this.Directories.Add(parentDirHash, parentDir);
    }
}

and add

private void AddLocalFilesRecursive(UberDirectory uberDirectory, string virtualPath, string realPath)
{
    var files = Directory.GetFiles(realPath, "*.*");

    foreach (string file in files)
    {
        string fileVirtualPath = virtualPath + "/" + new FileInfo(file).Name;
        this.Files.Add(CityHash.CityHash64(fileVirtualPath), new UberFile(new LocalFileEntry(file)));
        uberDirectory.AddSubFileName(new FileInfo(file).Name);
    }

    var dirs = Directory.GetDirectories(realPath);

    foreach (string dir in dirs)
    {
        uberDirectory.AddSubDirName(new DirectoryInfo(dir).Name);

        var subUberDir = new UberDirectory();                
        subUberDir.VirtualPath = virtualPath + "/" + new DirectoryInfo(dir).Name;
        this.AddLocalFilesRecursive(subUberDir, virtualPath + "/" + new DirectoryInfo(dir).Name, dir);
        this.Directories.Add(CityHash.CityHash64(virtualPath + "/" + new DirectoryInfo(dir).Name), subUberDir);
    }
}

creates a new class called LocalFileEntry under TsMap.FileSystem

using System; using System.Collections.Generic; using System.IO; using System.Linq; using System.Text; using System.Threading.Tasks; using TsMap.FileSystem.Zip; using TsMap.Helpers;

namespace TsMap.FileSystem
{
    public class LocalFileEntry : Entry
    {
        public LocalFileEntry(string path) : base(path)
        {
        }

        public override bool IsCompressed()
        {
            return false;
        }

        public override bool IsDirectory()
        {
            throw new NotImplementedException();
        }

        public override byte[] Read()
        {
            return File.ReadAllBytes(this._path);
        }

        protected override byte[] Inflate(byte[] buff)
        {
            throw new NotImplementedException();
        }
    }
}

modify Entry.cs adding

 internal string _path;

 public Entry(ArchiveFile fsFile)
 {
     _archiveFile = fsFile;
 }

Be aware also you must replace all references with code similar to

Logger.Instance.Error($"Unknown base file version ({Sector.Version}) for item {Type} " +
                      $"in file '{Path.GetFileName(Sector.FilePath)}' @ {startOffset} from '{Sector.GetUberFile().Entry.GetArchiveFile().GetPath()}'");

with Sector.GetUberFile().Entry.GetArchiveFile()?.GetPath() otherwise will throw an error when a reference is not found.

With this is not necessary repack all files but can work on all extracted files.

About locale.scs and others files without a root directory, i've discovered this: https://github.com/sk-zk/Extractor

@dowmeister Will you be able to publish the modified code of the entire project in your repository in some separate experimental branch? I didn't manage to apply this changes to the code successfully.

[dowmeister]

sure @meatlayer , give me few days, for the weekend.
I've left them unpublished hoping it was only a workaround.
I'm a bit busy these days, bear with me please

[jorensanbar]

BTW, TS-MAP is already updated for HASH v2