indxparser.py not working as expected

[hiddenillusion]

I ran into some issues using the latest commit but had mixed results using a previous version.

I received the following IndexError on the first image:

(test)root@heyWilli:~/python-ntfs-master# python examples/indxparse/indxparse.py /mnt/001/ewf1 $((2048*512)) /
DEBUG:ntfs.filesystem:mft: 0xc0000000
DEBUG:ntfs.filesystem:NonResidentAttributeData: len: run: cluster: 0xc0000 len: 0x5500
DEBUG:ntfs.filesystem:NonResidentAttributeData: len: run: cluster: 0x987618 len: 0x5f00
Traceback (most recent call last):
  File "examples/indxparse/indxparse.py", line 186, in <module>
    main(sys.argv[1], int(sys.argv[2]), sys.argv[3])
  File "examples/indxparse/indxparse.py", line 173, in main
    fs = NTFSFilesystem(v)
  File "build/bdist.linux-x86_64/egg/ntfs/filesystem/__init__.py", line 482, in __init__
  File "build/bdist.linux-x86_64/egg/ntfs/filesystem/__init__.py", line 388, in __getitem__
IndexError: string index out of range

and a CorruptNTFSFilesystemErorr on both NTFS partitions of the 2nd image (only showing output from 2nd) partition fail :

(test)root@heyWilli:~/python-ntfs-master# python examples/indxparse/indxparse.py /mnt/002/ewf1 $((409657*512)) /
DEBUG:ntfs.filesystem:mft: 0xf7304a8c81cefc7a000L
WARNING:ntfs.filesystem:failed to read MFT from image, will fall back to MFTMirr: Tried to parse beyond the end of the file (read: 0x2c, buffer length: 0x0)
DEBUG:ntfs.filesystem:mft mirr: 0x4aa15743498024da000L
ERROR:ntfs.filesystem:failed to read MFTMirr from image: Tried to parse beyond the end of the file (read: 0x406, buffer length: 0x0)
Traceback (most recent call last):
  File "examples/indxparse/indxparse.py", line 186, in <module>
    main(sys.argv[1], int(sys.argv[2]), sys.argv[3])
  File "examples/indxparse/indxparse.py", line 173, in main
    fs = NTFSFilesystem(v)
  File "build/bdist.linux-x86_64/egg/ntfs/filesystem/__init__.py", line 493, in __init__
ntfs.filesystem.CorruptNTFSFilesystemError: CorruptNTFSFilesystemError(failed to read MFT or MFTMirr from image) 

When I tested against an older version indxparse.py it worked fine against the 2nd partition of the 2nd disk:

(ntfs_old)root@heyWilli:~/python-ntfs-old# python examples/indxparse/indxparse.py /mnt/002/ewf1 $((40965750*512)) /
active,\,$AttrDef,36864,36000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$BadClus,0,0,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$Bitmap,14606336,14605096,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$Boot,8192,8192,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$Extend,0,0,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$LogFile,67108864,67108864,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$MFT,16384,16384,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$MFTMirr,4096,4096,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$Secure,0,0,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$UpCase,131072,131072,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
active,\,$Volume,0,0,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000,2009-04-28 15:56:01.750000
...

but failed on the first partitions on both disks. To help track down what might have been modified over commits, here's the traceback from the OverrunBufferException on the first 2 partitions using an older version of indxparser.py:

(ntfs_old)root@heyWilli:~/python-ntfs-old# python examples/inspect_directory/inspect_directory.py /mnt/002/ewf1 $((63*512)) /
Traceback (most recent call last):
  File "examples/inspect_directory/inspect_directory.py", line 72, in <module>
    main(sys.argv[1], int(sys.argv[2]), sys.argv[3])
  File "examples/inspect_directory/inspect_directory.py", line 25, in main
    fs = NTFSFilesystem(v)
  File "build/bdist.linux-x86_64/egg/ntfs/filesystem/__init__.py", line 443, in __init__
  File "build/bdist.linux-x86_64/egg/ntfs/filesystem/__init__.py", line 460, in get_mft_buffer
  File "build/bdist.linux-x86_64/egg/ntfs/mft/MFT.py", line 908, in __init__
    def is_directory(self):
  File "build/bdist.linux-x86_64/egg/ntfs/mft/MFT.py", line 58, in fixup
    class FixupBlock(Block):
  File "build/bdist.linux-x86_64/egg/ntfs/BinaryParser.py", line 817, in unpack_binary
    """
ntfs.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x0, buffer length: 0x200)

The E01's were successfully mounted with ewfmount & mount so there shouldn't be any issues with reading the $MFT etc. Here's the mmls output:

(test)root@heyWilli:~/python-ntfs# mmls /mnt/001/ewf1
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

     Slot    Start        End          Length       Description
00:  Meta    0000000000   0000000000   0000000001   Primary Table (#0)
01:  -----   0000000000   0000002047   0000002048   Unallocated
02:  00:00   0000002048   0625139711   0625137664   NTFS (0x07)
03:  -----   0625139712   0625142447   0000002736   Unallocated


(test)root@root@heyWilli:~/python-ntfs# mmls /mnt/002/ewf1
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

     Slot    Start        End          Length       Description
00:  Meta    0000000000   0000000000   0000000001   Primary Table (#0)
01:  -----   0000000000   0000000062   0000000063   Unallocated
02:  00:00   0000000063   0040965749   0040965687   NTFS (0x07)
03:  00:01   0040965750   0975691709   0934725960   NTFS (0x07)
04:  -----   0975691710   0975699967   0000008258   Unallocated

[williballenthin]

this is possibly fixed in #4

[paultag]

i'm seeing this as well on latest git, using an image from the internet

I get

Error in handler <ext.disk.Partition object at 0x7f12bf025cd0>: CorruptNTFSFilesystemError(failed to read MFT or MFTMirr from image)

Code is:

            with Mmap(file) as buf:
                v = FlatVolume(buf, start_bytes)
                fs = NTFSFilesystem(v)
                root = fs.get_root_directory()

                print(root)

[williballenthin]

thanks @paultag for pointing to the public image. let me take a look and report back.