I am trying to uncompress the Amstrad E3 cramfs file system
I managed to uncompress its cramfs file system partly, so some of the files are missing data so are all 0s and some files are OK, am I missing something, can't work out why, can anyone here help me, that's if anyone's around....? This is how I did it: About the cramfs and how to get it off the NAND Dump The OOB is 16 bytes after every block of 512 bytes. So for every 528 bytes you need to remove the last 16 bytes. In the E3-hacking email thread there's they Python script, that does it but also a simple command-line perl program that does it as well but modifies the existing file so you better copy it first: # cp e3-nand-backup.4 e3-nand-stripped.4 LC_ALL=C perl -i -0777pe 's/(.{512})(.{16})/$1/gs' e3-nand-stripped.4 So then e3-nand-stripped.4 is the copy of the nand part where the cramfs filesystems are without the OOB bytes. Now, to extract the actual cramfs parts you need to find where they are. From the E3-hacking email thread we know that each section in the nand starts with Q;Q; and also that the filesystem sections also have the word LNXFSYS. So it is easy to search for in hexedit. The first one you will find is at 0x1E4000: 001E4000 51 3B 51 3B 00 00 DB 2E 00 D1 1D 00 01 00 01 00 00 00 02 00 4C 4E 58 46 53 59 53 00 00 00 00 00 Q;Q;................LNXFSYS..... 001E4020 00 00 00 00 40 28 23 29 69 6E 69 74 72 64 2E 63 72 61 6D 20 41 4D 53 3A 30 30 30 31 00 00 00 00 ....@(#)initrd.cram AMS:0001.... 001E4040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................................ 001E4060 00 00 00 00 F8 00 10 11 13 B5 13 B5 00 00 10 11 00 00 00 00 00 00 00 00 00 00 00 00 FF FF FF FF ................................ 001E4080 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40A0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40C0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40E0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 00 00 00 00 00 00 00 00 ................................ 001E4100 45 3D CD 28 00 00 01 00 00 00 00 00 00 00 00 00 43 6F 6D 70 72 65 73 73 65 64 20 52 4F 4D 46 53 E=.(............Compressed ROMFS 001E4120 DC 49 D9 64 DE 0D 9A 00 68 8D F8 40 CF 52 8B 66 43 6F 6D 70 72 65 73 73 65 64 00 00 00 00 00 00 .I.d....h..@.R.fCompressed...... If you look more closely you see 0x1E4000-0x1E40FF is probably the header for this section with some info probably for the nand driver that Amstrad used? But as it is publicly information that a CRAMFS filesystem starts with a signature of 45 3D CD 28 it's easy to figure out where the acutal CRAMFS data starts (0x1E4100). Acording to how a CRAMFS system header is put together (https://www.kernel.org/doc/Documentation/filesystems/cramfs.txt) there some more info to be had: 00 ulelong 0x28cd3d45 CRAMFS signature 04 ulelong 0x00010000 size = 65535 08 ulelong 0x00000000 flags 12 ulelong 0x00000000 future 0x%x 16 string Compressed ROMFS signature 32 ulelong 0x64D949DC checksum 36 ulelong 0x009A0DDE edition 40 ulelong 0x40F88D68 #of blocks = **1090030952** (!!!!!) 44 ulelong 0x668B52CF #of files = **1720406735** (!!!!!) 48 string Compressed name But you already see something is off because the #blocks would be 0x40F88D68 and the number of files 0x668B52CF. Which is much too many, So this is already an indication that there is something odd. The next Q;Q; block starts at 0x464000 (this is the other CRAMFS filesystem, there are 2 cramfs) so the maximum length that the first CRAMFS data could be is 0x464000 - 0x1E4100 = 0x26FF00 = 2555648. This is a lot larger than the size in the header (65535) so also a bit fishy. They probably left a lot of room spare. If you look at the hex data the last bit of 'random' data ends at 0x3C093C, then it is 00's until 0x3C1100 and from there on FF's until 0x464000. But if you look at the start + 65535 it doesn't look like that is the real end of the file, the data seems to continue. So it is best to use the data at least up to where the 00's end. From this I assume that the start of the first CRAMFS system is at 0x1E4100 (6439168) and the end at 0x3C10FF so a length of 0x11DD000 (1953792). To dump this you then do: # dd if=e3-nand-stripped.4 of=cramfs-1 bs=1 skip=1982720 count=1953792 And from there on it is using standard linux tools for cramfs to try to extract it: # cramfsck -help usage: cramfsck [-hv] [-x dir] file -h print this help -x dir extract into dir -v be more verbose file file to test # sudo `mount -t cramfs cramfs-1 /mnt But these won't work because of the data being somehwat different. This one I compiled from source and it does a bit more uncramfs-fmk 57.41 KB # uncramfs-fmk v0.7 by Andrew Stitcher Usage: 'uncramfs-fmk [-d devfilename] [-m modefilename] dirname infile' where <dirname> is the root for the uncompressed (output) filesystem. # uncramfs-fmk e3-cramfs-1-root cramfs-1 This will extract it to a directory e3-cramfs-1-root and also output lot's of info including 'Uncompression failed; for each file wher it fails, Also: [Volume size: 0x1dd000] [Volume serial: dc49d964de0d9a00688df840cf528b66] [Volume name: Compressed] <lots of stuff removed> [Summary:] [Total uncompressed size: 4492954] [Total compressed size: -760580457] [Number of entries: 341] [Number of files compressed: 202] [Number of files expanded: 139] So there are 341 files in total apparently. And the size is 0x1dd000 which is different from 0x10000 in the header... To get the second cramfs you do the same analysis on the next Q;Q block and you arrive at # dd if=e3-nand-stripped.4 of=cramfs-2 bs=1 skip=4604160 count=1953792 And the extraction summary is: Volume size: 0x1dd000] [Volume serial: d07c38cb5ca01614f37d520c4e076ab6] [Volume name: Compressed] [Summary:] [Total uncompressed size: 4093206] [Total compressed size: -1905520397] [Number of entries: 280] [Number of files compressed: 141] [Number of files expanded: 139]
On Sunday 24 December 2023 at 12:17:21, Tina Poole wrote:
I managed to uncompress its cramfs file system partly, so some of the files are missing data so are all 0s and some files are OK, am I missing something, can't work out why, can anyone here help me, that's if anyone's around....?
I don't have an answer to your question/s, although I sincerely hope that someone else does. I just wanted to say that I'm very pleased to see further dialogue on this list. Merry Christmas, or however people choose to enjoy this time of year. Antony. -- What do you get when you cross a joke with a rhetorical question? Please reply to the list; please *don't* CC me.
Now, here's a thing I haven't heard for a long time! I wish I hadn't gotten rid of my E3s... I haven't touched flash with OOB directly, but it's possible that some sectors have been marked as unused by data in the OOB itself. e.g. to mark bad sectors in the flash. So you may need to be able to parse this data in order to determine whether the sector is really in use or you could be seeing a corrupt version of the data. Even if the sector is valid, there could also be error-correcting data in the OOB which needs processing in order to recover the correct version of the sector, otherwise there could be bit flips. I have a faint memory that there's a MTD flash emulator for Linux which does this for you, but I've never used it. On Sun, 24 Dec 2023 at 11:17, Tina Poole <span1922@live.com> wrote:
I managed to uncompress its cramfs file system partly, so some of the files are missing data so are all 0s and some files are OK, am I missing something, can't work out why, can anyone here help me, that's if anyone's around....?
This is how I did it: About the cramfs and how to get it off the NAND Dump
The OOB is 16 bytes after every block of 512 bytes. So for every 528 bytes you need to remove the last 16 bytes. In the E3-hacking email thread there's they Python script, that does it but also a simple command-line perl program that does it as well but modifies the existing file so you better copy it first: # cp e3-nand-backup.4 e3-nand-stripped.4
LC_ALL=C perl -i -0777pe 's/(.{512})(.{16})/$1/gs' e3-nand-stripped.4
So then e3-nand-stripped.4 is the copy of the nand part where the cramfs filesystems are without the OOB bytes.
Now, to extract the actual cramfs parts you need to find where they are. From the E3-hacking email thread we know that each section in the nand starts with Q;Q; and also that the filesystem sections also have the word LNXFSYS. So it is easy to search for in hexedit. The first one you will find is at 0x1E4000: 001E4000 51 3B 51 3B 00 00 DB 2E 00 D1 1D 00 01 00 01 00 00 00 02 00 4C 4E 58 46 53 59 53 00 00 00 00 00 Q;Q;................LNXFSYS..... 001E4020 00 00 00 00 40 28 23 29 69 6E 69 74 72 64 2E 63 72 61 6D 20 41 4D 53 3A 30 30 30 31 00 00 00 00 ....@(#)initrd.cram AMS:0001.... 001E4040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................................ 001E4060 00 00 00 00 F8 00 10 11 13 B5 13 B5 00 00 10 11 00 00 00 00 00 00 00 00 00 00 00 00 FF FF FF FF ................................ 001E4080 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40A0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40C0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40E0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 00 00 00 00 00 00 00 00 ................................ 001E4100 45 3D CD 28 00 00 01 00 00 00 00 00 00 00 00 00 43 6F 6D 70 72 65 73 73 65 64 20 52 4F 4D 46 53 E=.(............Compressed ROMFS 001E4120 DC 49 D9 64 DE 0D 9A 00 68 8D F8 40 CF 52 8B 66 43 6F 6D 70 72 65 73 73 65 64 00 00 00 00 00 00 .I.d....h..@.R.fCompressed......
If you look more closely you see 0x1E4000-0x1E40FF is probably the header for this section with some info probably for the nand driver that Amstrad used? But as it is publicly information that a CRAMFS filesystem starts with a signature of 45 3D CD 28 it's easy to figure out where the acutal CRAMFS data starts (0x1E4100). Acording to how a CRAMFS system header is put together ( https://www.kernel.org/doc/Documentation/filesystems/cramfs.txt) there some more info to be had: 00 ulelong 0x28cd3d45 CRAMFS signature 04 ulelong 0x00010000 size = 65535 08 ulelong 0x00000000 flags 12 ulelong 0x00000000 future 0x%x 16 string Compressed ROMFS signature 32 ulelong 0x64D949DC checksum 36 ulelong 0x009A0DDE edition 40 ulelong 0x40F88D68 #of blocks = **1090030952** (!!!!!) 44 ulelong 0x668B52CF #of files = **1720406735** (!!!!!) 48 string Compressed name
But you already see something is off because the #blocks would be 0x40F88D68 and the number of files 0x668B52CF. Which is much too many, So this is already an indication that there is something odd.
The next Q;Q; block starts at 0x464000 (this is the other CRAMFS filesystem, there are 2 cramfs) so the maximum length that the first CRAMFS data could be is 0x464000 - 0x1E4100 = 0x26FF00 = 2555648. This is a lot larger than the size in the header (65535) so also a bit fishy. They probably left a lot of room spare. If you look at the hex data the last bit of 'random' data ends at 0x3C093C, then it is 00's until 0x3C1100 and from there on FF's until 0x464000. But if you look at the start + 65535 it doesn't look like that is the real end of the file, the data seems to continue. So it is best to use the data at least up to where the 00's end. From this I assume that the start of the first CRAMFS system is at 0x1E4100 (6439168) and the end at 0x3C10FF so a length of 0x11DD000 (1953792). To dump this you then do: # dd if=e3-nand-stripped.4 of=cramfs-1 bs=1 skip=1982720 count=1953792
And from there on it is using standard linux tools for cramfs to try to extract it: # cramfsck -help usage: cramfsck [-hv] [-x dir] file -h print this help -x dir extract into dir -v be more verbose file file to test
# sudo `mount -t cramfs cramfs-1 /mnt
But these won't work because of the data being somehwat different.
This one I compiled from source and it does a bit more
uncramfs-fmk 57.41 KB
# uncramfs-fmk v0.7 by Andrew Stitcher Usage: 'uncramfs-fmk [-d devfilename] [-m modefilename] dirname infile' where <dirname> is the root for the uncompressed (output) filesystem.
# uncramfs-fmk e3-cramfs-1-root cramfs-1
This will extract it to a directory e3-cramfs-1-root and also output lot's of info including 'Uncompression failed; for each file wher it fails, Also: [Volume size: 0x1dd000] [Volume serial: dc49d964de0d9a00688df840cf528b66] [Volume name: Compressed]
<lots of stuff removed>
[Summary:] [Total uncompressed size: 4492954] [Total compressed size: -760580457] [Number of entries: 341] [Number of files compressed: 202] [Number of files expanded: 139]
So there are 341 files in total apparently.
And the size is 0x1dd000 which is different from 0x10000 in the header...
To get the second cramfs you do the same analysis on the next Q;Q block and you arrive at # dd if=e3-nand-stripped.4 of=cramfs-2 bs=1 skip=4604160 count=1953792
And the extraction summary is: Volume size: 0x1dd000] [Volume serial: d07c38cb5ca01614f37d520c4e076ab6] [Volume name: Compressed]
[Summary:] [Total uncompressed size: 4093206] [Total compressed size: -1905520397] [Number of entries: 280] [Number of files compressed: 141] [Number of files expanded: 139] _______________________________________________ e3-hacking mailing list -- e3-hacking@earth.li To unsubscribe send an email to e3-hacking-leave@earth.li
Ok, all OOB bytes are FF which means the blocks should be OK I think. ________________________________ From: David Given <dg@cowlark.com> Sent: 24 December 2023 12:05 To: Discussion of the Amstrad E3 emailer hardware/software <e3-hacking@earth.li> Subject: [E3-hacking] Re: I am trying to uncompress the Amstrad E3 cramfs file system Now, here's a thing I haven't heard for a long time! I wish I hadn't gotten rid of my E3s... I haven't touched flash with OOB directly, but it's possible that some sectors have been marked as unused by data in the OOB itself. e.g. to mark bad sectors in the flash. So you may need to be able to parse this data in order to determine whether the sector is really in use or you could be seeing a corrupt version of the data. Even if the sector is valid, there could also be error-correcting data in the OOB which needs processing in order to recover the correct version of the sector, otherwise there could be bit flips. I have a faint memory that there's a MTD flash emulator for Linux which does this for you, but I've never used it. On Sun, 24 Dec 2023 at 11:17, Tina Poole <span1922@live.com<mailto:span1922@live.com>> wrote: I managed to uncompress its cramfs file system partly, so some of the files are missing data so are all 0s and some files are OK, am I missing something, can't work out why, can anyone here help me, that's if anyone's around....? This is how I did it: About the cramfs and how to get it off the NAND Dump The OOB is 16 bytes after every block of 512 bytes. So for every 528 bytes you need to remove the last 16 bytes. In the E3-hacking email thread there's they Python script, that does it but also a simple command-line perl program that does it as well but modifies the existing file so you better copy it first: # cp e3-nand-backup.4 e3-nand-stripped.4 LC_ALL=C perl -i -0777pe 's/(.{512})(.{16})/$1/gs' e3-nand-stripped.4 So then e3-nand-stripped.4 is the copy of the nand part where the cramfs filesystems are without the OOB bytes. Now, to extract the actual cramfs parts you need to find where they are. From the E3-hacking email thread we know that each section in the nand starts with Q;Q; and also that the filesystem sections also have the word LNXFSYS. So it is easy to search for in hexedit. The first one you will find is at 0x1E4000: 001E4000 51 3B 51 3B 00 00 DB 2E 00 D1 1D 00 01 00 01 00 00 00 02 00 4C 4E 58 46 53 59 53 00 00 00 00 00 Q;Q;................LNXFSYS..... 001E4020 00 00 00 00 40 28 23 29 69 6E 69 74 72 64 2E 63 72 61 6D 20 41 4D 53 3A 30 30 30 31 00 00 00 00 ....@(#)initrd.cram AMS:0001.... 001E4040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................................ 001E4060 00 00 00 00 F8 00 10 11 13 B5 13 B5 00 00 10 11 00 00 00 00 00 00 00 00 00 00 00 00 FF FF FF FF ................................ 001E4080 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40A0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40C0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40E0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 00 00 00 00 00 00 00 00 ................................ 001E4100 45 3D CD 28 00 00 01 00 00 00 00 00 00 00 00 00 43 6F 6D 70 72 65 73 73 65 64 20 52 4F 4D 46 53 E=.(............Compressed ROMFS 001E4120 DC 49 D9 64 DE 0D 9A 00 68 8D F8 40 CF 52 8B 66 43 6F 6D 70 72 65 73 73 65 64 00 00 00 00 00 00 .I.d....h..@.R.fCompressed...... If you look more closely you see 0x1E4000-0x1E40FF is probably the header for this section with some info probably for the nand driver that Amstrad used? But as it is publicly information that a CRAMFS filesystem starts with a signature of 45 3D CD 28 it's easy to figure out where the acutal CRAMFS data starts (0x1E4100). Acording to how a CRAMFS system header is put together (https://www.kernel.org/doc/Documentation/filesystems/cramfs.txt) there some more info to be had: 00 ulelong 0x28cd3d45 CRAMFS signature 04 ulelong 0x00010000 size = 65535 08 ulelong 0x00000000 flags 12 ulelong 0x00000000 future 0x%x 16 string Compressed ROMFS signature 32 ulelong 0x64D949DC checksum 36 ulelong 0x009A0DDE edition 40 ulelong 0x40F88D68 #of blocks = **1090030952** (!!!!!) 44 ulelong 0x668B52CF #of files = **1720406735** (!!!!!) 48 string Compressed name But you already see something is off because the #blocks would be 0x40F88D68 and the number of files 0x668B52CF. Which is much too many, So this is already an indication that there is something odd. The next Q;Q; block starts at 0x464000 (this is the other CRAMFS filesystem, there are 2 cramfs) so the maximum length that the first CRAMFS data could be is 0x464000 - 0x1E4100 = 0x26FF00 = 2555648. This is a lot larger than the size in the header (65535) so also a bit fishy. They probably left a lot of room spare. If you look at the hex data the last bit of 'random' data ends at 0x3C093C, then it is 00's until 0x3C1100 and from there on FF's until 0x464000. But if you look at the start + 65535 it doesn't look like that is the real end of the file, the data seems to continue. So it is best to use the data at least up to where the 00's end. From this I assume that the start of the first CRAMFS system is at 0x1E4100 (6439168) and the end at 0x3C10FF so a length of 0x11DD000 (1953792). To dump this you then do: # dd if=e3-nand-stripped.4 of=cramfs-1 bs=1 skip=1982720 count=1953792 And from there on it is using standard linux tools for cramfs to try to extract it: # cramfsck -help usage: cramfsck [-hv] [-x dir] file -h print this help -x dir extract into dir -v be more verbose file file to test # sudo `mount -t cramfs cramfs-1 /mnt But these won't work because of the data being somehwat different. This one I compiled from source and it does a bit more uncramfs-fmk 57.41 KB # uncramfs-fmk v0.7 by Andrew Stitcher Usage: 'uncramfs-fmk [-d devfilename] [-m modefilename] dirname infile' where <dirname> is the root for the uncompressed (output) filesystem. # uncramfs-fmk e3-cramfs-1-root cramfs-1 This will extract it to a directory e3-cramfs-1-root and also output lot's of info including 'Uncompression failed; for each file wher it fails, Also: [Volume size: 0x1dd000] [Volume serial: dc49d964de0d9a00688df840cf528b66] [Volume name: Compressed] <lots of stuff removed> [Summary:] [Total uncompressed size: 4492954] [Total compressed size: -760580457] [Number of entries: 341] [Number of files compressed: 202] [Number of files expanded: 139] So there are 341 files in total apparently. And the size is 0x1dd000 which is different from 0x10000 in the header... To get the second cramfs you do the same analysis on the next Q;Q block and you arrive at # dd if=e3-nand-stripped.4 of=cramfs-2 bs=1 skip=4604160 count=1953792 And the extraction summary is: Volume size: 0x1dd000] [Volume serial: d07c38cb5ca01614f37d520c4e076ab6] [Volume name: Compressed] [Summary:] [Total uncompressed size: 4093206] [Total compressed size: -1905520397] [Number of entries: 280] [Number of files compressed: 141] [Number of files expanded: 139] _______________________________________________ e3-hacking mailing list -- e3-hacking@earth.li<mailto:e3-hacking@earth.li> To unsubscribe send an email to e3-hacking-leave@earth.li<mailto:e3-hacking-leave@earth.li>
Maybe my backup of the E3's nand is corrupt, I have had it a long time, has anyone got a copy on the E3's nand at all? All files are there in both file systems, but some files are missing data, so if you look at say busybox, it only has part of its data, the file is the right size, but the rest of the file is 0s so is empty... weird. I feel so close yet so far... 🙁 The E3 I copied from has the Linux shell on it now so can not copy from it again(I wish I didn't do that now, now I know more about Linux, I didn't before), I do have another Amstrad E3 but it has PBL 5.1 on it, I do not use it because it activated, I would like to find a way of hacking that PBL 5.1, just do not know where to start, it has its config file on it too, that why I never use it, I do have E1 and E2 Plus too, they are activated too, the E2 has the configuration change in it, the confg change it just the `gamma_e_live.config file telling it to stay activated, I do not see why they wouldn't all use the same file just with alpha, gamma, delta name on the file, all emailers work the same. My idea is to rebuilt the E3 firmware in the Linux Shell, the menu system(the main software) runs from a elf binary file`stnc_sys_boot.elf`, so the E3 works like a Mini Console does, it's the same idea. I added a picture of the Amstrad E3 cramfs file system in this post about it, all stuff I find out I add to this website, all in one place... 🙂https://amstrad-e3-hacking.freeforums.net/thread/15/amstrad-iler-mailer ________________________________ From: David Given <dg@cowlark.com> Sent: 24 December 2023 12:05 To: Discussion of the Amstrad E3 emailer hardware/software <e3-hacking@earth.li> Subject: [E3-hacking] Re: I am trying to uncompress the Amstrad E3 cramfs file system Now, here's a thing I haven't heard for a long time! I wish I hadn't gotten rid of my E3s... I haven't touched flash with OOB directly, but it's possible that some sectors have been marked as unused by data in the OOB itself. e.g. to mark bad sectors in the flash. So you may need to be able to parse this data in order to determine whether the sector is really in use or you could be seeing a corrupt version of the data. Even if the sector is valid, there could also be error-correcting data in the OOB which needs processing in order to recover the correct version of the sector, otherwise there could be bit flips. I have a faint memory that there's a MTD flash emulator for Linux which does this for you, but I've never used it. On Sun, 24 Dec 2023 at 11:17, Tina Poole <span1922@live.com<mailto:span1922@live.com>> wrote: I managed to uncompress its cramfs file system partly, so some of the files are missing data so are all 0s and some files are OK, am I missing something, can't work out why, can anyone here help me, that's if anyone's around....? This is how I did it: About the cramfs and how to get it off the NAND Dump The OOB is 16 bytes after every block of 512 bytes. So for every 528 bytes you need to remove the last 16 bytes. In the E3-hacking email thread there's they Python script, that does it but also a simple command-line perl program that does it as well but modifies the existing file so you better copy it first: # cp e3-nand-backup.4 e3-nand-stripped.4 LC_ALL=C perl -i -0777pe 's/(.{512})(.{16})/$1/gs' e3-nand-stripped.4 So then e3-nand-stripped.4 is the copy of the nand part where the cramfs filesystems are without the OOB bytes. Now, to extract the actual cramfs parts you need to find where they are. From the E3-hacking email thread we know that each section in the nand starts with Q;Q; and also that the filesystem sections also have the word LNXFSYS. So it is easy to search for in hexedit. The first one you will find is at 0x1E4000: 001E4000 51 3B 51 3B 00 00 DB 2E 00 D1 1D 00 01 00 01 00 00 00 02 00 4C 4E 58 46 53 59 53 00 00 00 00 00 Q;Q;................LNXFSYS..... 001E4020 00 00 00 00 40 28 23 29 69 6E 69 74 72 64 2E 63 72 61 6D 20 41 4D 53 3A 30 30 30 31 00 00 00 00 ....@(#)initrd.cram AMS:0001.... 001E4040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................................ 001E4060 00 00 00 00 F8 00 10 11 13 B5 13 B5 00 00 10 11 00 00 00 00 00 00 00 00 00 00 00 00 FF FF FF FF ................................ 001E4080 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40A0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40C0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40E0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 00 00 00 00 00 00 00 00 ................................ 001E4100 45 3D CD 28 00 00 01 00 00 00 00 00 00 00 00 00 43 6F 6D 70 72 65 73 73 65 64 20 52 4F 4D 46 53 E=.(............Compressed ROMFS 001E4120 DC 49 D9 64 DE 0D 9A 00 68 8D F8 40 CF 52 8B 66 43 6F 6D 70 72 65 73 73 65 64 00 00 00 00 00 00 .I.d....h..@.R.fCompressed...... If you look more closely you see 0x1E4000-0x1E40FF is probably the header for this section with some info probably for the nand driver that Amstrad used? But as it is publicly information that a CRAMFS filesystem starts with a signature of 45 3D CD 28 it's easy to figure out where the acutal CRAMFS data starts (0x1E4100). Acording to how a CRAMFS system header is put together (https://www.kernel.org/doc/Documentation/filesystems/cramfs.txt) there some more info to be had: 00 ulelong 0x28cd3d45 CRAMFS signature 04 ulelong 0x00010000 size = 65535 08 ulelong 0x00000000 flags 12 ulelong 0x00000000 future 0x%x 16 string Compressed ROMFS signature 32 ulelong 0x64D949DC checksum 36 ulelong 0x009A0DDE edition 40 ulelong 0x40F88D68 #of blocks = **1090030952** (!!!!!) 44 ulelong 0x668B52CF #of files = **1720406735** (!!!!!) 48 string Compressed name But you already see something is off because the #blocks would be 0x40F88D68 and the number of files 0x668B52CF. Which is much too many, So this is already an indication that there is something odd. The next Q;Q; block starts at 0x464000 (this is the other CRAMFS filesystem, there are 2 cramfs) so the maximum length that the first CRAMFS data could be is 0x464000 - 0x1E4100 = 0x26FF00 = 2555648. This is a lot larger than the size in the header (65535) so also a bit fishy. They probably left a lot of room spare. If you look at the hex data the last bit of 'random' data ends at 0x3C093C, then it is 00's until 0x3C1100 and from there on FF's until 0x464000. But if you look at the start + 65535 it doesn't look like that is the real end of the file, the data seems to continue. So it is best to use the data at least up to where the 00's end. From this I assume that the start of the first CRAMFS system is at 0x1E4100 (6439168) and the end at 0x3C10FF so a length of 0x11DD000 (1953792). To dump this you then do: # dd if=e3-nand-stripped.4 of=cramfs-1 bs=1 skip=1982720 count=1953792 And from there on it is using standard linux tools for cramfs to try to extract it: # cramfsck -help usage: cramfsck [-hv] [-x dir] file -h print this help -x dir extract into dir -v be more verbose file file to test # sudo `mount -t cramfs cramfs-1 /mnt But these won't work because of the data being somehwat different. This one I compiled from source and it does a bit more uncramfs-fmk 57.41 KB # uncramfs-fmk v0.7 by Andrew Stitcher Usage: 'uncramfs-fmk [-d devfilename] [-m modefilename] dirname infile' where <dirname> is the root for the uncompressed (output) filesystem. # uncramfs-fmk e3-cramfs-1-root cramfs-1 This will extract it to a directory e3-cramfs-1-root and also output lot's of info including 'Uncompression failed; for each file wher it fails, Also: [Volume size: 0x1dd000] [Volume serial: dc49d964de0d9a00688df840cf528b66] [Volume name: Compressed] <lots of stuff removed> [Summary:] [Total uncompressed size: 4492954] [Total compressed size: -760580457] [Number of entries: 341] [Number of files compressed: 202] [Number of files expanded: 139] So there are 341 files in total apparently. And the size is 0x1dd000 which is different from 0x10000 in the header... To get the second cramfs you do the same analysis on the next Q;Q block and you arrive at # dd if=e3-nand-stripped.4 of=cramfs-2 bs=1 skip=4604160 count=1953792 And the extraction summary is: Volume size: 0x1dd000] [Volume serial: d07c38cb5ca01614f37d520c4e076ab6] [Volume name: Compressed] [Summary:] [Total uncompressed size: 4093206] [Total compressed size: -1905520397] [Number of entries: 280] [Number of files compressed: 141] [Number of files expanded: 139] _______________________________________________ e3-hacking mailing list -- e3-hacking@earth.li<mailto:e3-hacking@earth.li> To unsubscribe send an email to e3-hacking-leave@earth.li<mailto:e3-hacking-leave@earth.li>
I do actually have the opportunity to pick up an E3. I deeply regret getting rid of my old ones (I had all the models!) and am wondering whether to get it... except I don't know whether it has PBL 5.1 on it. I do know more about soldering now and *should* be able to get work around that --- didn't someone find a JTAG port? --- but, hmm... is there any way to know ahead of time whether it's a PBL 5.1 device? It's NOS in-the-box. They're *so* iconic I feel like I should get it even if it's just for a display piece... On Sun, 24 Dec 2023 at 16:44, Tina Poole <span1922@live.com> wrote:
Maybe my backup of the E3's nand is corrupt, I have had it a long time, has anyone got a copy on the E3's nand at all? All files are there in both file systems, but some files are missing data, so if you look at say busybox, it only has part of its data, the file is the right size, but the rest of the file is 0s so is empty... weird. I feel so close yet so far... 🙁 The E3 I copied from has the Linux shell on it now so can not copy from it again(I wish I didn't do that now, now I know more about Linux, I didn't before), I do have another Amstrad E3 but it has PBL 5.1 on it, I do not use it because it activated, I would like to find a way of hacking that PBL 5.1, just do not know where to start, it has its config file on it too, that why I never use it, I do have E1 and E2 Plus too, they are activated too, the E2 has the configuration change in it, the confg change it just the `gamma_e_live.config file telling it to stay activated, I do not see why they wouldn't all use the same file just with alpha, gamma, delta name on the file, all emailers work the same. My idea is to rebuilt the E3 firmware in the Linux Shell, the menu system(the main software) runs from a elf binary file`stnc_sys_boot.elf`, so the E3 works like a Mini Console does, it's the same idea. I added a picture of the Amstrad E3 cramfs file system in this post about it, all stuff I find out I add to this website, all in one place... 🙂 https://amstrad-e3-hacking.freeforums.net/thread/15/amstrad-iler-mailer ------------------------------ *From:* David Given <dg@cowlark.com> *Sent:* 24 December 2023 12:05 *To:* Discussion of the Amstrad E3 emailer hardware/software < e3-hacking@earth.li> *Subject:* [E3-hacking] Re: I am trying to uncompress the Amstrad E3 cramfs file system
Now, here's a thing I haven't heard for a long time! I wish I hadn't gotten rid of my E3s...
I haven't touched flash with OOB directly, but it's possible that some sectors have been marked as unused by data in the OOB itself. e.g. to mark bad sectors in the flash. So you may need to be able to parse this data in order to determine whether the sector is really in use or you could be seeing a corrupt version of the data. Even if the sector is valid, there could also be error-correcting data in the OOB which needs processing in order to recover the correct version of the sector, otherwise there could be bit flips.
I have a faint memory that there's a MTD flash emulator for Linux which does this for you, but I've never used it.
On Sun, 24 Dec 2023 at 11:17, Tina Poole <span1922@live.com> wrote:
I managed to uncompress its cramfs file system partly, so some of the files are missing data so are all 0s and some files are OK, am I missing something, can't work out why, can anyone here help me, that's if anyone's around....?
This is how I did it: About the cramfs and how to get it off the NAND Dump
The OOB is 16 bytes after every block of 512 bytes. So for every 528 bytes you need to remove the last 16 bytes. In the E3-hacking email thread there's they Python script, that does it but also a simple command-line perl program that does it as well but modifies the existing file so you better copy it first: # cp e3-nand-backup.4 e3-nand-stripped.4
LC_ALL=C perl -i -0777pe 's/(.{512})(.{16})/$1/gs' e3-nand-stripped.4
So then e3-nand-stripped.4 is the copy of the nand part where the cramfs filesystems are without the OOB bytes.
Now, to extract the actual cramfs parts you need to find where they are. From the E3-hacking email thread we know that each section in the nand starts with Q;Q; and also that the filesystem sections also have the word LNXFSYS. So it is easy to search for in hexedit. The first one you will find is at 0x1E4000: 001E4000 51 3B 51 3B 00 00 DB 2E 00 D1 1D 00 01 00 01 00 00 00 02 00 4C 4E 58 46 53 59 53 00 00 00 00 00 Q;Q;................LNXFSYS..... 001E4020 00 00 00 00 40 28 23 29 69 6E 69 74 72 64 2E 63 72 61 6D 20 41 4D 53 3A 30 30 30 31 00 00 00 00 ....@(#)initrd.cram AMS:0001.... 001E4040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................................ 001E4060 00 00 00 00 F8 00 10 11 13 B5 13 B5 00 00 10 11 00 00 00 00 00 00 00 00 00 00 00 00 FF FF FF FF ................................ 001E4080 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40A0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40C0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................................ 001E40E0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 00 00 00 00 00 00 00 00 ................................ 001E4100 45 3D CD 28 00 00 01 00 00 00 00 00 00 00 00 00 43 6F 6D 70 72 65 73 73 65 64 20 52 4F 4D 46 53 E=.(............Compressed ROMFS 001E4120 DC 49 D9 64 DE 0D 9A 00 68 8D F8 40 CF 52 8B 66 43 6F 6D 70 72 65 73 73 65 64 00 00 00 00 00 00 .I.d....h..@.R.fCompressed......
If you look more closely you see 0x1E4000-0x1E40FF is probably the header for this section with some info probably for the nand driver that Amstrad used? But as it is publicly information that a CRAMFS filesystem starts with a signature of 45 3D CD 28 it's easy to figure out where the acutal CRAMFS data starts (0x1E4100). Acording to how a CRAMFS system header is put together ( https://www.kernel.org/doc/Documentation/filesystems/cramfs.txt) there some more info to be had: 00 ulelong 0x28cd3d45 CRAMFS signature 04 ulelong 0x00010000 size = 65535 08 ulelong 0x00000000 flags 12 ulelong 0x00000000 future 0x%x 16 string Compressed ROMFS signature 32 ulelong 0x64D949DC checksum 36 ulelong 0x009A0DDE edition 40 ulelong 0x40F88D68 #of blocks = **1090030952** (!!!!!) 44 ulelong 0x668B52CF #of files = **1720406735** (!!!!!) 48 string Compressed name
But you already see something is off because the #blocks would be 0x40F88D68 and the number of files 0x668B52CF. Which is much too many, So this is already an indication that there is something odd.
The next Q;Q; block starts at 0x464000 (this is the other CRAMFS filesystem, there are 2 cramfs) so the maximum length that the first CRAMFS data could be is 0x464000 - 0x1E4100 = 0x26FF00 = 2555648. This is a lot larger than the size in the header (65535) so also a bit fishy. They probably left a lot of room spare. If you look at the hex data the last bit of 'random' data ends at 0x3C093C, then it is 00's until 0x3C1100 and from there on FF's until 0x464000. But if you look at the start + 65535 it doesn't look like that is the real end of the file, the data seems to continue. So it is best to use the data at least up to where the 00's end. From this I assume that the start of the first CRAMFS system is at 0x1E4100 (6439168) and the end at 0x3C10FF so a length of 0x11DD000 (1953792). To dump this you then do: # dd if=e3-nand-stripped.4 of=cramfs-1 bs=1 skip=1982720 count=1953792
And from there on it is using standard linux tools for cramfs to try to extract it: # cramfsck -help usage: cramfsck [-hv] [-x dir] file -h print this help -x dir extract into dir -v be more verbose file file to test
# sudo `mount -t cramfs cramfs-1 /mnt
But these won't work because of the data being somehwat different.
This one I compiled from source and it does a bit more
uncramfs-fmk 57.41 KB
# uncramfs-fmk v0.7 by Andrew Stitcher Usage: 'uncramfs-fmk [-d devfilename] [-m modefilename] dirname infile' where <dirname> is the root for the uncompressed (output) filesystem.
# uncramfs-fmk e3-cramfs-1-root cramfs-1
This will extract it to a directory e3-cramfs-1-root and also output lot's of info including 'Uncompression failed; for each file wher it fails, Also: [Volume size: 0x1dd000] [Volume serial: dc49d964de0d9a00688df840cf528b66] [Volume name: Compressed]
<lots of stuff removed>
[Summary:] [Total uncompressed size: 4492954] [Total compressed size: -760580457] [Number of entries: 341] [Number of files compressed: 202] [Number of files expanded: 139]
So there are 341 files in total apparently.
And the size is 0x1dd000 which is different from 0x10000 in the header...
To get the second cramfs you do the same analysis on the next Q;Q block and you arrive at # dd if=e3-nand-stripped.4 of=cramfs-2 bs=1 skip=4604160 count=1953792
And the extraction summary is: Volume size: 0x1dd000] [Volume serial: d07c38cb5ca01614f37d520c4e076ab6] [Volume name: Compressed]
[Summary:] [Total uncompressed size: 4093206] [Total compressed size: -1905520397] [Number of entries: 280] [Number of files compressed: 141] [Number of files expanded: 139] _______________________________________________ e3-hacking mailing list -- e3-hacking@earth.li To unsubscribe send an email to e3-hacking-leave@earth.li
_______________________________________________ e3-hacking mailing list -- e3-hacking@earth.li To unsubscribe send an email to e3-hacking-leave@earth.li
participants (3)
-
Antony Stone -
David Given -
Tina Poole