Message: 2 Date: Tue, 17 Sep 2002 08:51:36 +0100 To: Ian Bell <ruffrecords@uklinux.net> Cc: main@lists.alug.org.uk Subject: Re: [Alug] Hard Drive performance From: Adam Bower <abower@thebowery.co.uk> On Mon, Sep 16, 2002 at 10:44:33PM +0100, Ian Bell wrote:
I mentioned in an earlier post my move from a P!! igig PC to a new dell laptop with a P4 1.6 gig and how with rh7.3 it seemed slower than the old system. I have now investigated this a little further.
< Snippage - Topic is DMA settings To use, or not to use hdparm... >
Oh and finally hdparm can destroy data on your harddisk if you go for too many settings and the disk/computer doesn't like them....
Too right! ( For details, and a little PC history read on... ) Message: 3 Subject: Re: [Alug] Hard Drive performance From: Douglas Willis <ddw@bas.ac.uk> To: Alug Mailing List <main@lists.alug.org.uk> Date: 17 Sep 2002 09:24:48 +0100
On some of the earlier laptops turning on DMA caused the cdrom drives to lockup requiring a hare reset of the machine. I this was due to slightly non standard IDE bus architecture.
I run an IBM A21M which has a PIII ant about 800MHz and now have no problems with disk access once I turned on the DMA settings.
The RedHat install by default disabled the DMA .
This is considered ( quite rightly ) to be the safest default option that will operate properly across all PC architecture hardware. The old timers may remember the war that IBM fought to keep their PC hardware and firmware (BIOS) proprietary - forcing competitors making clone machines to do "clean room" BIOs coding and design their own hardware, too. Gaining a cost and time to market advantage for IBM for quite a while. It guaranteed a lot of flaky hardware designs, too. ( Remember the Amstrad 286 HD fiasco? ) Compaq were established in this era; and never quit designing their own hardware. Even today, when it is no longer necessary, or desirable, to deviate too far from accepted "industry standards". They were beginning to get wise to the fact that it is no longer desirable! Two decades later... Only to be swallowed by HP. Go figure. It is cheaper to use "industry standard" chipsets from a third party than to re-invent complex hardware, in the design process. Some big corporations just failed to adapt, and tried to lock in customers to their particular hardware, when the game had moved on to proprietary lock-ins via software: EULAs and proprietary protocols "embracing and extending" open standards. I can pretty much guarantee data loss ( or total operating system loss ) on *any* Compaq where you attempt to force an OS to use DMA when it is not selected by the default install. Quite a few laptop clones are like the Compaq ( I had a Twinheed one just a few weeks ago ). Desktop PCs can also be found with iffy DMA hardware design. Beware the data corruption bug when shifting large quantities of data between IDE ports on some chipsets, too. ( One range of SiS chipsets suffered from this AFAIR ) Although IDE is a blocking type device, the DMA behaviour can also vary from a single device per IDE port to multiple devices daisy-chained on one IDE port. Had I decided to use a cheaper IDE CD Burner on my SiS chipset Desktop ( 2001 Motherboard design ) I'd have walked right into the data corruption problem! ( I prefer to have at least two separate hardware disk channels, one of them SCSI, for best VM performance, anyway. ). TESTING: If you multiboot with Windows 9x ( perhaps to use Office 97 from Wine ) it is easy to test. Make sure that the Windows boot comes to a real DOS prompt, so you have to start Windows with WIN at the prompt ( edit MSDOS.SYS ). Make COPIES of the registry SYSTEM.D* and USER.D* files! Then - in windows, play with the DMA enables for the IDE Harddrive and CDROM. When windows fails to load you have an answer... ( Restore the registry files, and it should be OK ). I haven't known this to corrupt data on the drive, but I'm not making any promises. The worst that usually happens is that it fails to load the "DOS extender" <*grin*> Try this with Linux, and you will probably trash the filesystem. A disposable Windows test multiboot can be a less shattering test option to determine if your machine has well behaved DMA hardware! But there are no guarantees if you start tweaking hardware related stuff. ( Overclocking is another example ). Eventually, you will reach a point where the probability of a timing error will corrupt data, as timing margins are reduced. Consider that slower disk performance means a higher safety margin against data corruption. The size of IDE hard drives now is such that there is a reasonable statistical likelihood of hitting random data corruption, anyway. -- James To reply by private email DONT delete the "nospam" in Reply To or From header addresses!
On Wednesday 18 September 2002 09:55, James - listmail wrote:
The old timers may remember the war that IBM fought to keep their PC hardware and firmware (BIOS) proprietary - forcing competitors making clone machines to do "clean room" BIOs coding and design their own hardware, too. Gaining a cost and time to market advantage for IBM for quite a while. It guaranteed a lot of flaky hardware designs, too. ( Remember the Amstrad 286 HD fiasco? )
That's not exactly how I remember it :o) IBM were quite liberal (for IBM) with information regarding the internals of the PC, Anybody could get BIOS Source code,Bus specs etc. The only reason Compaq got someone to do a clean room re-implementation of it was that they had to, in order to evade any copyright issues regarding the original Bios code...Compaq actually had to track down engineers who had never seen the freely available IBM BOIS code just to make sure that none of them accidently "copied" part of it. Releasing the Bios was the single biggest leap in the widespread adoption of the PC, it enabled 3rd party developers (software and hardware) to fast-track the development of their own PC compatable products (in some cases even before the hardware was even available) The rest of the hardware in the original PC was entirely off the shelf, the Bios was the only piece of custom hardware (and even then only because of the firmware it contained) The reason for the shoddy hardware designs was because the early clone makers had a big problem....they couldn't match IBM's buying power (IBM were bloody massive back then) so their raw component costs were higher, IBM had already figured this and that's why they were never bothered about the clone manufacturers. What they didn't count on was that someone would figure out how to reduce the component count by having a smaller number of non "off the shelf" IC's custom produced (again I think Compaq did this first) hence the chipset was born. This hurt IBM, but not as much as when internal politics (with the Mainframe department) caused them to drag their feet on the i386, that meant that Microsoft stopped loving them and Compaq were actually first to market with a 386 based machine. Anyway back to the shoddy hardware, with so many people developing custom chipsets, mistakes were often made (and still are, thanks Via) The Amstrad was dodgy because it was so severely cut down to a price (remember the PC1512/1640 with the PC's power supply in the monitor, how very clever...I remember seeing people with brand new colour cards and monitors, and the Amstrad monitor under the desk powering the PC) Diversity is both the PC's greatest gift and it's Achilles heal I'm surprised that you've had so much trouble with Compaq's and DMA, in my experience Compaq have offered some of the most Linux compatable big name machines I have seen. Regards Wayne
On Wed, 18 Sep 2002 21:54:26 +0000 Wayne Stallwood <wayne.stallwood@btinternet.com> wrote:
That's not exactly how I remember it :o)
IBM were quite liberal (for IBM) with information regarding the internals of the PC, Anybody could get BIOS Source code,Bus specs etc. The only reason Compaq got someone to do a clean room re-implementation of it was that they had to, in order to evade any copyright issues regarding the original Bios code...Compaq actually had to track down engineers who had never seen the freely available IBM BOIS code just to make sure that none of them accidently "copied" part of it.
I am not quite sure what you mean here about copyright issues. Possible reasons I can see for compaq wanting an implementation not copyrighted by IBM are: 1. Compaq didn't trust IBM to keep the same license terms and so wanted to avoid being held to ransome in the future. 2. Compaq had plans for the BIOS which the IBM license didn't allow. 3. Compaq though they could do a better job than IBM and when their BIOS was finished they would have something other people would pay for.
Diversity is both the PC's greatest gift and it's Achilles heal
That could be said of Unix and Free Software too.
I'm surprised that you've had so much trouble with Compaq's and DMA, in my experience Compaq have offered some of the most Linux compatable big name machines I have seen.
Well, except for some Compaq specific graphics cards which appear to use a well known graphics chip but differ in a subtle way such that XFree86 doesn't know how to drive them correctly. HP have also done something similar with one of their Vectras which has a Matrox MGA 200 chip but fails to work at the highest dot clocks the MGA 200 will support.
Not to mention the scsi cards and raid controllers in the early Prolient range that used a not quite standard future domain controller chip set. On Wed, 2002-09-18 at 22:57, Steve Fosdick wrote:
On Wed, 18 Sep 2002 21:54:26 +0000 Wayne Stallwood <wayne.stallwood@btinternet.com> wrote:
Well, except for some Compaq specific graphics cards which appear to use a well known graphics chip but differ in a subtle way such that XFree86 doesn't know how to drive them correctly.
-- Douglas Willis (ddw@nerc-bas.ac.uk) British Antarctic Survey High Cross, Madingley Road Cambridge, CB3 0ET, United Kingdom tel: +44 1223 221400, fax: +44 1223 362616
I have made a clean install of RH7.3 on the same system. Base performance is identical. Removing unnecessary daemons and upgrading kernel improves boot time by a few seconds and eliminates excessive memory usage (caused by a cron job) but makes no difference to program loading times and base disk asccess speed of 3MB/sec. However hdparn -d1 /dev/hda1 now does not work and returns an error of HDIO_SET_DMA failed. Operation not permitted. Man is silent on this. Anyone know what it means? ian
On Thu, 19 Sep 2002 23:02:31 +0100 Ian Bell <ian@redtommo.com> wrote:
I have made a clean install of RH7.3 on the same system. Base performance is identical. Removing unnecessary daemons and upgrading kernel improves boot time by a few seconds and eliminates excessive memory usage (caused by a cron job) but makes no difference to program loading times and base disk asccess speed of 3MB/sec. However hdparn -d1 /dev/hda1 now does not work and returns an error of HDIO_SET_DMA failed. Operation not permitted. Man is silent on this. Anyone know what it means?
The error message "Operation not permitted" corresponds to an error code for which the C macro EPERM is defined, so I grepped the ide driver for EPERM and found the following function which looks relevant: static int set_using_dma (ide_drive_t *drive, int arg) { if (!drive->driver || !DRIVER(drive)->supports_dma) return -EPERM; if (!drive->id || !(drive->id->capability & 1) || !HWIF(drive)->dmaproc) return -EPERM; if (HWIF(drive)->dmaproc(arg ? ide_dma_on : ide_dma_off, drive)) return -EIO; return 0; } Looking at that it would appear that reason is any of the following: * The drive has no driver assciated with it. * The driver associated with this drive doesn't know how to handle DMA. * The drive capabilty information is missing. * The drive appears not to support DMA. * The IDE interface doesn't have method to set DMA. The most likely of these if probably that the drive capability info says the drive doesn't support DMA so the next thing would be to find out where the IDE driver gets that from, whether it is from the drive itself or from the BIOS. Steve.
On Thu, 19 Sep 2002 23:42:05 +0100 Steve Fosdick <fozzy@pelvoux.demon.co.uk> wrote: ....
The most likely of these if probably that the drive capability info says the drive doesn't support DMA so the next thing would be to find out where the IDE driver gets that from, whether it is from the drive itself or from the BIOS.
Further to my last message, I have found what appear to be the rules for whether Linux uses DMA automatically (i.e. without hdparm setting it on). 1. If the drive model number is on the bad list then don't use DMA and don't check any further. Drives on this list as of Kernel 2.4.19 are: WDC AC11000H WDC AC22100H WDC AC32500H WDC AC33100H WDC AC31600H WDC AC32100H WDC AC23200L Compaq CRD-8241B CRD-8400B CRD-8480B CRD-8480C CRD-8482B CRD-84 SanDisk SDP3B SanDisk SDP3B-64 SANYO CD-ROM CRD HITACHI CDR-8 HITACHI CDR-8335 HITACHI CDR-8435 Toshiba CD-ROM XM-6202B CD-532E-A E-IDE CD-ROM CR-840 CD-ROM Drive/F5A RICOH CD-R/RW MP7083A WPI CDD-820 SAMSUNG CD-ROM SC-148C SAMSUNG CD-ROM SC-148F SAMSUNG CD-ROM SC SanDisk SDP3B-64 SAMSUNG CD-ROM SN-124 PLEXTOR CD-R PX-W8432T ATAPI CD-ROM DRIVE 40X MAXIMUM _NEC DV5800A 2. If the BIOS has enabled DMA on the drive then leave it on and set Linux to use it. 3. If the drive is a known good list then turn DMA on. Drives on the good list are: Micropolis 2112A CONNER CTMA 4000 CONNER CTT8000-A ST34342A Steve.
Many thank to all, and especially Steve, who responded on this topic. it turns out to be a problem with certain chipsets and the 2.4.18 kernel which will fixed in 2.4.20. Ian
Maybe that's true, I am not completely sure about all of Compaq's reasons, all I know is that at the time Compaq had the Bios Reverse Engineered the original Bios source was available to read. Was it that back then program source was treated like literature, in that you could write a book and copyright it, yet I can write a book that tells essentially the same story as long as I do not use the same passages ? Compaq did have plans for the Bois beyond that defined by IBM, I think they implemented the Bios setup menu before IBM, plus the move to a "Chipset" required changes to the original Bios anyway.
Well, except for some Compaq specific graphics cards which appear to use a well known graphics chip but differ in a subtle way such that XFree86 doesn't know how to drive them correctly.
Never experienced that myself, if you are referring to the older Deskpro series with the intergrated Video hardware, then that is quite common (changes are frequently made when usually external chipsets are intergrated into the system board. Certainly my later Compaq workstation (Which came with a GF3) has a Compaq branded card that appears to work fine with the standard NVidia drivers. As to how earlier (non NVidia) cards behaved I don't know, personally I have never found a NVidia based card that will not work with the detonator drivers. W
HP have also done something similar with one of their Vectras which has a Matrox MGA 200 chip but fails to work at the highest dot clocks the MGA 200 will support.
On Wednesday 18 September 2002 21:57, Steve Fosdick wrote:
On Wed, 18 Sep 2002 21:54:26 +0000
Wayne Stallwood <wayne.stallwood@btinternet.com> wrote:
That's not exactly how I remember it :o)
IBM were quite liberal (for IBM) with information regarding the internals of the PC, Anybody could get BIOS Source code,Bus specs etc. The only reason Compaq got someone to do a clean room re-implementation of it was that they had to, in order to evade any copyright issues regarding the original Bios code...Compaq actually had to track down engineers who had never seen the freely available IBM BOIS code just to make sure that none of them accidently "copied" part of it.
I am not quite sure what you mean here about copyright issues. Possible reasons I can see for compaq wanting an implementation not copyrighted by IBM are:
1. Compaq didn't trust IBM to keep the same license terms and so wanted to avoid being held to ransome in the future.
2. Compaq had plans for the BIOS which the IBM license didn't allow.
3. Compaq though they could do a better job than IBM and when their BIOS was finished they would have something other people would pay for.
Diversity is both the PC's greatest gift and it's Achilles heal
That could be said of Unix and Free Software too.
I'm surprised that you've had so much trouble with Compaq's and DMA, in my experience Compaq have offered some of the most Linux compatable big name machines I have seen.
Well, except for some Compaq specific graphics cards which appear to use a well known graphics chip but differ in a subtle way such that XFree86 doesn't know how to drive them correctly.
HP have also done something similar with one of their Vectras which has a Matrox MGA 200 chip but fails to work at the highest dot clocks the MGA 200 will support.
_______________________________________________ main@lists.alug.org.uk http://www.alug.org.uk/ http://lists.alug.org.uk/mailman/listinfo/main Unsubscribe? See message headers or the web site above!
participants (5)
-
Douglas Willis -
Ian Bell -
James - listmail -
Steve Fosdick -
Wayne Stallwood