From erich@uruk.org Tue Apr 4 12:12:02 1995 id MAA15974; Tue, 4 Apr 1995 12:11:58 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: baford@schirf.cs.utah.edu cc: multiboot@schirf.cs.utah.edu Subject: MultiBoot Standard Date: Tue, 04 Apr 1995 10:47:30 -0700 Sender: erich@uruk.org Hey, Bryan, is there anyone else on the list yet? I have some comments on the "MultiBoot Standard" proposal. (clips are from the version 0.1 document grabbed from "kahlua.cs.utah.edu:/private/boot") > It would be nice if commercial operating system vendors eventually adopted > this standard as well, but that's probably a pipe dream. I think it is for some. Vendors such as Microsoft want total control in the first place. Others such as OSF might be willing to use it. > It _must_ be contained completely within the first > 8192 bytes of the file, and must be longword (32-bit) aligned. Hmmm.... not having a well-defined location for the multiboot header means it won't always work when not at the beginning, but such is life I guess. I presume this lack of a requirement for it being in the beginning is to be compatible with other boot-loaders? > The layout > of the header is as follows: > > +-------------------+ > 0 | magic: 0x1BADB002 | (required) > 4 | flags | (required) > +-------------------+ > 8 | header_addr | (present if flags[16] is set) > 12 | load_addr | (present if flags[16] is set) > 16 | load_end_addr | (present if flags[16] is set) > 20 | bss_end_addr | (present if flags[16] is set) > 24 | entry_addr | (present if flags[16] is set) > +-------------------+ > > All fields are in little-endian byte order, of course. The first field is > the magic number identifying the header, which must be the hex value > 0x1BADB002. "BAD BOOZ" ? Pretty funny. My only addition here would be a header version number of some sort right after the magic number. This would allow incompatible changes in the future to be supported for as long as you like, and to at least be detected. > If bit 16 in the flags word is set, then the fields at offsets 8-24 in the > multiboot_header are valid, and the boot loader should use them instead of > the fields in the actual executable header to calculate where to load the > OS image. This bit should generally be set in OS images in a.out format; > boot loaders will generally refuse to load a.out images that don't have > this bit set. So, is the presumption that the executable header is in its normal location in the file? Or is there some offset induced by the presence of the MultiBoot header? > Machine State > ~~~~~~~~~~~~~ Looks great! > Boot Information Format > ~~~~~~~~~~~~~~~~~~~~~~~ Also looks good, but the padding that is on the end of each structure seems unnecessary, and better handled by a "MultiBoot Info" version number. > ISSUES > ~~~~~~ Adding the two version numbers, one for the header, and one for the info structure, is a good idea, I think... that way you don't need lots of padding or guessing about what might be wrong in the future anyway. Below, I'm responding to those I haven't done in the main document. > Should the boot loader zero the kernel's bss segment? It might be a good "assumption test" to not do it, but why provoke possible problems? I'd prefer the consistency of always zeroing it, but if others object, I'd be willing to say "zero it if possible, but don't assume it is zeroed". (The only case I can think of where you couldn't zero the bss is when it overlaps the loader itself and you don't want to erase your loader while setting up the kernel ;-) A cleaner solution might be for the loader to simply not place anything in this area, and require that kernels who need it zeroed do it first thing. > Should the boot loader pass additional information > that the OS typically wants to ask the BIOS for, > but is often too stupid to switch into 16-bit mode to do it? > * Screen mode > * Hard disk parameter tables This is relatively easy to do for a boot-block. The question in my mind is how easy is this to implement from other OS's... DOS is easy, but what others might there be? > Is the 'avail' list in the multiboot_info unnecessary, since the OS could > easily ask the BIOS for the available memory itself, or probe for memory? > > Similarly, is the 'occupied' list unnecessary, since the OS could derive it > from the other information in the multiboot_info structure? I think having the 'avail' list is a good idea (that way the OS doesn't need to muck with that BIOS call, which it would only do once anyway). The 'occupied' list seems unnecessary, since it is easy to derive, and it is in much the same format in the boot-module list anyway, except for the kernel location itself. I'd say just replace the "occupied.count" and "occupied.regions" with a "kernel.start" and "kernel.end", then you have everything you need. > Should command lines be passed as a list of null terminated strings, > more like Unix command lines, instead of just one string? I don't see how this really matters that much. What's easier for the kernel to use... offhand I'd say a list of null-terminated strings is easier since people are used to parsing them as options. Let's just make a decision and stick to it. Sorry for the long-winded response... Erich -- Erich Stefan Boleyn \___ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \____ home #: +1 (503) 226-0741 WWW Site | USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From erich@uruk.org Tue Apr 4 12:33:04 1995 id MAA16033; Tue, 4 Apr 1995 12:33:01 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: baford@schirf.cs.utah.edu cc: multiboot@schirf.cs.utah.edu, mach4-users@schirf.cs.utah.edu, davidg@implode.root.com Subject: FlexBoot Date: Tue, 04 Apr 1995 11:08:28 -0700 Sender: erich@uruk.org I'm writing a flexible boot-loader (standalone only at the moment). So far the MBR/stage 1 loader, called "FlexMBR", is complete, and I'm working on the second stage loader, called "FlexBoot", which will be compliant with the MultiBoot standard that Bryan Ford is working on. (Utah's Mach4 will also comply to this standard, for those interested, it is a multi-module loading standard) The stage 1 is independent of the stage 2, so could be used for other projects if one chose to do so. It allows loading in *very* large multi-blocklist hunks of code into specified (possibly non-contiguous) areas of memory (still real-mode i386, though, so under 512K is best, under 1MB is required), and jumping into a specified address after completing the loads. This all fits into the normal 1 sector (512 byte) size limits. The stage2 will be a full 32-bit flat-memory model with BIOS call support built in, multiple filesystem support, ability to read multiple modules from different devices/partitions/filesystems for the same boot process, and a user-interface that both supports a host of easy-to-use presets and an expressive mechanism to specify (or re-specify for the presets) any options or modules that are desired. An alpha version will be available reasonably soon, I'll keep this list posted. (good documentation is a strong goal here) Erich Boleyn -- Erich Stefan Boleyn \___ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \____ home #: +1 (503) 226-0741 WWW Site | USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From baford Tue Apr 4 19:41:56 1995 id TAA17120; Tue, 4 Apr 1995 19:41:55 -0600 To: erich@uruk.org cc: multiboot@schirf Subject: Re: MultiBoot Standard In-reply-to: Your message of "Tue, 04 Apr 95 10:47:30 PDT." <199504041747.KAA06826@uruk.org> Date: Tue, 04 Apr 95 19:41:55 MDT From: Bryan Ford >Hey, Bryan, is there anyone else on the list yet? Yes, a few people. I'm sure they'll announce themselves when they're ready. :-) >> It _must_ be contained completely within the first >> 8192 bytes of the file, and must be longword (32-bit) aligned. > >Hmmm.... not having a well-defined location for the multiboot header >means it won't always work when not at the beginning, but such is >life I guess. I presume this lack of a requirement for it being in the >beginning is to be compatible with other boot-loaders? Partly that - this way, new MultiBoot-compatible boot images can remain compatible with old 32-bit boot loaders by having an alternate entrypoint at the true beginning of the text segment. But more importantly, it allows the boot loader to _find_ the text segment without knowing the conventions of the a.out variant in use. For example, in Linux ZMAGIC format, the text segment starts 1024 bytes into the file, whereas in traditional BSD/Mach ZMAGIC format, the text segment starts at the very beginning like in QMAGIC. There's no really straightforward way to distinguish these formats, and the boot loader really shouldn't have to know a zillion a.out variants anyway. Finally, it allows MultiBoot-compatible OS boot images to be created without even knowing which a.out variant the boot image will become. For example, if you build Mach using GNU binutils configured for i386-gnu, the resulting kernel image will be in Mach ZMAGIC format; whereas if you build it with GNU binutils configured for i386-linux, the resulting image will be in Linux ZMAGIC format (or possibly QMAGIC). Depending on the format, the MultiBoot header will wind up in different places in the output file. >> The layout >> of the header is as follows: >> >> +-------------------+ >> 0 | magic: 0x1BADB002 | (required) >> 4 | flags | (required) >> +-------------------+ >> 8 | header_addr | (present if flags[16] is set) >> 12 | load_addr | (present if flags[16] is set) >> 16 | load_end_addr | (present if flags[16] is set) >> 20 | bss_end_addr | (present if flags[16] is set) >> 24 | entry_addr | (present if flags[16] is set) >> +-------------------+ >> >> All fields are in little-endian byte order, of course. The first field is >> the magic number identifying the header, which must be the hex value >> 0x1BADB002. > >"BAD BOOZ" ? Pretty funny. Actually, I had in mind "1 BAD BOOT" (a "2" is as close as I could find to something that sounds like a "T"); but I guess BOOZ is not bad either. :-) >My only addition here would be a header version number of some sort right >after the magic number. This would allow incompatible changes in the future >to be supported for as long as you like, and to at least be detected. That's what the flags longword is for. The lower 16 bits act like a "major version" in a sense, in that any unexpected bits set will be noticed by the boot loader and the boot loader will know that it can't load the boot image. The upper 16 bits act as more of a "minor version", providing hints that the boot loader doesn't need to understand in order to boot the image. But one thing I just thought of... It might be a good idea to have a version field of some kind (maybe another flags field) in the multiboot_info structure passed to the boot image, for the same purpose... >> If bit 16 in the flags word is set, then the fields at offsets 8-24 in the >> multiboot_header are valid, and the boot loader should use them instead of >> the fields in the actual executable header to calculate where to load the >> OS image. This bit should generally be set in OS images in a.out format; >> boot loaders will generally refuse to load a.out images that don't have >> this bit set. > >So, is the presumption that the executable header is in its normal location >in the file? Or is there some offset induced by the presence of the >MultiBoot header? The "real" executable header is always in its normal location; the MultiBoot header can simply be embedded in the beginning of the text segment. That way you don't need any special tools to build MultiBoot-compliant boot images; just link an ordinary executable containing the magic MultiBoot header in crt0. >> Boot Information Format >> ~~~~~~~~~~~~~~~~~~~~~~~ > >Also looks good, but the padding that is on the end of each structure >seems unnecessary, and better handled by a "MultiBoot Info" version number. Yes, I agree. (Although I'd prefer a flags field, like in the multiboot header; that should be more flexible and cause fewer unnecessary incompatibilities.) >> Should the boot loader zero the kernel's bss segment? > >It might be a good "assumption test" to not do it, but why provoke >possible problems? I'd prefer the consistency of always zeroing it, >but if others object, I'd be willing to say "zero it if possible, but >don't assume it is zeroed". I think we need to state explicitly whether upon entry to the OS the bss segment is zeroed or "undefined". I'm leaning toward leaving it undefined, because it's generally very easy for the OS to zero its own bss segment (all the ones I've seen already do it anyway), and it may not be so easy for the boot loader to do it, because of the funny gymnastics boot loaders sometimes have to perform in order to load the OS "on top of" themselves. >A cleaner solution might be for the loader to simply not place >anything in this area, and require that kernels who need it zeroed do it >first thing. That's the way it is right now (although perhaps not stated clearly enough; I'll fix that). >> Should the boot loader pass additional information >> that the OS typically wants to ask the BIOS for, >> but is often too stupid to switch into 16-bit mode to do it? >> * Screen mode >> * Hard disk parameter tables > >This is relatively easy to do for a boot-block. The question in >my mind is how easy is this to implement from other OS's... DOS is easy, >but what others might there be? That's one issue. Another is where exactly do you draw the line? For example, Linux traditionally doesn't just want to know about the "current screen mode"; it also expects the boot loader to prompt the user for the mode if necessary, and pass the screen size along with the mode. Should we specify that boot loaders must do this, even though it's irrelevant for every OS except Linux? Lately I've been leaning more toward removing of all "helpful BIOS-derived info" from the multiboot_info structure, and simply requiring the OS call the BIOS itself if it needs to. FreeBSD already knows how to do this, with the advanced power management support; it could probably easily be modified to grab the hard disk parameter tables at the same time. Linux doesn't yet have the necessary support, I don't think, but the BIOS call code I developed for my DOS extender and the boot adaptors could probably be grafted onto it quite easily; I'm willing to do that myself if necessary. >> Is the 'avail' list in the multiboot_info unnecessary, since the OS could >> easily ask the BIOS for the available memory itself, or probe for memory? >> >> Similarly, is the 'occupied' list unnecessary, since the OS could derive it >> from the other information in the multiboot_info structure? > >I think having the 'avail' list is a good idea (that way the OS doesn't >need to muck with that BIOS call, which it would only do once anyway). The reason I'm having second thoughts about it is it seems that if the OS wants to ask the BIOS for memory info, it's almost certainly going to want to ask the BIOS for other stuff as well, so it'll have to be able to make BIOS calls anyway. For example, Linux wants both RAM info and HD params and screen mode; FreeBSD wants HD params but doesn't want RAM info - it does a manual probe. So the 'avail' list is really helping neither of them. And if we try to throw enough stuff in the multiboot_info structure so that _no_ OS ever needs to go to the BIOS, we could quickly reach absurdity... >The 'occupied' list seems unnecessary, since it is easy to derive, and it >is in much the same format in the boot-module list anyway, except for the >kernel location itself. I'd say just replace the "occupied.count" and >"occupied.regions" with a "kernel.start" and "kernel.end", then you have >everything you need. Technically yes; but I've found it extremely useful in the OS to have a nice, uniform list available that it can simply iterate through, without having to test explicitly against the kernel and the kernel's command line and each boot module and each boot module's command line and... and... Although the OS could calculate this list the first thing, I've found it easier for the boot loader to do it because at that point the boot loader is all set-up and running and can allocate non-conflicting storage for it the same way it does for all the other stuff; whereas if the OS had to create the list, it would have to do it before its own internal memory allocation mechanisms are set up, and thus allocate space for the list staticly or using some other horrible kludge. >> Should command lines be passed as a list of null terminated strings, >> more like Unix command lines, instead of just one string? > >I don't see how this really matters that much. What's easier for >the kernel to use... offhand I'd say a list of null-terminated strings >is easier since people are used to parsing them as options. Let's just >make a decision and stick to it. I did it the way it is for starters because that's what Linux expects, but it's easy to change. I'm leaning a little toward Unix-style pre-parsed command lines too. Any other preferences? >Sorry for the long-winded response... Well, as you probably noticed, this response is even worse. :-) Bryan From erich@uruk.org Wed Apr 5 09:18:14 1995 id JAA20866; Wed, 5 Apr 1995 09:18:10 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Bryan Ford cc: multiboot@schirf.cs.utah.edu Subject: Re: MultiBoot Standard In-reply-to: Your message of "Tue, 04 Apr 1995 19:41:55 MDT." <199504050141.TAA17120@schirf.cs.utah.edu> Date: Wed, 05 Apr 1995 07:54:09 -0700 Sender: erich@uruk.org > That's what the flags longword is for. The lower 16 bits act like a "major > version" in a sense, in that any unexpected bits set will be noticed by > the boot loader and the boot loader will know that it can't load the > boot image. The upper 16 bits act as more of a "minor version", providing > hints that the boot loader doesn't need to understand in order to boot the > image. The same kind of function could be done with it, the only offhand difference being that once you run out of 16-bits, you need another sequence word or number, while a number tends to last much longer on that scale (it could also be a major vs. minor number thing). It isn't enough to quibble about, though. > The "real" executable header is always in its normal location; the MultiBoot > header can simply be embedded in the beginning of the text segment. That way > you don't need any special tools to build MultiBoot-compliant boot images; > just link an ordinary executable containing the magic MultiBoot header in > crt0. I understand what you were driving at now. My only issue with that is reliable detection, and I feel unsure that a 4-byte value for the magic number is sufficient (especially since it is even, an odd number is more unlikely to occur by itself in code). What I could do is search the entire area for MultiBoot headers, and if it finds more than one, it could check the numbers for "sanity", and if it finds multiple "sane" headers, it will whimper and complain about it. > >> Boot Information Format > >> ~~~~~~~~~~~~~~~~~~~~~~~ > > > >Also looks good, but the padding that is on the end of each structure > >seems unnecessary, and better handled by a "MultiBoot Info" version number. > > Yes, I agree. (Although I'd prefer a flags field, like in the multiboot > header; that should be more flexible and cause fewer unnecessary > incompatibilities.) Same comment as above about the bits vs. sequence number... ...[talking about zeroing the kernel-images bss]... > >A cleaner solution might be for the loader to simply not place > >anything in this area, and require that kernels who need it zeroed do it > >first thing. > > That's the way it is right now (although perhaps not stated clearly enough; > I'll fix that). Good enough. > Lately I've been leaning more toward removing of all "helpful BIOS-derived > info" from the multiboot_info structure, and simply requiring the OS call > the BIOS itself if it needs to. FreeBSD already knows how to do this, with > the advanced power management support; it could probably easily be modified > to grab the hard disk parameter tables at the same time. Linux doesn't yet > have the necessary support, I don't think, but the BIOS call code I developed > for my DOS extender and the boot adaptors could probably be grafted onto it > quite easily; I'm willing to do that myself if necessary. I see your point. Sounds reasonable enough to me. > The reason I'm having second thoughts about it is it seems that if the OS > wants to ask the BIOS for memory info, it's almost certainly going to want > to ask the BIOS for other stuff as well, so it'll have to be able to make > BIOS calls anyway. For example, Linux wants both RAM info and HD params > and screen mode; FreeBSD wants HD params but doesn't want RAM info - it does > a manual probe. So the 'avail' list is really helping neither of them. > And if we try to throw enough stuff in the multiboot_info structure so that > _no_ OS ever needs to go to the BIOS, we could quickly reach absurdity... Again, letting the OS do the work, especially with all the sample code around, seems simple enough. ...[talking about a "memory occupied list"]... > Technically yes; but I've found it extremely useful in the OS to have > a nice, uniform list available that it can simply iterate through, without > having to test explicitly against the kernel and the kernel's command line > and each boot module and each boot module's command line and... and... > Although the OS could calculate this list the first thing, I've found > it easier for the boot loader to do it because at that point the boot > loader is all set-up and running and can allocate non-conflicting storage > for it the same way it does for all the other stuff; whereas if the OS > had to create the list, it would have to do it before its own internal > memory allocation mechanisms are set up, and thus allocate space for the > list staticly or using some other horrible kludge. It wouldn't be much harder to have an iterator be capable of going through the kernel and module + command line values vs. a simple "space used" list. The only problem I have is deciding what's used by what. If something is in the "space used" list, does that mean one has to iterate through the other list to decide what it is? It seems to add complexity later on. I suppose one advantage of the "space used" list is that it can reserve a special area such as the BIOS variable area easily, but unless it could be tagged as something special anyway (so that you can decide what to do with it later) then I'm not sure I think it is a good idea... Basically, I feel hesitant to use an untagged "space used" list. It seems like it might bring up synchronization issues in later code if something is deleted from one list but not the other, etc. > >I don't see how this really matters that much. What's easier for > >the kernel to use... offhand I'd say a list of null-terminated strings > >is easier since people are used to parsing them as options. Let's just > >make a decision and stick to it. > > I did it the way it is for starters because that's what Linux expects, > but it's easy to change. I'm leaning a little toward Unix-style > pre-parsed command lines too. Any other preferences? Nope. -- Erich Stefan Boleyn \___ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \____ home #: +1 (503) 226-0741 WWW Site | USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From phk@ref.tfs.com Thu Apr 13 13:58:34 1995 id NAA21847; Thu, 13 Apr 1995 13:58:31 -0600 From: Poul-Henning Kamp Subject: Hi gang, just arrived :-) To: multiboot@schirf.cs.utah.edu Date: Thu, 13 Apr 1995 12:53:30 -0700 (PDT) Content-Type: text Content-Length: 628 Hi guys, I just joined you, I'm from the FreeBSD core team. I admit I havn't had time to read all the stuff yet, but I will. FreeBSD has a couple of problems we would like to get solved, and we also have some code which I hope you can benefit from. In particular I would like to point out that we have working netboot and "dosboot" programs. We should probably try to make these follow this standard too, if people are interested that is ? -- Poul-Henning Kamp -- TRW Financial Systems, Inc. 'All relevant people are pertinent' && 'All rude people are impertinent' => 'no rude people are relevant' From baford Thu Apr 13 14:03:20 1995 id OAA21870; Thu, 13 Apr 1995 14:03:20 -0600 To: Poul-Henning Kamp cc: multiboot@schirf Subject: Re: Hi gang, just arrived :-) In-reply-to: Your message of "Thu, 13 Apr 95 12:53:30 PDT." <199504131953.MAA14355@ref.tfs.com> Date: Thu, 13 Apr 95 14:03:19 MDT From: Bryan Ford >FreeBSD has a couple of problems we would like to get solved, and we >also have some code which I hope you can benefit from. > >In particular I would like to point out that we have working netboot >and "dosboot" programs. We should probably try to make these follow >this standard too, if people are interested that is ? Yes, definitely! A good OS-independent network boot mechanism would be very valuable. Bryan From baford Tue Apr 18 10:17:16 1995 id KAA12144; Tue, 18 Apr 1995 10:17:15 -0600 To: erich@uruk.org cc: multiboot@schirf Subject: Re: MultiBoot Standard In-reply-to: Your message of "Wed, 05 Apr 95 07:54:09 PDT." <199504051454.HAA09627@uruk.org> Date: Tue, 18 Apr 95 10:17:15 MDT From: Bryan Ford >> The "real" executable header is always in its normal location; the MultiBoot >> header can simply be embedded in the beginning of the text segment. That way >> you don't need any special tools to build MultiBoot-compliant boot images; >> just link an ordinary executable containing the magic MultiBoot header in >> crt0. > >I understand what you were driving at now. My only issue with that is >reliable detection, and I feel unsure that a 4-byte value for the magic >number is sufficient (especially since it is even, an odd number is more >unlikely to occur by itself in code). I doubt it'll be much of a problem in practice. The MultiBoot header will probably generally be after only the a.out header and possibly a single 'jmp' instruction and a few 'nop's - not much chance of the 32-bit value 0x1BADB002 appearing in any of those I don't think. If the boot loader simply searches for the first instance of this magic value, it doesn't matter if it appears elsewhere in the "real" program code (still unlikely I think). >What I could do is search the entire area for MultiBoot headers, and >if it finds more than one, it could check the numbers for "sanity", and >if it finds multiple "sane" headers, it will whimper and complain about >it. Sure, you're welcome to do that, although I really don't think it's worth the trouble. :-) >...[talking about a "memory occupied list"]... > >> Technically yes; but I've found it extremely useful in the OS to have >> a nice, uniform list available that it can simply iterate through, without >> having to test explicitly against the kernel and the kernel's command line >> and each boot module and each boot module's command line and... and... >> Although the OS could calculate this list the first thing, I've found >> it easier for the boot loader to do it because at that point the boot >> loader is all set-up and running and can allocate non-conflicting storage >> for it the same way it does for all the other stuff; whereas if the OS >> had to create the list, it would have to do it before its own internal >> memory allocation mechanisms are set up, and thus allocate space for the >> list staticly or using some other horrible kludge. > >It wouldn't be much harder to have an iterator be capable of going >through the kernel and module + command line values vs. a simple >"space used" list. Yes, you're probably right. Maybe the best thing to do is just to go with the "absolute minimalist" approach, and get rid of both the avail and occupied lists. >The only problem I have is deciding what's used by what. If something is >in the "space used" list, does that mean one has to iterate through the >other list to decide what it is? It seems to add complexity later on. Actually, the occupied list was _only_ intended to be used by the OS as a convenience to avoid overwriting stuff it's going to need later - the OS isn't supposed to be able to (or need to) know exactly what's in those reserved memory chunks. To grok the actual information, the OS must follow the pointers in the rest of the Info structure. >I suppose one advantage of the "space used" list is that it can reserve >a special area such as the BIOS variable area easily, but unless it could >be tagged as something special anyway (so that you can decide what to do >with it later) then I'm not sure I think it is a good idea... As designed it wouldn't be useful for this purpose - the OS would be free to overwrite that area after it's done with the Info structure and everything it contains. The boot loader would instead have to leave the BIOS variables area out of the avail list, or the OS itself would have to know not to tromp on that area of memory. The boot standard as written doesn't define what boot loaders should do in this regard, and it seems like only the OS can know whether or not it needs to avoid tromping on BIOS variables because it is (or isn't) going to make BIOS callbacks. I guess this just puts another nail in the coffin of the 'avail' list. >>[...whether command lines should be one string or multiple strings, >> like Unix command lines...] >> >> >I don't see how this really matters that much. What's easier for >> >the kernel to use... offhand I'd say a list of null-terminated strings >> >is easier since people are used to parsing them as options. Let's just >> >make a decision and stick to it. >> >> I did it the way it is for starters because that's what Linux expects, >> but it's easy to change. I'm leaning a little toward Unix-style >> pre-parsed command lines too. Any other preferences? > >Nope. Actually, I was trying to solicit opinions from the other so-far-silent members of the list. :-) I'd like to hear from others about these issues, just to be sure we aren't missing something - a simple "yes" or "no" would be fine if you don't think there's more to say: * Whether or not to axe the 'avail' list * Whether or not to axe the 'occupied' list * Whether command lines should be one string (Linux kernel/LILO-style), or multiple strings (Unix-style) Bryan From roland@gnu.ai.mit.edu Tue Apr 18 11:30:23 1995 id LAA12208; Tue, 18 Apr 1995 11:30:22 -0600 Date: Tue, 18 Apr 1995 13:25:02 -0400 From: Roland McGrath To: baford@schirf.cs.utah.edu Cc: erich@uruk.org, multiboot@schirf.cs.utah.edu Subject: Re: MultiBoot Standard In-Reply-To: Bryan Ford's message of Tue, 18 April 1995 10:17:15 MDT <199504181617.KAA12144@schirf.cs.utah.edu> X-Windows: A moment of convenience, a lifetime of regret. My inclination is to be very minimalist: > * Whether or not to axe the 'avail' list > * Whether or not to axe the 'occupied' list Yes, axe both. It simplifies life and reduces assumptions about what the OS will be doing once it loads. It is easy enough for the kernel to copy all the data passed from the boot loader into space in its own data segment very early on, so it need not worry about the issue further and can just use all memory as it sees fit. > * Whether command lines should be one string (Linux kernel/LILO-style), > or multiple strings (Unix-style) Again, simplicity. If what the boot loader does is read ASCII from a file or a keyboard, then it might as well just give the kernel the ASCII it got. This is trivial code in the kernel to parse it, and simplifies the boot loader. From roland@gnu.ai.mit.edu Thu Apr 27 12:18:18 1995 id MAA23642; Thu, 27 Apr 1995 12:18:17 -0600 Date: Thu, 27 Apr 1995 14:12:22 -0400 From: Roland McGrath To: multiboot@schirf.cs.utah.edu Subject: status X-Zippy-Says: My CODE of ETHICS is vacationing at famed SCHROON LAKE in upstate New York!! What is the current state of the work related to multiboot? Last I recall hearing, Erich had a 386 boot block written and in some unclear state of functionality; and Bryan has been doing things to let mach4 be built in the ELF format and booted that way by something or other. Can you both elaborate on the current state of your work? Can I get a snapshot of your development code, just to read for informational purposes? Is there anyone else actively working on related code? Thanks, Roland From erich@uruk.org Thu Apr 27 12:41:39 1995 id MAA23794; Thu, 27 Apr 1995 12:41:37 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Roland McGrath cc: multiboot@schirf.cs.utah.edu Subject: Re: status In-reply-to: Your message of "Thu, 27 Apr 1995 14:12:22 EDT." <199504271812.OAA15945@churchy.gnu.ai.mit.edu> Date: Thu, 27 Apr 1995 11:37:24 -0700 Sender: erich@uruk.org > What is the current state of the work related to multiboot? > > Last I recall hearing, Erich had a 386 boot block written and in some > unclear state of functionality; and Bryan has been doing things to let > mach4 be built in the ELF format and booted that way by something or other. > Can you both elaborate on the current state of your work? Can I get a > snapshot of your development code, just to read for informational purposes? I haven't had any time to work on this stuff since my last status message, as I've been unusually busy at work. I'll make a snapshot available on Monday sometime that will have the basic functionality, and have a lot of the rest of the fanciness in after the next week. > Is there anyone else actively working on related code? Just Poul-Henning Kamp saying that the FreeBSD group would like to update their netboot and dosboot code to support the multiboot format. This wouldn't guarantee support of multi-module loading, just the loading and kernel interface. An effort to make my code sufficiently modular to be used to update both the netboot and dosboot will probably be made after the next week, as well. The dosboot one is going to be tricky, though... so I may just leave that to the people who've studied the myriad of DOS extender interfaces. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From erich@uruk.org Sun Apr 30 22:17:03 1995 id WAA20550; Sun, 30 Apr 1995 22:17:00 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Roland McGrath cc: multiboot@schirf.cs.utah.edu Subject: Status update on my boot-block stuff (was -> Re: status ) In-reply-to: Your message of "Thu, 27 Apr 1995 14:12:22 EDT." <199504271812.OAA15945@churchy.gnu.ai.mit.edu> Date: Sun, 30 Apr 1995 21:13:28 -0700 Sender: erich@uruk.org > What is the current state of the work related to multiboot? I have 2 different stage1 boot-blocks written (also referred to as an MBR, the first 512-byte sector of the disk). The first one is BIOS disk geometry dependent, but can have many sets of cylinder/head/sector start and sector length parameters. The big limitations are (1) that each one can't cross head or cylinder boundaries, and (2) it is disk controller and mapping dependent. #2 is generally only a problem when moving a SCSI disk from one controller to another, but it is a pain to have to specify each one in terms of the BIOS translation when programming it. This one is finished and tested. The second one performs automatic BIOS disk geometry translation, so the parameters you program it with are logical sector start and length parameters (any length up to available memory in real mode, i.e. anything under 1MB). This kind of data is easily available to programs in UNIX, and is also independent of the controller or disk in use. This one is coded, but needs futher fixing to enable it to work right, and that will likely be finished sometime monday or tuesday. Supposedly, some BIOS's produce bad results for automatic BIOS translation code with floppies (i.e. inconsistent or missing results)... those cases don't seem to be handled by most of the other boot-loaders I've looked at either, as certainly when they get to the "stage2" part, they almost exclusively use logical sector mapping, though almost never in the stage1. The basic result being I'd prefer to use the second one since it is more easy to use and is generally guaranteed to work when moving disks between different machines and controllers... but people can have the code for both if they want to use them. The stage2 I'm working on is substantially unwritten yet. The main parts that are done is a partial re-write of the protected-mode switching code and BIOS code to allow the main body of code to run in a true 32-bit flat memory model with all of memory accessible. This is complete and tested with some sample UI code I was playing with. > Can I get a > snapshot of your development code, just to read for informational purposes? Sure. There is a snapshot available (with some of my notes, but not sufficiently documented yet) on my ftp site "ftp.uruk.org" as "anonymous" in "/private/flexboot/flexboot.tar.gz". The stage2 subdirectory in the tar file, "flexboot/flexboot", doesn't have much in it, since I was testing switching code, and removed almost all the FreeBSD code from it during the tests. I'll have enough time this week to make substantial progress, so I'll send an update on Friday or next Monday. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From baford Fri May 5 09:43:55 1995 id JAA06927; Fri, 5 May 1995 09:43:55 -0600 To: multiboot Subject: Proposal-0.2 Reply-to: baford@cs.utah.edu Date: Fri, 05 May 95 09:43:55 MDT From: Bryan Ford I've just put a new version of the proposal in the FTP directory, called 'Proposal-0.2'. The first one is still there, as 'Proposal-0.1'. Note that the current Mach4 source code still reflects version 0.1; it hasn't yet been updated for version 0.2. The changes basically just reflect what's been discussed on the list so far. Anyway, I'm hoping that version 0.2 will be pretty close to the final version... I'd like to get it nailed down pretty soon so we can start using it seriously without having to keep updating everything. Please pick apart Proposal-0.2 with a fine-tooth comb and help me find any missing or unclear aspects still remaining, or problems that might prevent it from being usable in some situations. For your convenience, I'm attaching a diff to this message so you can see exactly what's been changed. Thanks! Bryan *** Proposal-0.1 Mon Apr 3 19:44:06 1995 --- Proposal-0.2 Fri May 5 09:26:23 1995 *************** *** 1,7 **** MultiBoot Standard PRELIMINARY PROPOSAL ! Version 0.1 MOTIVATION --- 1,7 ---- MultiBoot Standard PRELIMINARY PROPOSAL ! Version 0.2 MOTIVATION *************** *** 115,130 **** formats in existence in order to load the OS image - otherwise the boot loader effectively becomes OS-specific again. ! The tentative solution this standard adopts is based on two assumptions: ! (1) that most free Unix-like OS's are likely to move to ELF before too ! long, which is sufficiently standardized and well-defined that boot loaders ! should be able to load OS images in ELF format without becoming too ! OS-specific; and (2) that the current problems of a.out format can be ! kludged around in the interim. This should make it easy for OS images to ! be generated using the standard tools available on free OS's, while ! limiting the number of OS image formats that boot loaders must understand ! to one or, at most, two. The details of this solution appear later in this ! document. Boot modules ~~~~~~~~~~~~ --- 115,128 ---- formats in existence in order to load the OS image - otherwise the boot loader effectively becomes OS-specific again. ! This standard adopts a compromise solution to this problem. ! MultiBoot compliant boot images always either (a) are in ELF format, or (b) ! contain a "magic MultiBoot header", described below, which allows the boot ! loader to load the image without having to understand numerous a.out ! variants or other executable formats. This magic header does not need ! to be at the very beginning of the executable file, so kernel images can ! still conform to the local a.out format variant in addition to being ! MultiBoot compliant. Boot modules ~~~~~~~~~~~~ *************** *** 172,180 **** the text segment is supposed to start. Therefore, this standard requires that an additional header, known as a 'multiboot_header', appear somewhere near the beginning of the executable file. In general it should come "as ! early as possible". It _must_ be contained completely within the first ! 8192 bytes of the file, and must be longword (32-bit) aligned. The layout ! of the header is as follows: +-------------------+ 0 | magic: 0x1BADB002 | (required) --- 170,182 ---- the text segment is supposed to start. Therefore, this standard requires that an additional header, known as a 'multiboot_header', appear somewhere near the beginning of the executable file. In general it should come "as ! early as possible", and is typically embedded in the beginning of the text ! segment after the "real" executable header. It _must_ be contained ! completely within the first 8192 bytes of the executable file, and must be ! longword (32-bit) aligned. These rules allow the boot loader to find and ! synchronize with the text segment in the a.out file without knowing ! beforehand the details of the a.out variant. The layout of the header is ! as follows: +-------------------+ 0 | magic: 0x1BADB002 | (required) *************** *** 199,205 **** bits in this range are set but the boot loader doesn't understand them, it can simply ignore them and proceed as usual. Naturally, all as-yet-undefined bits in the flags word must be set to zero in OS ! images. If bit 0 in the flags word is set, then all boot modules loaded along with the OS must be aligned on page (4KB) boundaries. Some OS's expect to be --- 201,208 ---- bits in this range are set but the boot loader doesn't understand them, it can simply ignore them and proceed as usual. Naturally, all as-yet-undefined bits in the flags word must be set to zero in OS ! images. This way, the flags fields serves for version control as well as ! simple feature selection. If bit 0 in the flags word is set, then all boot modules loaded along with the OS must be aligned on page (4KB) boundaries. Some OS's expect to be *************** *** 209,217 **** If bit 16 in the flags word is set, then the fields at offsets 8-24 in the multiboot_header are valid, and the boot loader should use them instead of the fields in the actual executable header to calculate where to load the ! OS image. This bit should generally be set in OS images in a.out format; ! boot loaders will generally refuse to load a.out images that don't have ! this bit set. All of the address fields enabled by flag bit 16 are physical addresses. The meaning of each is as follows: --- 212,224 ---- If bit 16 in the flags word is set, then the fields at offsets 8-24 in the multiboot_header are valid, and the boot loader should use them instead of the fields in the actual executable header to calculate where to load the ! OS image. This information does not need to be provided if the kernel ! image is in ELF format, but it should be provided if the images is in a.out ! format or in some other format. Compliant boot loaders must be able to ! load images that either are in ELF format or contain the load address ! information embedded in the multiboot_header; they may also directly ! support other executable formats, such as particular a.out variants, but ! are not required to. All of the address fields enabled by flag bit 16 are physical addresses. The meaning of each is as follows: *************** *** 261,269 **** * The processor interrupt flag must be turned off. ! * EAX must contain the 32-bit physical address ! of the multiboot_info structure provided by the boot loader ! (see below). All other registers and flag bits are undefined. This includes, in particular: --- 268,280 ---- * The processor interrupt flag must be turned off. ! * EAX must contain the magic value 0x2BADB002; the presence of this value ! indicates to the OS that it was loaded by a MultiBoot-compliant boot ! loader (e.g. as opposed to another type of boot loader that the OS can ! also be loaded from). ! ! * EBX must contain the 32-bit physical address of the multiboot_info ! structure provided by the boot loader (see below). All other registers and flag bits are undefined. This includes, in particular: *************** *** 278,283 **** --- 289,298 ---- * PIC: The interrupt controller may be in any state, and programmed for any set of interrupt vectors; the OS must reprogram it before enabling interrupts. + * All RAM not occupied by data created by the boot loader for the OS. + This includes the OS's bss segment: if it needs it initialized to zero, + the OS must zero it itself the first thing. + Boot Information Format ~~~~~~~~~~~~~~~~~~~~~~~ *************** *** 292,375 **** reserved for the kernel and boot modules, of course). It is the OS's responsibility to avoid overwriting this memory until it is done using it. ! The format of the multiboot_info structure is as follows: +-------------------+ ! 0 | avail.count | ! 4 | avail.regions | ! +-------------------+ ! 8 | occupied.count | ! 12 | occupied.regions | +-------------------+ ! 16 | cmdline.start | ! 20 | cmdline.end | +-------------------+ ! 24 | mods.count | ! 28 | mods.regions | ! +-------------------+ ! 32 | reserved (0) | ! 36 | reserved (0) | ! 40 | reserved (0) | ! 44 | reserved (0) | ! +-------------------+ ! ! The first two longwords describe the physical memory available to the OS, ! as a list of regions. 'avail.count' contains the number of regions, and ! 'avail.regions' contains the physical address of the first region ! structure. Each region structure has the following format: ! +-------------------+ ! 0 | start | ! 4 | end | +-------------------+ ! The memory region extends from byte 'start' to byte 'end-1' inclusive. ! The memory listed in the 'avail' list may (and probably does) include the ! memory in which the kernel, the boot modules, and the multiboot_info and ! related structures are loaded. The entries in the 'avail' list must not ! overlap each other. ! ! The second two longwords in the multiboot_info structure contain a second ! region list, stored the same way as the 'avail' list, indicating all ! regions of memory occupied by things the boot loader loaded into memory ! that might be relevant to the OS. This includes the kernel, the boot ! modules, the multiboot_header, and all substructures it points to. ! This list allows the OS to easily avoid overwriting those memory regions ! until it is done using them. ! The 'cmdline' fields in the multiboot_info contain the physical start and ! end addresses of a region containing the command line to be passed to the ! kernel. The command line is a normal C-style null-terminated string. The ! 'end' pointer points at least one byte past the string's terminating null. ! If there is no command line, the start and end pointers will both be zero. ! The 'mods' fields indicate to the kernel what boot modules were loaded ! along with the kernel image, and where they can be found. 'mods.count' ! contains the number of modules loaded; 'mods.regions' contains the physical ! address of the first module structure. Each module structure is formatted ! as follows: +-------------------+ 0 | mod.start | 4 | mod.end | +-------------------+ ! 8 | cmdline.start | ! 12 | cmdline.end | +-------------------+ ! 16 | reserved (0) | ! 20 | reserved (0) | ! 24 | reserved (0) | ! 28 | reserved (0) | +-------------------+ The first two fields contain the start and end addresses of the boot module itself. The second two fields provide any command line associated with ! that particular boot module; it is formatted the same way as the main ! kernel command line. ! ! All fields marked 'reserved', both in the multiboot_info structure and the ! multiboot_module structures, must be initialized to zero by the boot ! loader. EXAMPLE CODE --- 307,365 ---- reserved for the kernel and boot modules, of course). It is the OS's responsibility to avoid overwriting this memory until it is done using it. ! The format of the multiboot_info structure (as defined so far) follows: +-------------------+ ! 0 | flags | (required) +-------------------+ ! 4 | cmdline | (present if flags[0] is set) +-------------------+ ! 8 | mods.count | (present if flags[1] is set) ! 12 | mods.pointer | (present if flags[1] is set) +-------------------+ ! 16 | syms.count | (present if flags[2] is set) ! 20 | syms.pointer | (present if flags[2] is set) +-------------------+ ! The first longword indicates the presence and validity of other fields in ! the multiboot_info structure. All as-yet-undefined bits must be set to ! zero by the boot loader. Any set bits that the OS does not understand ! should be ignored. Thus, the flags field also functions as a version ! indicator, allowing the multiboot_info structure to be expanded in the ! future without breaking anything. ! If bit 0 of the flags longword is set, the 'cmdline' field is valid, and ! contains the physical address of the the command line to be passed to the ! kernel. The command line is a normal C-style null-terminated string. ! If bit 1 of the flags is set, then the 'mods' fields indicate to the kernel ! what boot modules were loaded along with the kernel image, and where they ! can be found. 'mods.count' contains the number of modules loaded; ! 'mods.regions' contains the physical address of the first module structure. ! 'mods.count' may be zero, indicating no boot modules were loaded, even if ! bit 1 of 'flags' is set. Each module structure is formatted as follows: +-------------------+ 0 | mod.start | 4 | mod.end | +-------------------+ ! 8 | cmdline | +-------------------+ ! 12 | reserved (0) | +-------------------+ The first two fields contain the start and end addresses of the boot module itself. The second two fields provide any command line associated with ! that particular boot module; it is a null-terminated ASCII string, just ! like the kernel command line. The 'reserved' field must be set to 0 by the ! boot loader and ignored by the OS. ! ! If bit 2 in the multiboot_info.flags word is set, then the 'syms.count' and ! 'syms.pointer' fields are valid and indicate where the symbol table from ! the kernel image can be found. 'syms.pointer' is the physical address of ! the first element in an array of a.out-format 'nlist' structures, and ! 'syms.count' is the number of elements in the array. 'syms.count' may be ! 0, indicating no symbols, even if bit 2 in the flags word is set. EXAMPLE CODE *************** *** 391,396 **** --- 381,387 ---- directories of primary relevance: mach4-i386/boot + mach4-i386/boot/bsd mach4-i386/boot/linux The Mach kernel itself in this distribution contains code that demonstrates *************** *** 408,425 **** ISSUES ~~~~~~ Should the boot loader zero the kernel's bss segment? - - Should the boot loader pass additional information - that the OS typically wants to ask the BIOS for, - but is often too stupid to switch into 16-bit mode to do it? - * Screen mode - * Hard disk parameter tables - - Is the 'avail' list in the multiboot_info unnecessary, since the OS could - easily ask the BIOS for the available memory itself, or probe for memory? - - Similarly, is the 'occupied' list unnecessary, since the OS could derive it - from the other information in the multiboot_info structure? Should command lines be passed as a list of null terminated strings, more like Unix command lines, instead of just one string? --- 399,404 ---- From erich@uruk.org Sun May 7 23:20:45 1995 id XAA24861; Sun, 7 May 1995 23:20:43 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: baford@cs.utah.edu cc: multiboot@schirf.cs.utah.edu Subject: Re: Proposal-0.2 In-reply-to: Your message of "Fri, 05 May 1995 09:43:55 MDT." <199505051543.JAA06927@schirf.cs.utah.edu> Date: Sun, 07 May 1995 22:15:10 -0700 Sender: erich@uruk.org Bryan Ford writes: > I've just put a new version of the proposal in the FTP directory, > called 'Proposal-0.2'. The first one is still there, as 'Proposal-0.1'. > Note that the current Mach4 source code still reflects version 0.1; > it hasn't yet been updated for version 0.2. The changes basically > just reflect what's been discussed on the list so far. > > Anyway, I'm hoping that version 0.2 will be pretty close to the final > version... I'd like to get it nailed down pretty soon so we can start > using it seriously without having to keep updating everything. Please > pick apart Proposal-0.2 with a fine-tooth comb and help me find any > missing or unclear aspects still remaining, or problems that might > prevent it from being usable in some situations. For your convenience, > I'm attaching a diff to this message so you can see exactly what's > been changed. My comments are all from a somewhat cursory look, but these were the only things that jumped out at me. First of all, there is a presumption of an "a.out" style symbol table. I don't know very much about debugging info, but this seems somewhat restrictive unless the "syms.count, syms.pointer" model is equivalent to the others in use. >From "Proposal-0.2": ... > * EAX must contain the magic value 0x2BADB002; the presence of this value > indicates to the OS that it was loaded by a MultiBoot-compliant boot > loader (e.g. as opposed to another type of boot loader that the OS can > also be loaded from). > > * EBX must contain the 32-bit physical address of the multiboot_info > structure provided by the boot loader (see below). ... > Boot Information Format > ~~~~~~~~~~~~~~~~~~~~~~~ > Upon entry to the OS, the EAX register contains the physical address of > a 'multiboot_info' data structure, through which the boot loader > communicates vital information to the OS. The OS can use or ignore any > parts of the structure as it chooses; all information passed by the boot > loader is advisory only. This seems to be a typo. Is EAX the physical address, or EBX ? ... > ISSUES > ~~~~~~ > Should the boot loader zero the kernel's bss segment? I don't think it really matters much each way as long as we make a decision. The only real advantage I can see of the kernel zeroing it's own bss is if you're loading it in low memory and it would overlap the boot-loader, like it does with some current ones. > Should the PIC be guaranteed to be programmed for the normal BIOS/DOS > interrupt vectors on entry? For low-level boot, the PIC is already set up. For other situations, it can be a pain. There are some advantages to presuming it is pre-set-up, though, in that you can have re-programmed the interrupt vectors for something special... but I don't know how valuable this is. Erich -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From erich@uruk.org Mon May 8 19:53:37 1995 id TAA03380; Mon, 8 May 1995 19:53:33 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Multiboot Proposal 0.2 Date: Mon, 08 May 1995 18:48:15 -0700 Sender: erich@uruk.org Well, a few things I forgot to add to my comments on "Proposal-0.2" is: 1) How does the kernel know from which drive it was booted? 2) How does the kernel know from which partition it was booted? I already wanted to get away from the BSD naming scheme, because it precludes using this booting system for anything that doesn't have a BSD disklabel, quite annoying. It turns out that the FreeBSD people solved a similar problem, so FreeBSD-current (and the lastest snapshot) now has code to read generic DOS-style partitions and to distinguish how they lie on the disk. Their scheme is based on the concept of "slices". An article explaining the concept from one of the FreeBSD newsgroups is included at the end of this message. I'm still a little unclear on the terminology for DOS extended partitions, so I'll get that part clarified. I suggest passing the "root device" name as a string. The "slice" naming scheme is sufficient to define precisely what partition to go to on what drive, including naming funny root devices like CD-ROMs (i.e. a root device for booting that is different from where the file came from). Erich ------------------------(begin article here)------------------------- > Path: usenet.ee.pdx.edu!news.reed.edu!sun.lclark.edu!netnews.nwnet.net!oracle.pnl.gov!osi-east2.es.net!lll-winken.llnl.gov!uwm.edu!math.ohio-state.edu!howland.reston.ans.net!agate!violet.berkeley.edu!jkh > From: jkh@violet.berkeley.edu (Jordan K. Hubbard) > Newsgroups: comp.unix.bsd.freebsd.misc > Subject: Re: Mount MSDOS Extended Partition on FreeBSD > Date: 5 Apr 1995 09:37:17 GMT > Organization: University of California, Berkeley > Lines: 120 > Message-ID: <3ltocd$d81@agate.berkeley.edu> > References: <3lqu3k$7d@pc110.cs.cuhk.hk> <3lt5ga$m0t@eng_ser1.erg.cuhk.hk> > NNTP-Posting-Host: violet.berkeley.edu > > In article <3lt5ga$m0t@eng_ser1.erg.cuhk.hk>, > Vincent S.Y. Yiu wrote: > >Stephen Wong (smwong@cs.cuhk.hk) wrote: > >> Can FreeBSD (2.0R) mount MSDOS extended partitions? > > > >> The installation procedure only maps the primary MSDOS partition, and I > >> don't know how to make the other partitions 'visible' in the FreeBSD disk > >> map. > > > >> Is it possible? Can anyone help? > > > >I don't know if it works with 'extended partitions' however, I've tested > >that the 2.0-920210-SNAP installation automatically imports all the 'primiary > >msdos partitions' into the disklabel. > > Hmmm.. Is that enough context? I think so. Let me speak to you of > "Slices". > > About 2 weeks ago, Bruce Evans started hacking in support > for a more generic mechanism, the slices I mentioned. What slices are > is another abstraction on top of how PCs traditionally map their "partitions". > > In order to more properly explain it, let me first say how things are in > 2.0R (which most here are familiar with) for comparison: > > You have up to 4 "fdisk" partitions per drive, e.g. things like "DOS Primary > and Extended" partitions, Linux partitions, 0xa5 (*BSD) partitions, etc. > Inside the first BSD partition found (which is why you can't have two :-), > the "disklabel" is read. The disklabel divides our fdisk partition further > into (confusingly named) BSD "partitions". These are given letters, > e.g. /dev/sd0a, /dev/sd0b, /dev/sd0e, .. - up to eight of them. /dev/sd0c > and /dev/sd0d are "special" partitions in that they point to the whole > BSD partition and the entire drive, respectively. They are reserved > and cannot be used for pointing at user partitions. > > FreeBSD can also recognise non-BSD partitions (DOS, etc) by simply pointing a > BSD partition entry outside the "BSD partition" (assuming, of course, that > you know where the one you're interested in starts and stops). This has its > drawbacks. For one thing, if you try to read a disk that ONLY has a DOS > partition on it, FreeBSD will spit it up. Why? Because FreeBSD can only > read disks through the disklabel. If a disk contains no BSD partition > containing in turn a valid BSD disk label with sub-partition information > in it, FreeBSD doesn't know what to do. > > Mounting DOS filesystems "inside" extended partitions is even more complicated. > You have to calculate the offsets by hand, and it's basically a messy affair > with disklabel and a calculator. > > Well, that's how it all was until recently, anyway. > > Now we do things through Bruce's "slice" code. Slices give us a much nicer > abstraction for the whole disk than the current system, and they also take > care to do certain useful calculations (like extended DOS partition offsets) > automatically on your behalf. > > Say we have a disk with 1 FreeBSD partition and one DOS Primary partition on > the first drive and one DOS extended partition containing two logical > volumes on the second. We'd like to mount them all. How might this look? > Well.. > > > The data for partition 0 is: > sysid 6,(DOS Primary) > start 32, size 614368 (299 Meg), flag 0 > beg: cyl 0/ sector 1/ head 1; > end: cyl 299/ sector 32/ head 63 > > ... [2 empty partitions] .. > > The data for partition 3 is: > sysid 165,(FreeBSD/NetBSD/386BSD) > start 1433600, size 2674688 (1306 Meg), flag 80 > beg: cyl 700/ sector 1/ head 0; > end: cyl 1023/ sector 32/ head 63 > > > > .. show one extended partition - I don't happen to have one handy to paste > in as a good example, shucks! :-) .. > > > And then furthermore, we'll say that our FreeBSD partition on disk0 > is partitioned into root (a), swap (b), specials (c&d) and user (e). > > Under the slice code, our partitions would look like this: > > /dev/sd0 Entire 0 drive. > /dev/sd1 Entire 1 drive. > > /dev/sd0s1 DOS partition > /dev/sd0s3 Entire FreeBSD slice. > /dev/sd0s3a: root fs > /dev/sd0s3b swap > /dev/sd0s3e user > > /dev/sd1s1a DOS volume A > /dev/sd1s1b DOS volume B > > Several wins: > 1. DOS partition is located easily and irrespectively of any > existent BSD slice. > 2. No partition letters are wasted for special meanings like > partition "c" (whole FreeBSD partition) and "d" (whole disk) > currently are. You can use a-h for yourself. /dev/sd0 > and /dev/sd0s3 designations also make a lot more sense! > > 3. Logical volumes in DOS extended partitions are handled > automatically (there - the reason for this whole post! :-). > > /dev/sd0[a-h] are also provided for backwards compatability and are > synonymous with /dev/sd0s3[a-h]. I don't know if the special behavior > of /dev/sd0c and /dev/sd0d are also emulated, but perhaps Bruce can tell > us.. :-) > > The FreeBSD 2.1 installation will deal entirely with "slices" and not > even use /dev/sd0[a-h] assignments in /etc/fstab. This should make things > quite a bit simpler to understand in the future and a lot less error-prone > besides. > > Jordan > -------------------------(end article here)-------------------------- -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From phk@ref.tfs.com Mon May 8 21:15:15 1995 id VAA03431; Mon, 8 May 1995 21:15:15 -0600 From: Poul-Henning Kamp Subject: Re: Multiboot Proposal 0.2 To: erich@uruk.org Date: Mon, 8 May 1995 20:08:31 -0700 (PDT) Cc: multiboot@schirf.cs.utah.edu In-Reply-To: <199505090148.SAA28121@uruk.org> from "erich@uruk.org" at May 8, 95 06:48:15 pm Content-Type: text Content-Length: 556 > Their scheme is based on the concept of "slices". An article explaining > the concept from one of the FreeBSD newsgroups is included at the end of > this message. I'm still a little unclear on the terminology for DOS > extended partitions, so I'll get that part clarified. Extended partitions are treated as an extension of the MBR, and thus they appear as slices #5... -- Poul-Henning Kamp -- TRW Financial Systems, Inc. 'All relevant people are pertinent' && 'All rude people are impertinent' => 'no rude people are relevant' From roland@gnu.ai.mit.edu Mon May 8 22:24:08 1995 id WAA03487; Mon, 8 May 1995 22:24:07 -0600 Date: Tue, 9 May 1995 00:17:20 -0400 From: Roland McGrath To: erich@uruk.org Cc: multiboot@schirf.cs.utah.edu cc: Shantanu Goel Subject: Re: Multiboot Proposal 0.2 In-Reply-To: erich@uruk.org's message of Mon, 8 May 1995 18:48:15 -0700 <199505090148.SAA28121@uruk.org> X-Shopping-List: (1) Dysfunctional friction stones (2) Laborious unhappy inhibitions (3) Anomalous selectors Shantanu Goel (is he on this mailing list?) had agreed to write for us a general-purpose disk partition reading library, which could be used in the boot loader, possibly the kernel (unless we rip out partitioning from Mach altogether finally), and in the Hurd disk-device io server. He also planned to write a very general `fdisk' or `disklabel' like program to handle editting partition tables (in all formats, with arbitrary nesting). Of course, Shantanu is very busy and I don't think he has gone very far with this work as yet (I have not heard from him in a while). Perhaps this can leverage the freebsd work. I just thought you, Erich, should be aware of Goel's work in this area so you can work together on it. Roland From erich@uruk.org Mon May 8 22:58:22 1995 id WAA03514; Mon, 8 May 1995 22:58:20 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Roland McGrath cc: multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu Subject: Re: Multiboot Proposal 0.2 In-reply-to: Your message of "Tue, 09 May 1995 00:17:20 EDT." <199505090417.AAA02161@churchy.gnu.ai.mit.edu> Date: Mon, 08 May 1995 21:53:02 -0700 Sender: erich@uruk.org Roland McGrath writes: > Shantanu Goel (is he on this mailing list?) had agreed to write for us a > general-purpose disk partition reading library, which could be used in the > boot loader, possibly the kernel (unless we rip out partitioning from Mach > altogether finally), and in the Hurd disk-device io server. I had started on this when I realized that someone else would be working on it, and remembered the FreeBSD slice code. I like their model except for how extended DOS partitions are dealt with. > He also > planned to write a very general `fdisk' or `disklabel' like program to > handle editting partition tables (in all formats, with arbitrary nesting). I realized it would have to be done, but hadn't gone that far yet. FreeBSD's "fdisk" is decent, but it is really oriented on FreeBSD slices, and not very user-friendly (having built-in partition-type menus would be nice, for example). I'd say junk "disklabel" and make it part of "fdisk", at the very least. Having it separate will lead to confusion, and since reading arbitrary partitions is the goal, then a BSD FFS partitioned slice is only one possible configuration of many. > Of course, Shantanu is very busy and I don't think he has gone very far > with this work as yet (I have not heard from him in a while). Perhaps this > can leverage the freebsd work. I just thought you, Erich, should be aware > of Goel's work in this area so you can work together on it. Thanks for the info. (Shantanu, shall we discuss this?) -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From phk@ref.tfs.com Mon May 8 23:41:38 1995 id XAA03550; Mon, 8 May 1995 23:41:37 -0600 From: Poul-Henning Kamp Subject: Re: Multiboot Proposal 0.2 To: roland@gnu.ai.mit.edu (Roland McGrath) Date: Mon, 8 May 1995 22:34:50 -0700 (PDT) Cc: erich@uruk.org, multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu In-Reply-To: <199505090417.AAA02161@churchy.gnu.ai.mit.edu> from "Roland McGrath" at May 9, 95 00:17:20 am Content-Type: text Content-Length: 1095 > Shantanu Goel (is he on this mailing list?) had agreed to write for us a > general-purpose disk partition reading library, which could be used in the > boot loader, possibly the kernel (unless we rip out partitioning from Mach > altogether finally), and in the Hurd disk-device io server. He also > planned to write a very general `fdisk' or `disklabel' like program to > handle editting partition tables (in all formats, with arbitrary nesting). > > Of course, Shantanu is very busy and I don't think he has gone very far > with this work as yet (I have not heard from him in a while). Perhaps this > can leverage the freebsd work. I just thought you, Erich, should be aware > of Goel's work in this area so you can work together on it. I suggest you jump the freebsd bandwagon here, I belive we have got it right. Talk to Julian Elisher (julian@tfs.com) who has some plans to add Mach 2.5 support to it. -- Poul-Henning Kamp -- TRW Financial Systems, Inc. 'All relevant people are pertinent' && 'All rude people are impertinent' => 'no rude people are relevant' From phk@ref.tfs.com Mon May 8 23:52:51 1995 id XAA03565; Mon, 8 May 1995 23:52:50 -0600 From: Poul-Henning Kamp Subject: Re: Multiboot Proposal 0.2 To: erich@uruk.org Date: Mon, 8 May 1995 22:46:06 -0700 (PDT) Cc: roland@gnu.ai.mit.edu, multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu In-Reply-To: <199505090453.VAA28533@uruk.org> from "erich@uruk.org" at May 8, 95 09:53:02 pm Content-Type: text Content-Length: 1415 > I had started on this when I realized that someone else would be working > on it, and remembered the FreeBSD slice code. I like their model except > for how extended DOS partitions are dealt with. Which I hope I have now explained to you why it's the correct way of doing it... :-) > > He also > > planned to write a very general `fdisk' or `disklabel' like program to > > handle editting partition tables (in all formats, with arbitrary nesting). > > I realized it would have to be done, but hadn't gone that far yet. > > FreeBSD's "fdisk" is decent, but it is really oriented on FreeBSD slices, > and not very user-friendly (having built-in partition-type menus would > be nice, for example). I'd say junk "disklabel" and make it part of > "fdisk", at the very least. Having it separate will lead to confusion, > and since reading arbitrary partitions is the goal, then a BSD FFS > partitioned slice is only one possible configuration of many. I just made the first version of a library to deal with both level in the FreeBSD installation. You can find it in FreeBSD-current:src/release/libdisk It may or may not be useful, it wasn't written to be that general, but it actually is very general when it comes down to it. -- Poul-Henning Kamp -- TRW Financial Systems, Inc. 'All relevant people are pertinent' && 'All rude people are impertinent' => 'no rude people are relevant' From <@punt.demon.co.uk:dave@humbug.demon.co.uk> Tue May 9 03:20:33 1995 id DAA11039; Tue, 9 May 1995 03:20:32 -0600 9 May 95 9:55 GMT-60:00 9 May 95 9:54 GMT-60:00 id m0s8kYX-0003EzC; Tue, 9 May 95 09:22 BST From: Dave Hudson Subject: Re: Multiboot Proposal 0.2 To: Roland McGrath Date: Tue, 9 May 1995 09:22:13 +0100 (BST) Cc: multiboot@schirf.cs.utah.edu In-Reply-To: <199505090417.AAA02161@churchy.gnu.ai.mit.edu> from "Roland McGrath" at May 9, 95 00:17:20 am X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 1217 Roland McGrath wrote: > > Shantanu Goel (is he on this mailing list?) had agreed to write for us a > general-purpose disk partition reading library, which could be used in the > boot loader, possibly the kernel (unless we rip out partitioning from Mach > altogether finally), and in the Hurd disk-device io server. He also > planned to write a very general `fdisk' or `disklabel' like program to > handle editting partition tables (in all formats, with arbitrary nesting). I don't know if it's much help, but we've had a pretty simple disk partition handling library under VSTa since last year that we use for our IDE and SCSI disk device servers. It doesn't handle *BSD partitioning, but it will pick up the Linux/DOS extended partitioning quite nicely. Perhaps more importantly however, our lib doesn't attempt to do any of the device I/O to read the partition info sectors itself - it simply returns details of any sectors it need back to the caller so that the caller can go and fetch the sectors from the disk. The advantage in out case has been that the two places we currently use the library both request the sectors from the disk at I/O level, and thus in quite different ways. Regards, Dave From roland@gnu.ai.mit.edu Tue May 9 03:47:34 1995 id DAA11060; Tue, 9 May 1995 03:47:33 -0600 Date: Tue, 9 May 1995 05:40:44 -0400 From: Roland McGrath To: dave@humbug.demon.co.uk Cc: multiboot@schirf.cs.utah.edu cc: Shantanu Goel Subject: Re: Multiboot Proposal 0.2 In-Reply-To: Dave Hudson's message of Tue, 9 May 1995 09:22:13 +0100 X-Nsa-Fodder: Panama cracking plutonium Ortega SDI BATF Croatian > I don't know if it's much help, but we've had a pretty simple disk partition > handling library under VSTa since last year that we use for our IDE and SCSI > disk device servers. It doesn't handle *BSD partitioning, but it will pick > up the Linux/DOS extended partitioning quite nicely. > > Perhaps more importantly however, our lib doesn't attempt to do any of the > device I/O to read the partition info sectors itself - it simply returns > details of any sectors it need back to the caller so that the caller can go > and fetch the sectors from the disk. The advantage in out case has been > that the two places we currently use the library both request the sectors > from the disk at I/O level, and thus in quite different ways. Such a general library is exactly what I have spoken about with Shantanu Goel. It should be very generic, assuming nothing about disk io; you pass it callback functions to do the disk io for reading the partition table, and it passes back the locations of the partitions. It sounds like there are several versions of similar things floating around, each handling a subset of all the extant partition types. This should be a good basis for Goel's work, which will be much more general, handling partition types from every machine and OS we can think of, and in either byte order. From erich@uruk.org Tue May 9 08:42:04 1995 id IAA11451; Tue, 9 May 1995 08:42:02 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Roland McGrath cc: multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu Subject: Partition types. In-reply-to: Your message of "Tue, 09 May 1995 05:40:44 EDT." <199505090940.FAA03718@churchy.gnu.ai.mit.edu> Date: Tue, 09 May 1995 07:36:45 -0700 Sender: erich@uruk.org Roland McGrath writes: ...[comments on VSTa library from Dave Hudson deleted]... > Such a general library is exactly what I have spoken about with Shantanu > Goel. It should be very generic, assuming nothing about disk io; you pass > it callback functions to do the disk io for reading the partition table, > and it passes back the locations of the partitions. > > It sounds like there are several versions of similar things floating > around, each handling a subset of all the extant partition types. This > should be a good basis for Goel's work, which will be much more general, > handling partition types from every machine and OS we can think of, and in > either byte order. Most of these "partition libraries" are for handling Intel-order DOS/PC BIOS compatible partitions. I'd love to see such a generic system, though. I presume that free OS's ported to other architectures have had partition libraries written to handle their methodologies as well. The real question from there is if there exists a guaranteed detection algorithm for each one. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From erich@uruk.org Tue May 9 13:39:30 1995 id NAA11754; Tue, 9 May 1995 13:39:27 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Shantanu Goel cc: multiboot@schirf.cs.utah.edu Subject: Re: Multiboot Proposal 0.2 In-reply-to: Your message of "Tue, 09 May 1995 03:36:46 EDT." <199505090736.DAA01864@monk.mcl.cs.columbia.edu> Date: Tue, 09 May 1995 12:34:12 -0700 Sender: erich@uruk.org Shantanu Goel writes: > I wasn't aware of this list. I'll get myself on it. I forget if you can add yourself automatically, but Bryan Ford can either add you or tell you how to get on. > >> Of course, Shantanu is very busy and I don't think he has gone > >> very far with this work as yet (I have not heard from him in a > >> while). Perhaps this can leverage the freebsd work. I just > >> thought you, Erich, should be aware of Goel's work in this area so > >> you can work together on it. > > I haven't had a chance to work on it since I spoke to you last. > I've been finishing up the Linux driver stuff. That work is > almost complete. The only thing remaining is integrating it into > patchlevel 13. Unfortunately, loading a kernel with all the Linux > drivers triggers a bug in the new boot loader causing it to crash. Great to hear of the progress! > Finals have intervened, so I haven't gotten around to fixing this bug. > I plan to fix it and release the Linux stuff after finals are over. > Then, I'm going work on the partitioning code. > Again, apologies for the delay. When would that be? Another week or two? I need the partition info stuff pretty soon (basically now). I've read the various documents on boot-loaders, and given what I've seen in them it should be easy to write a basic "read-only" version, and it's what I've already started on. I'm also looking at the "libdisk" package that Poul-Henning Kemp placed on the FreeBSD site. Since boot-loaders don't really have a heap for malloc and such, a simple, very small library is all I need. The same thing is true of various filesystems, what a boot-loader needs is a simple read-only interface. The code in even the Mach4 bootstrap task for each of its filesystems is pretty big compared to what's in the FreeBSD bootloader today. Something else I'd prefer to move toward is always using linear addressing. The DOS partition tables have entries for linear addresses, and as long as a consistent mapping from the linear down to the Cylinder, Head, Sector addresses are done, it is OK. From what I've read, the only problems come with *some* IDE drives on *some* BIOSes. SCSI drives are generally OK. The various upper-level boot-loader and filesystem codes I've seen so far already use linear addressing, so I don't see how it would cause more problems to carry the trend downto even the MBR. As Roland said, it would be REALLY nice to be able to move disks between machines and not have to worry about which specific mapping was used. (I'm thinking mainly of differences in SCSI controller automatic remapping schemes... it would mean that the "real OS" would work OK, but DOS and such might have a fit since the partition sizes looked like they were out of alignment). This brings up a serious question, does the DOS FDISK and other ones update the linear address fields of the partition tables? It seems like something of a pain maintaining the multiple table entries, but it could be used to ensure operation under all conditions. If the partition code can't find the linear addresses, it would fall back on the BIOS geometry addresses. Both could be maintained automatically, perhaps even supporting a "translation" feature that would re-align BIOS geometry partitions according to the linear addresses (when moved to a different system that changes the BIOS geometry), or the other way around to "stamp" the correct linear addresses onto a new disk. The clever part about this feature is that you could even move a DOS disk to a new controller after you've used the re-aligning facility. The idea of re-partitioning/etc just for a new controller is rather annoying! The more general case of other architecture partitioning schemes won't be dealt with in this version of the boot-loader, though it could be added later, if people want it. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From shap@viper.cis.upenn.edu Tue May 9 13:55:42 1995 id NAA11804; Tue, 9 May 1995 13:55:41 -0600 (1.36.108.7/16.2) id AA08891; Tue, 9 May 1995 15:50:21 -0400 Date: Tue, 9 May 1995 15:50:21 -0400 From: shap@viper.cis.upenn.edu (Jonathan Shapiro) To: erich@uruk.org Cc: goel@cs.columbia.edu, multiboot@schirf.cs.utah.edu In-Reply-To: <199505091934.MAA00224@uruk.org> (erich@uruk.org) Subject: Re: Multiboot Proposal 0.2 Something else I'd prefer to move toward is always using linear addressing. The DOS partition tables have entries for linear addresses, and as long as a consistent mapping from the linear down to the Cylinder, Head, Sector addresses are done, it is OK. From what I've read, the only problems come with *some* IDE drives on *some* BIOSes. SCSI drives are generally OK. I don't know if what follows is a problem in practice or not, but it may account for the BIOS weirdness. The IDE spec allows drives to implement different numbers of sectors per track at different places on the disk and do sector remapping. If the OS uses translated sectors (as most do), this is all transparent, but if it uses physical sectors I do not know what it looks like to the software. While all of this *ought* to be transparent to you, you can bet that the controllers only get right that portion of the spec that is exercised by current operating systems. I also remember something about there being a version change in the IDE spec at some point that was intended to support very large drives. I have a book to recommend, but rather than post the wrong info I'll send the title/ISBN out from home tonight. I recently went through the bootstrap construction process with EROS. The problem is simplified a bit by the fact that EROS takes a fairly low level view of the disk (no file system at all). One thing I decided fairly early in the game was that the DOS partitioning logic is something I ought to just view as machine dependent. After a lot of discussion, Bryan Ford eventually convinced me to use the BIOS on the PC to do the I/O. For the bootstrap code, there isn't much of an advantage to doing otherwise, but it does force you to have a linear to hd/cyl/sec mapper somewhere in your bootstrap code. Given the number of portability problems that people have encountered with the bootstrap on DOS machines, using the BIOS seems compelling. Actually, I'm rapidly getting to the point where I'm ready to have the OS use the BIOS as well - disk drivers are proving to be a serious pain in the ass. Jonathan From shap@viper.cis.upenn.edu Tue May 9 14:05:48 1995 id OAA11870; Tue, 9 May 1995 14:05:43 -0600 (1.36.108.7/16.2) id AA08893; Tue, 9 May 1995 16:00:31 -0400 Date: Tue, 9 May 1995 16:00:31 -0400 From: shap@viper.cis.upenn.edu (Jonathan Shapiro) To: erich@uruk.org Cc: goel@cs.columbia.edu, multiboot@schirf.cs.utah.edu In-Reply-To: <199505091934.MAA00224@uruk.org> (erich@uruk.org) Subject: Re: Multiboot Proposal 0.2 This brings up a serious question, does the DOS FDISK and other ones update the linear address fields of the partition tables? It seems like something of a pain maintaining the multiple table entries, but it could be used to ensure operation under all conditions. If the partition code can't find the linear addresses, it would fall back on the BIOS geometry addresses. There are some gotchas here: 1. Some older fdisk variants are not good citizens about the linear sector number in the partition table. I don't know if any such are still in use out there (I kind of doubt it). I used to have problems with SVR3 from AT&T. Up to a point you can take the view that it's not your problem if the other guy doesn't meet the spec. 2. Some fdisk variants actually check that the linear and hd/cyl/sec values are consistent, and happily tell you they cannot figure out what's going on if they aren't - I think the idea is not to make a suspicious disk any worse. In any case, failure to properly set the field really does violate the spec. 3. The linear addresses written to the boot sector will be logical linear addresses (i.e. linear address before remapping) rather than physical linear addresses. By this I mean that code which does the proper math to convert hd/cyl/sec to linear sector number should arrive at the same number that is stored in the partition entry. Per the above, there are programs out there that actually do check. So the problem in general is that you can't safely reuse that field, and storing your info anywhere else will cause it to become out of date whenever someone else's fdisk variant is run. And yes, I agree completely that the whole thing is a mess. But I was under the impression that there were only a couple of translation schemes in use out there, and that there actually was a convention among the manufacturers concerning this. Is this wrong? From erich@uruk.org Tue May 9 15:11:27 1995 id PAA11972; Tue, 9 May 1995 15:11:24 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: shap@viper.cis.upenn.edu (Jonathan Shapiro) cc: multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu Subject: Re: Multiboot Proposal 0.2 In-reply-to: Your message of "Tue, 09 May 1995 15:50:21 EDT." <199505091955.NAA11804@schirf.cs.utah.edu> Date: Tue, 09 May 1995 14:06:10 -0700 Sender: erich@uruk.org shap@viper.cis.upenn.edu (Jonathan Shapiro) writes (message #1): ...[my comment on preferring linear addressing deleted]... > I don't know if what follows is a problem in practice or not, but it > may account for the BIOS weirdness. > > The IDE spec allows drives to implement different numbers of sectors > per track at different places on the disk and do sector remapping. If > the OS uses translated sectors (as most do), this is all transparent, > but if it uses physical sectors I do not know what it looks like to > the software. > > While all of this *ought* to be transparent to you, you can bet that > the controllers only get right that portion of the spec that is > exercised by current operating systems. ??? I'm not sure I see how my using linear addressing will cause this problem. I'm planning on using linear addressing by converting between a logical block number (starting at 0, and represented by a long) and cylinder/head/sector addresses as used by the int 0x13, %ah=0x0 BIOS call. If you query the BIOS geometry with the int 0x13, %ah=0x8 BIOS call, it will give you a geometry compatible with a linear address calculated in the fashion (since the calculation is fully reversible). > I also remember something about there being a version change in the > IDE spec at some point that was intended to support very large drives. That interface is taken care of below the level of the int 0x13, %ah=0x0 BIOS call, according to the documents that I've looked at so far. The translation I will provide is simply a layer on top of the logically provided cylinder/head/sector scheme. > I have a book to recommend, but rather than post the wrong info I'll > send the title/ISBN out from home tonight. I've only recently (read this morning) got good documentation on the topic, but it seems consistent with what I've read so far. The big hole in my understanding was extended partitions, which has been greatly clarified. I'll attach the info that Poul-Henning Kamp sent me at the end of the message. > I recently went through the bootstrap construction process with EROS. > The problem is simplified a bit by the fact that EROS takes a fairly > low level view of the disk (no file system at all). One thing I > decided fairly early in the game was that the DOS partitioning logic > is something I ought to just view as machine dependent. Well, given some attention to detail (i.e. it wouldn't necessarily be automatic when moving between machines), even this could be overcome on free OS's (DOS, etc. would need us to perform a translation and write the different partition C/H/S structures back to disk) for those cases. > After a lot of discussion, Bryan Ford eventually convinced me to use > the BIOS on the PC to do the I/O. For the bootstrap code, there isn't > much of an advantage to doing otherwise, but it does force you to have > a linear to hd/cyl/sec mapper somewhere in your bootstrap code. Given > the number of portability problems that people have encountered with > the bootstrap on DOS machines, using the BIOS seems compelling. I am using the BIOS calls for I/O in the boot-loader. The only deal is trying to account for different disk controllers particular geometry mappings. > Actually, I'm rapidly getting to the point where I'm ready to have the > OS use the BIOS as well - disk drivers are proving to be a serious > pain in the ass. While I think it would be useful to have a "BIOS driver" that allows controllers without drivers to be used, performance is greatly improved by a native driver... but I won't argue the details since I know of others much more zealous than myself about it, and this isn't the forum for such a discussion. shap@viper.cis.upenn.edu (Jonathan Shapiro) writes (message #2): ...[my question about the consistency of DOS FDISK in supporting linear addressing in partition tables deleted]... > There are some gotchas here: > > 1. Some older fdisk variants are not good citizens about the linear > sector number in the partition table. I don't know if any such are > still in use out there (I kind of doubt it). I used to have > problems with SVR3 from AT&T. Up to a point you can take the view > that it's not your problem if the other guy doesn't meet the spec. I was envisioning a translation program that could partially diagnose and apply corrective action (enforced by some manual controls when necessary, and you know the diagnosis routine doesn't work right) in such cases. > 2. Some fdisk variants actually check that the linear and hd/cyl/sec > values are consistent, and happily tell you they cannot figure out > what's going on if they aren't - I think the idea is not to make a > suspicious disk any worse. In any case, failure to properly set > the field really does violate the spec. OK. That's what I wanted to take advantage of in the first place. > 3. The linear addresses written to the boot sector will be logical > linear addresses (i.e. linear address before remapping) rather than > physical linear addresses. By this I mean that code which does the > proper math to convert hd/cyl/sec to linear sector number should > arrive at the same number that is stored in the partition entry. > Per the above, there are programs out there that actually do check. This also follows what I was considering. > So the problem in general is that you can't safely reuse that field, > and storing your info anywhere else will cause it to become out of > date whenever someone else's fdisk variant is run. ??? I wasn't planning on adding any extra data, the main use I had in mind was for the filesystem, etc. code on the free OS's to be capable of running when the numbers don't match up, so you could at least boot your system or interpret a disk correctly even if DOS would barf all over the place. The tool mentioned above could "re-align" the parameters for you, again always relying on the linear addresses, which are somewhat independent of the controller. The really nice thing about this is that using a 32-bit value for the start/size of a partition allows the higher level drivers that can access outside of the BIOS geometry limitations the capability to deal with the partitions in that manner consistently. The limitation would be more like 2 terabytes (2^32 512 byte blocks) in this case. This will still run out, but it gives a lot more headroom for now. > But I was under the impression that there were only a couple of > translation schemes in use out there, and that there actually was a > convention among the manufacturers concerning this. Is this wrong? Yes, there is a convention, but there are a few variants. For example, on SCSI controllers, they usually have a "standard translation mode" with up to 1024 cylinders, 64 heads, and 32 sectors (I think you're for various reasons it's best not to use the last cylinder, so say 1023 max cylinders). Offhand, I might have been wrong and it was 32 heads and 63 sectors (not 64, 63 is the largest sector number). Anyway, this allows a maximum BIOS addressible size of approx 1GB. Your OS specific driver can go farther, but you're limited to that with basic DOS and the bootloaders. The problem is that several SCSI controllers have their own "extended translation modes", and these lead to situations where if you change the translation mode or move it to another controller where the translation mode isn't available, you're hosed and you have to repartition. What I'm suggesting is a way, at least for free OS's, to be mainly compatible, but allow a way out, even the opportunity to convert the partition C/H/S numbers in the event you want to run DOS/etc. on it. At this stage, there doesn't appear to be a really good reason not to go this route, so I'm going to go for the linear address/geometry calculation for my bootloader. As I said, all the other ones I've come across that work well are basically performing the same job, simply not starting at the level of the stage 1 boot-loader. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From erich@uruk.org Tue May 9 15:18:23 1995 id PAA11989; Tue, 9 May 1995 15:17:40 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: shap@viper.cis.upenn.edu (Jonathan Shapiro) cc: multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu Subject: Partitioning and linear addressing In-reply-to: Your message of "Tue, 09 May 1995 16:00:31 EDT." <199505092000.NAA00278@uruk.org> Date: Tue, 09 May 1995 14:12:07 -0700 Sender: erich@uruk.org Whoops, forgot to include the docs Poul-Henning Kamp sent me. -----------------------(start docs here)------------------------- How It Works -- CHS Translation Plus BIOS Types, LBA and Other Good Stuff Version 4a by Hale Landis (landis@sugs.tware.com) THE "HOW IT WORKS" SERIES This is one of several How It Works documents. The series currently includes the following: * How It Works -- CHS Translation * How It Works -- Master Boot Record * How It Works -- DOS Floppy Boot Sector * How It Works -- OS2 Boot Sector * How It Works -- Partition Tables Introduction (READ THIS!) ------------------------- This is very technical. Please read carefully. There is lots of information here that can sound confusing the first time you read it. Why is an understanding of how a BIOS works so important? The basic reason is that the information returned by INT 13H AH=08H is used by FDISK, it is used in the partition table entries within a partition record (like the Master Boot Record) that are created by FDISK, and it is used by the small boot program that FDISK places into the Master Boot Record. The information returned by INT 13H AH=08H is in cylinder/head/sector (CHS) format -- it is not in LBA format. The boot processing done by your computer's BIOS (INT 19H and INT 13H) is all CHS based. Read this so that you are not confused by all the false information going around that says "LBA solves the >528MB problem". Read this so that you understand the possible data integrity problem that a WD EIDE type BIOS creates. Any BIOS that has a "LBA mode" in the BIOS setup could be a WD EIDE BIOS. Be very careful and NEVER chage the "LBA mode" setting after you have partitioned and installed your software. History ------- Changes between this version and the preceeding version are marked by "!" at left margin of the first line of a changed or new paragraph. Version 4 -- BIOS Types 8 and 10 updated. Version 3 -- New BIOS types found and added to this list. More detailed information is listed for each BIOS type. A section describing CHS translation was added. Version 2 -- A rewrite of version 1 adding BIOS types not included in version 1. Version 1 -- First attempt to classify the BIOS types and describe what each does or does not do. Definitions ----------- * 528MB - The maximun drive capacity that is supported by 1024 cylinders, 16 heads and 63 sectors (1024x16x63x512). This is the limit for CHS addressing in the original IBM PC/XT and IBM PC/AT INT 13H BIOS. * 8GB - The maximum drive capacity that can be supported by 1024 cylinders, 256 heads and 63 sectors (1024x256x63x512). This is the limit for the BIOS INT 13H AH=0xH calls. * ATA - AT Attachment -- The real name of what is widely known as IDE. * CE Cylinder - Customer Engineering cylinder. This is the last cylinder in P-CHS mode. IBM has always reserved this cylinder for use of disk diagnostic programs. Many BIOS do not account for it correctly. It is of questionable value these days and probably should be considered obsolete. However, since there is no industry wide agreement, beware. There is no CE Cylinder reserved in the L-CHS address. Also beware of diagnostic programs that don't realize they are operating in L-CHS mode and think that the last L-CHS cylinder is the CE Cylinder. * CHS - Cylinder/Head/Sector. This is the traditional way to address sectors on a disk. There are at least two types of CHS addressing: the CHS that is used at the INT 13H interface and the CHS that is used at the ATA device interface. In the MFM/RLL/ESDI and early ATA days the CHS used at the INT 13H interface was the same as the CHS used at the device interface. Today we have CHS translating BIOS types that can use one CHS at the INT 13H interface and a different CHS at the device interface. These two types of CHS will be called the logical CHS or L-CHS and the physical CHS or P-CHS in this document. L-CHS is the CHS used at the INT 13H interface and P-CHS is the CHS used at the device interface. The L-CHS used at the INT 13 interface allows up to 256 heads, up to 1024 cylinders and up to 63 sectors. This allows support of up to 8GB drives. This scheme started with either ESDI or SCSI adapters many years ago. The P-CHS used at the device interface allows up to 16 heads up to 65535 cylinders, and up to 63 sectors. This allows access to 2^28 sectors (136GB) on an ATA device. When a P-CHS is used at the INT 13H interface it is limited to 1024 cylinders, 16 heads and 63 sectors. This is where the old 528MB limit originated. ATA devices may also support LBA at the device interface. LBA allows access to approximately 2^28 sectors (137GB) on an ATA device. A SCSI host adapter can convert a L-CHS directly to an LBA used in the SCSI read/write commands. On a PC today, SCSI is also limited to 8GB when CHS addressing is used at the INT 13H interface. * EDPT - Enhanced fixed Disk Parameter Table -- This table returns additional information for BIOS drive numbers 80H and 81H. The EDPT for BIOS drive 80H is pointed to by INT 41H. The EDPT for BIOS drive 81H is pointed to by INT 46H. The EDPT is a fixed disk parameter table with an AxH signature byte. This table format returns two sets of CHS information. One set is the L-CHS and is probably the same as returned by INT 13H AH=08H. The other set is the P-CHS used at the drive interface. This type of table allows drives with >1024 cylinders or drives >528MB to be supported. The translated CHS will have <=1024 cylinders and (probably) >16 heads. The CHS used at the drive interface will have >1024 cylinders and <=16 heads. It is unclear how the IBM defined CE cylinder is accounted for in such a table. Compaq probably gets the credit for the original definition of this type of table. * FDPT - Fixed Disk Parameter Table - This table returns additional information for BIOS drive numbers 80H and 81H. The FDPT for BIOS drive 80H is pointed to by INT 41H. The FDPT for BIOS drive 81H is pointed to by INT 46H. A FDPT does not have a AxH signature byte. This table format returns a single set of CHS information. The L-CHS information returned by this table is probably the same as the P-CHS and is also probably the same as the L-CHS returned by INT 13H AH=08H. However, not all BIOS properly account for the IBM defined CE cylinder and this can cause a one or two cylinder difference between the number of cylinders returned in the AH=08H data and the FDPT data. IBM gets the credit for the original definition of this type of table. * LBA - Logical Block Address. Another way of addressing sectors that uses a simple numbering scheme starting with zero as the address of the first sector on a device. The ATA standard requires that cylinder 0, head 0, sector 1 address the same sector as addressed by LBA 0. LBA addressing can be used at the ATA interface if the ATA device supports it. LBA addressing is also used at the INT 13H interface by the AH=4xH read/write calls. * L-CHS -- Logical CHS. The CHS used at the INT 13H interface by the AH=0xH calls. See CHS above. * MBR - Master Boot Record (also known as a partition table) - The sector located at cylinder 0 head 0 sector 1 (or LBA 0). This sector is created by an "FDISK" utility program. The MBR may be the only partition table sector or the MBR can be the first of multiple partition table sectors that form a linked list. A partition table entry can describe the starting and ending sector addresses of a partition (also known as a logical volume or a logical drive) in both L-CHS and LBA form. Partition table entries use the L-CHS returned by INT 13H AH=08H. Older FDISK programs may not compute valid LBA values. * OS - Operating System. * P-CHS -- Physical CHS. The CHS used at the ATA device interface. This CHS is also used at the INT 13H interface by older BIOS's that do not support >1024 cylinders or >528MB. See CHS above. Background and Assumptions -------------------------- First, please note that this is written with the OS implementor in mind and that I am talking about the possible BIOS types as seen by an OS during its hardware configuration search. It is very important that you not be confused by all the misinformation going around these days. All OS's that want to be co-resident with another OS (and that is all of the PC based OS's that I know of) MUST use INT 13H to determine the capacity of a hard disk. And that capacity information MUST be determined in L-CHS mode. Why is this? Because: 1) FDISK and the partition tables are really L-CHS based, and 2) MS/PC DOS uses INT 13H AH=02H and AH=03H to read and write the disk and these BIOS calls are L-CHS based. The boot processing done by the BIOS is all L-CHS based. During the boot processing, all of the disk read accesses are done in L-CHS mode via INT 13H and this includes loading the first of the OS's kernel code or boot manager's code. Second, because there can be multiple BIOS types in any one system, each drive may be under the control of a different type of BIOS. For example, drive 80H (the first hard drive) could be controlled by the original system BIOS, drive 81H (the second drive) could be controlled by a option ROM BIOS and drive 82H (the third drive) could be controlled by a software driver. Also, be aware that each drive could be a different type, for example, drive 80H could be an MFM drive, drive 81H could be an ATA drive, drive 82H could be a SCSI drive. Third, not all OS's understand or use BIOS drive numbers greater than 81H. Even if there is INT 13H support for drives 82H or greater, the OS may not use that support. Fourth, the BIOS INT 13H configuration calls are: * AH=08H, Get Drive Parameters -- This call is restricted to drives up to 528MB without CHS translation and to drives up to 8GB with CHS translation. For older BIOS with no support for >1024 cylinders or >528MB, this call returns the same CHS as is used at the ATA interface (the P-CHS). For newer BIOS's that do support >1024 cylinders or >528MB, this call returns a translated CHS (the L-CHS). The CHS returned by this call is used by FDISK to build partition records. * AH=41H, Get BIOS Extensions Support -- This call is used to determine if the IBM/Microsoft Extensions or if the Phoenix Enhanced INT 13H calls are supported for the BIOS drive number. * AH=48H, Extended Get Drive Parameters -- This call is used to determine the CHS geometries, LBA information and other data about the BIOS drive number. * the FDPT or EDPT -- While not actually a call, but instead a data area, the FDPT or EDPT can return additional information about a drive. * other tables -- The IBM/Microsoft extensions provide a pointer to a drive parameter table via INT 13H AH=48H. The Phoenix enhancement provides a pointer to a drive parameter table extension via INT 13H AH=48H. These tables are NOT the same as the FDPT or EDPT. Note: The INT 13H AH=4xH calls duplicate the older AH=0xH calls but use a different parameter passing structure. This new structure allows support of drives with up to 2^64 sectors (really BIG drives). While at the INT 13H interface the AH=4xH calls are LBA based, these calls do NOT require that the drive support LBA addressing. CHS Translation Algorithms -------------------------- NOTE: Before you send me email about this, read this entire section. Thanks! As you read this, don't forget that all of the boot processing done by the system BIOS via INT 19H and INT 13H use only the INT 13H AH=0xH calls and that all of this processing is done in CHS mode. First, lets review all the different ways a BIOS can be called to perform read/write operations and the conversions that a BIOS must support. ! * An old BIOS (like BIOS type 1 below) does no CHS translation and does not use LBA. It only supports the AH=0xH calls: INT 13H (L-CHS == P-CHS) ATA AH=0xH --------------------------------> device (L-CHS) (P-CHS) * A newer BIOS may support CHS translation and it may support LBA at the ATA interface: INT 13H L-CHS ATA AH=0xH --+--> to --+----------------> device (L-CHS) | P-CHS | (P-CHS) | | | | P-CHS | +--> to --+ | LBA | | | | L-CHS | ATA +--> to -----------------+---> device LBA (LBA) * A really new BIOS may also support the AH=4xH in addtion to the older AH\0xH calls. This BIOS must support all possible combinations of CHS and LBA at both the INT 13H and ATA interfaces: INT 13H ATA AH=4xH --+-----------------------------> device (LBA) | (LBA) | | LBA +--> to ---------------+ P-CHS | | INT 13H L-CHS | ATA AH=0xH --+--> to --+------------+---> device (L-CHS) | P-CHS | (P-CHS) | | | | P-CHS | +--> to --+ | LBA | | | | L-CHS | ATA +--> to -----------------+---> device LBA (LBA) You would think there is only one L-CHS to P-CHS translation algorithm, only one L-CHS to LBA translation algorithm and only one P-CHS to LBA translation algorithm. But this is not so. Why? Because there is no document that standardizes such an algorithm. You can not rely on all BIOS's and OS's to do these translations the same way. The following explains what is widely accepted as the "correct" algorithms. An ATA disk must implement both CHS and LBA addressing and must at any given time support only one P-CHS at the device interface. And, the drive must maintain a strick relationship between the sector addressing in CHS mode and LBA mode. Quoting the ATA-2 document: LBA = ( (cylinder * heads_per_cylinder + heads ) * sectors_per_track ) + sector - 1 where heads_per_cylinder and sectors_per_track are the current translation mode values. This algorithm can also be used by a BIOS or an OS to convert a L-CHS to an LBA as we'll see below. This algorithm can be reversed such that an LBA can be converted to a CHS: cylinder = LBA / (heads_per_cylinder * sectors_per_track) temp = LBA % (heads_per_cylinder * sectors_per_track) head = temp / sectors_per_track sector = temp % sectors_per_track + 1 While most OS's compute disk addresses in an LBA scheme, an OS like DOS must convert that LBA to a CHS in order to call INT 13H. Technically an INT 13H should follow this process when converting an L-CHS to a P-CHS: 1) convert the L-CHS to an LBA, 2) convert the LBA to a P-CHS, If an LBA is required at the ATA interface, then this third step is needed: 3) convert the P-CHS to an LBA. All of these conversions are done by normal arithmetic. However, while this is the technically correct way to do things, certain short cuts can be taken. It is possible to convert an L-CHS directly to a P-CHS using bit a bit shifting algorithm. This combines steps 1 and 2. And, if the ATA device being used supports LBA, steps 2 and 3 are not needed. The LBA value produced in step 1 is the same as the LBA value produced in step 3. AN EXAMPLE Lets look at an example. Lets say that the L-CHS is 1000 cylinders 10 heads and 50 sectors, the P-CHS is 2000 cylinders, 5 heads and 50 sectors. Lets say we want to access the sector at L-CHS 2,4,3. * step 1 converts the L-CHS to an LBA, lba = 1202 = ( ( 2 * 10 + 4 ) * 50 ) + 3 - 1 * step 2 converts the LBA to the P-CHS, cylinder = 4 = ( 1202 / ( 5 * 50 ) temp = 202 = ( 1202 % ( 5 * 50 ) ) head = 4 = ( 202 / 50 ) sector = 3 = ( 202 % 50 ) + 1 * step 3 converts the P-CHS to an LBA, lba = 1202 = ( ( 4 * 5 + 4 ) * 50 ) + 3 - 1 Most BIOS (or OS) software is not going to do all of this to convert an address. Most will use some other algorithm. There are many such algorithms. BIT SHIFTING INSTEAD If the L-CHS is produced from the P-CHS by 1) dividing the P-CHS cylinders by N, and 2) multiplying the P-CHS heads by N, where N is 2, 4, 8, ..., then this bit shifting algorithm can be used and N becomes a bit shift value. This is the most common way to make the P-CHS geometry of a >528MB drive fit the INT 13H L-CHS rules. Plus this algorithm maintains the same sector ordering as the more complex algorithm above. Note the following: Lcylinder = L-CHS cylinder being accessed Lhead = L-CHS head being accessed Lsector = L-CHS sector being accessed Pcylinder = the P-CHS cylinder being accessed Phead = the P-CHS head being accessed Psector = P-CHS sector being accessed NPH = is the number of heads in the P-CHS N = 2, 4, 8, ..., the bit shift value The algorithm, which can be implemented using bit shifting instead of multiply and divide operations is: Pcylinder = ( Lcylinder * N ) + ( Lhead / NPH ); Phead = ( Lhead % NPH ); Psector = Lsector; A BIT SHIFTING EXAMPLE Lets apply this to our example above (L-CHS = 1000,10,50 and P-CHS = 2000, 5, 50) and access the same sector at at L-CHS 2,4,3. Pcylinder = 4 = ( 2 * 2 ) + ( 4 / 5 ) Phead = 4 = ( 4 % 5 ) Psector = 3 = 3 As you can see, this produces the same P-CHS as the more complex method above. SO WHAT IS THE PROBLEM? The basic problem is that there is no requirement that a CHS translating BIOS followed these rules. There are many other algorithms that can be implemented to perform a similar function. Today, there are at least two popular implementions: the Phoenix implementation (described above) and the non-Phoenix implementations. SO WHY IS THIS A PROBLEM IF IT IS HIDDEN INSIDE THE BIOS? Because a protected mode OS that does not want to use INT 13H must implement the same CHS translation algorithm. If it doesn't, your data gets scrambled. WHY USE CHS AT ALL? In the perfect world of tomorrow, maybe only LBA will be used. But today we are faced with the following problems: * Some drives >528MB don't implement LBA. * Some drives are optimized for CHS and may have lower performance when given commands in LBA mode. Don't forget that LBA is something new for the ATA disk designers who have worked very hard for many years to optimize CHS address handling. And not all drive designs require the use of LBA internally. * The L-CHS to LBA conversion is more complex and slower than the bit shifting L-CHS to P-CHS conversion. * DOS, FDISK and the MBR are still CHS based -- they use the CHS returned by INT 13H AH=08H. Any OS that can be installed on the same disk with DOS must understand CHS addressing. * The BIOS boot processing and loading of the first OS kernel code is done in CHS mode -- the CHS returned by INT 13H AH=08H is used. * Microsoft has said that their OS's will not use any disk capacity that can not also be accessed by INT 13H AH=0xH. These are difficult problems to overcome in today's industry environment. The result: chaos. DANGER TO YOUR DATA! See the description of BIOS Type 7 below to understand why a WD EIDE BIOS is so dangerous to your data. The BIOS Types -------------- I assume the following: a) All BIOS INT 13H support has been installed by the time the OS starts its boot processing. I'm don't plan to cover what could happen to INT 13H once the OS starts loading its own device drivers. b) Drives supported by INT 13H are numbered sequentially starting with drive number 80H (80H-FFH are hard drives, 00-7FH are floppy drives). And remember, any time a P-CHS exists it may or may not account for the CE Cylinder properly. I have identified the following types of BIOS INT 13H support as seen by an OS during its boot time hardware configuration determination: BIOS Type 1 Origin: Original IBM PC/XT. BIOS call support: INT 13H AH=0xH and FDPT for BIOS drives 80H and 81H. There is no CHS translation. INT 13H AH=08H returns the P-CHS. The FDPT should contain the same P-CHS. Description: Supports up to 528MB from a table of drive descriptions in BIOS ROM. No support for >1024 cylinders or drives >528MB or LBA. Support issues: For >1024 cylinders or >528MB support, either an option ROM with an INT 13H replacement (see BIOS types 4-7) -or- a software driver (see BIOS type 8) must be added to the system. BIOS Type 2 Origin: Unknown, but first appeared on systems having BIOS drive type table entries defining >1024 cylinders. Rumored to have originated at the request of Novell or SCO. BIOS call support: INT 13H AH=0xH and FDPT for BIOS drives 80H and 81H. INT 13H AH=08H should return a L-CHS with the cylinder value limited to 1024. Beware, many BIOS perform a logical AND on the cylinder value. A correct BIOS will limit the cylinder value as follows: cylinder = cylinder > 1024 ? 1024 : cylinder; An incorrect BIOS will limit the cylinder value as follows (this implementation turns a 540MB drive into a 12MB drive!): cylinder = cylinder & 0x03ff; The FDPT will return a P-CHS that has the full cylinder value. Description: For BIOS drive numbers 80H and 81H, this BIOS type supports >1024 cylinders or >528MB without using a translated CHS in the FDPT. INT 13H AH=08H truncates cylinders to 1024 (beware of buggy implementations). The FDPT can show >1024 cylinders thereby allowing an OS to support drives >528MB. May convert the L-CHS or P-CHS directly to an LBA if the ATA device supports LBA. Support issues: Actual support of >1024 cylinders is OS specific -- some OS's may be able to place OS specific partitions spanning or beyond cylinder 1024. Usually all OS boot code must be within first 1024 cylinders. The FDISK program of an OS that supports such partitions uses an OS specific partition table entry format to identify these paritions. There does not appear to be a standard (de facto or otherwise) for this unusual partition table entry. Apparently one method is to place -1 into the CHS fields and use the LBA fields to describe the location of the partition. This DOES NOT require the drive to support LBA addressing. Using an LBA in the partition table entry is just a trick to get around the CHS limits in the partition table entry. It is unclear if such a partition table entry will be ignored by an OS that does not understand what it is. For an OS that does not support such partitions, either an option ROM with an INT 13H replacement (see BIOS types 4-7) -or- a software driver (see BIOS type 8) must be added to the system. Note: OS/2 can place HPFS partitions and Linux can place Linux partitions beyond or spanning cylinder 1024. (Anyone know of other systems that can do the same?) BIOS Type 3 Origin: Unknown, but first appeared on systems having BIOS drive type table entires defining >1024 cylinders. Rumored to have originated at the request of Novell or SCO. BIOS call support: INT 13H AH=0xH and FDPT for BIOS drives 80H and 81H. INT 13H AH=08H can return an L-CHS with more than 1024 cylinders. Description: This BIOS is like type 2 above but it allows up to 4096 cylinders (12 cylinder bits). It does this in the INT 13H AH=0xH calls by placing two most significant cylinder bits (bits 11 and 10) into the upper two bits of the head number (bits 7 and 6). Support issues: Identification of such a BIOS is difficult. As long as the drive(s) supported by this type of BIOS have <1024 cylinders this BIOS looks like a type 2 BIOS because INT 13H AH=08H should return zero data in bits 7 and 6 of the head information. If INT 13H AH=08H returns non zero data in bits 7 and 6 of the head information, perhaps it can be assumed that this is a type 3 BIOS. For more normal support of >1024 cylinders or >528MB, either an option ROM with an INT 13H replacement (see BIOS types 4-7) -or- a software driver (see BIOS type 8) must be added to the system. Note: Apparently this BIOS type is no longer produced by any BIOS vendor. BIOS Type 4 Origin: Compaq. Probably first appeared in systems with ESDI drives having >1024 cylinders. BIOS call support: INT 13H AH=0xH and EDPT for BIOS drives 80H and 81H. If the drive has <1024 cylinders, INT 13H AH=08H returns the P-CHS and a FDPT is built. If the drive has >1024 cylinders, INT 13H AH=08H returns an L-CHS and an EDPT is built. Description: Looks like a type 2 BIOS when an FDPT is built. Uses CHS translation when an EDPT is used. May convert the L-CHS directly to an LBA if the ATA device supports LBA. Support issues: This BIOS type may support up to four drives with a EDPT (or FDPT) for BIOS drive numbers 82H and 83H located in memory following the EDPT (or FDPT) for drive 80H. Different CHS translation algorithms may be used by the BIOS and an OS. BIOS Type 5 Origin: The IBM/Microsoft BIOS Extensions document. For many years this document was marked "confidential" so it did not get wide spread distribution. BIOS call support: INT 13H AH=0xH, AH=4xH and EDPT for BIOS drives 80H and 81H. INT 13H AH=08H returns an L-CHS. INT 13H AH=41H and AH=48H should be used to get P-CHS configuration. The FDPT/EDPT should not be used. In some implementations the FDPT/EDPT may not exist. Description: A BIOS that supports very large drives (>1024 cylinders, >528MB, actually >8GB), and supports the INT 13H AH=4xH read/write functions. The AH=4xH calls use LBA addressing and support drives with up to 2^64 sectors. These calls do NOT require that a drive support LBA at the drive interface. INT 13H AH=48H describes the L-CHS used at the INT 13 interface and the P-CHS or LBA used at the drive interface. This BIOS supports the INT 13 AH=0xH calls the same as a BIOS type 4. Support issues: While the INT 13H AH=4xH calls are well defined, they are not implemented in many systems shipping today. Currently undefined is how such a BIOS should respond to INT 13H AH=08H calls for a drive that is >8GB. Different CHS translation algorithms may be used by the BIOS and an OS. Note: Support of LBA at the drive interface may be automatic or may be under user control via a BIOS setup option. Use of LBA at the drive interface does not change the operation of the INT 13 interface. BIOS Type 6 Origin: The Phoenix Enhanced Disk Drive Specification. BIOS call support: INT 13H AH=0xH, AH=4xH and EDPT for BIOS drives 80H and 81H. INT 13H AH=08H returns an L-CHS. INT 13H AH=41H and AH=48H should be used to get P-CHS configuration. INT 13H AH=48H returns the address of the Phoenix defined "FDPT Extension" table. Description: A BIOS that supports very large drives (>1024 cylinders, >528MB, actually >8GB), and supports the INT 13H AH=4xH read/write functions. The AH=4xH calls use LBA addressing and support drives with up to 2^64 sectors. These calls do NOT require that a drive support LBA at the drive interface. INT 13H AH=48H describes the L-CHS used at the INT 13 interface and the P-CHS or LBA used at the drive interface. This BIOS supports the INT 13 AH=0xH calls the same as a BIOS type 4. The INT 13H AH=48H call returns additional information such as host adapter addresses, DMA support, LBA support, etc, in the Phoenix defined "FDPT Extension" table. Phoenix says this this BIOS need not support the INT 13H AH=4xH read/write calls but this BIOS is really an extension/enhancement of the original IBM/MS BIOS so most implementations will probably support the full set of INT 13H AH=4xH calls. Support issues: Currently undefined is how such a BIOS should respond to INT 13H AH=08H calls for a drive that is >8GB. Different CHS translation algorithms may be used by the BIOS and an OS. Note: Support of LBA at the drive interface may be automatic or may be under user control via a BIOS setup option. Use of LBA at the drive interface does not change the operation of the INT 13 interface. BIOS Type 7 Origin: Described in the Western Digital Enhanced IDE Implementation Guide. BIOS call support: INT 13H AH=0xH and FDPT or EDPT for BIOS drives 80H and 81H. An EDPT with a L-CHS of 16 heads and 63 sectors is built when "LBA mode" is enabled. An FDPT is built when "LBA mode" is disabled. Description: Supports >1024 cylinders or >528MB using a EDPT with a translated CHS *** BUT ONLY IF *** the user requests "LBA mode" in the BIOS setup *** AND *** the drive supports LBA. As long as "LBA mode" is enabled, CHS translation is enabled using a L-CHS with <=1024 cylinders, 16, 32, 64, ..., heads and 63 sectors. Disk read/write commands are issued in LBA mode at the ATA interface but other commands are issued in P-CHS mode. Because the L-CHS is determined by table lookup based on total drive capacity and not by a multiply/divide of the P-CHS cylinder and head values, it may not be possible to use the simple (and faster) bit shifting L-CHS to P-CHS algorithms. When "LBA mode" is disabled, this BIOS looks like a BIOS type 2 with an FDPT. The L-CHS used is taken either from the BIOS drive type table or from the device's Identify Device data. This L-CHS can be very different from the L-CHS returned when "LBA mode" is enabled. This BIOS may support FDPT/EDPT for up to four drives in the same manner as described in BIOS type 4. The basic problem with this BIOS is that the CHS returned by INT 13H AH=08H changes because of a change in the "LBA mode" setting in the BIOS setup. This should not happen. This use or non-use of LBA at the ATA interface should have no effect on the CHS returned by INT 13H AH=08H. This is the only BIOS type know to have this problem. Support issues: If the user changes the "LBA mode" setting in BIOS setup, INT 13H AH=08H and the FDPT/EDPT change which may cause *** DATA CORRUPTION ***. The user should be warned to not change the "LBA mode" setting in BIOS setup once the drive has been partitioned and software installed. Different CHS translation algorithms may be used by the BIOS and an OS. BIOS Type 8 Origin: Unknown. Perhaps Ontrack's Disk Manager was the first of these software drivers. Another example of such a driver is Micro House's EZ Drive. BIOS call support: Unknown (anyone care to help out here?). Mostly likely only INT 13H AH=0xH are support. Probably a FDPT or EDPT exists for drives 80H and 81H. ! Description: A software driver that "hides" in the MBR such that it is loaded into system memory before any OS boot processing starts. These drivers can have up to three parts: a part that hides in the MBR, a part that hides in the remaining sectors of cylinder 0, head 0, and an OS device driver. The part in the MBR loads the second part of the driver from cylinder 0 head 0. The second part provides a replacement for INT 13H that enables CHS translation for at least the boot drive. Usually the boot drive is defined in CMOS setup as a type 1 or 2 (5MB or 10MB drive). Once the second part of the driver is loaded, this definition is changed to describe the true capacity of the drive and INT 13H is replaced by the driver's version of INT 13H that does CHS translation. In some cases the third part, an OS specific device driver, must be loaded to enable CHS translation for devices other than the boot device. ! I don't know the details of how these drivers respond to INT 13H AH=08H or how they set up drive parameter tables (anyone care to help out here?). Some of these drivers convert the L-CHS to an LBA, then they add a small number to the LBA and finally they convert the LBA to a P-CHS. This in effect skips over some sectors at the front of the disk. Support issues: Several identified -- Some OS installation programs will remove or overlay these drivers; some of these drivers do not perform CHS translation using the same algorithms used by the other BIOS types; special OS device drivers may be required in order to use these software drivers For example, under MS Windows the standard FastDisk driver (the 32-bit disk access driver) must be replaced by a driver that understands the Ontrack, Micro House, etc, version of INT 13H. Different CHS translation algorithms may be used by the driver and an OS. ! The hard disk vendors have been shipping these drivers with their drives over 528MB during the last year and they have been ignoring the statements of Microsoft and IBM that these drivers would not be supported in future OS's. Now it appears that both Microsoft and IBM are in a panic trying to figure out how to support some of these drivers in WinNT, Win95 and OS/2. It is unclear what the outcome of this will be at this time. ! NOTE: THIS IS NOT A PRODUCT ENDORSEMENT! An alternate solution for an older ISA system is one of the BIOS replacement cards. This cards have a BIOS option ROM. AMI makes such a card called the "Disk Extender". This card replaces the motherboard's INT 13H BIOS with a INT 13H BIOS that does some form of CHS translation. Another solution for older VL-Bus systems is an ATA-2 (EIDE) type host adapter card that provides a option ROM with an INT 13H replacement. BIOS Type 9 Origin: SCSI host adapters. BIOS call support: Probably INT 13H AH=0xH and FDPT for BIOS drives 80H and 81H, perhaps INT 13H AH=4xH. Description: Most SCSI host adapters contain an option ROM that enables INT 13 support for the attached SCSI hard drives. It is possible to have more than one SCSI host adapter, each with its own option ROM. The CHS used at the INT 13H interface is converted to the LBA that is used in the SCSI commands. INT 13H AH=08H returns a CHS. This CHS will have <=1024 cylinders, <=256 heads and <=63 sectors. The FDPT probably will exist for SCSI drives with BIOS drive numbers of 80H and 81H and probably indicates the same CHS as that returned by INT 13H AH=08H. Even though the CHS used at the INT 13H interface looks like a translated CHS, there is no need to use a EDPT since there is no CHS-to-CHS translation used. Other BIOS calls (most likely host adapter specific) must be used to determine other information about the host adapter or the drives. The INT 13H AH=4xH calls can be used to get beyond 8GB but since there is little support for these calls in today's OS's, there are probably few SCSI host adapters that support these newer INT 13H calls. Support issues: Some SCSI host adapters will not install their option ROM if there are two INT 13H devices previously installed by another INT 13H BIOS (for example, two MFM/RLL/ESDI/ATA drives). Other SCSI adapters will install their option ROM and use BIOS drive numbers greater than 81H. Some older OS's don't understand or use BIOS drive numbers greater than 81H. SCSI adapters are currently faced with the >8GB drive problem. BIOS Type 10 Origin: A european system vendor. BIOS call support: INT 13H AH=0xH and FDPT for BIOS drives 80H and 81H. Description: This BIOS supports drives >528MB but it does not support CHS translation. It supports only ATA drives with LBA capability. INT 13H AH=08H returns an L-CHS. The L-CHS is converted directly to an LBA. The BIOS sets the ATA drive to a P-CHS of 16 heads and 63 sectors using the Initialize Drive Parameters command but it does not use this P-CHS at the ATA interface. ! Support issues: OS/2 will probably work with this BIOS as long as the drive's power on default P-CHS mode uses 16 heads and 63 sectors. Because there is no EDPT, OS/2 uses the ATA Identify Device power on default P-CHS, described in Identify Device words 1, 3 and 6 as the current P-CHS for the drive. However, this may not represent the correct P-CHS. A newer drive will have the its current P-CHS information in Identify Device words 53-58 but for some reason OS/2 does not use this information. ---------------------------------------------------------------------- How it Works -- Partition Tables Version 1b by Hale Landis (landis@sugs.tware.com) THE "HOW IT WORKS" SERIES This is one of several How It Works documents. The series currently includes the following: * How It Works -- CHS Translation * How It Works -- Master Boot Record * How It Works -- DOS Floppy Boot Sector * How It Works -- OS2 Boot Sector * How It Works -- Partition Tables PARTITION SECTOR/RECORD/TABLE BASICS FDISK creates all partition records (sectors). The primary purpose of a partition record is to hold a partition table. The rules for how FDISK works are unwritten but so far most FDISK programs (DOS, OS/2, WinNT, etc) seem to follow the same basic idea. First, all partition table records (sectors) have the same format. This includes the partition table record at cylinder 0, head 0, sector 1 -- what is known as the Master Boot Record (MBR). The last 66 bytes of a partition table record contain a partition table and a 2 byte signature. The first 446 bytes of these sectors usually contain a program but only the program in the MBR is ever executed (so extended partition table records could contain something other than a program in the first 466 bytes). See "How It Works -- The Master Boot Record". Second, extended partitions are "nested" inside one another and extended partition table records form a "linked list". I will attempt to show this in a diagram below. PARTITION TABLE ENTRY FORMAT Each partition table entry is 16 bytes and contains things like the start and end location of a partition in CHS, the start in LBA, the size in sectors, the partition "type" and the "active" flag. Warning: older versions of FDISK may compute incorrect LBA or size values. And note: When your computer boots itself, only the CHS fields of the partition table entries are used (another reason LBA doesn't solve the >528MB problem). The CHS fields in the partition tables are in L-CHS format -- see "How It Works -- CHS Translation". There is no central clearing house to assign the codes used in the one byte "type" field. But codes are assigned (or used) to define most every type of file system that anyone has ever implemented on the x86 PC: 12-bit FAT, 16-bit FAT, HPFS, NTFS, etc. Plus, an extended partition also has a unique type code. Note: I know of no complete list of all the type codes that have been used to date. However, I try to include such a list in a future version of this document. The 16 bytes of a partition table entry are used as follows: +--- Bit 7 is the active partition flag, bits 6-0 are zero. | | +--- Starting CHS in INT 13 call format. | | | | +--- Partition type byte. | | | | | | +--- Ending CHS in INT 13 call format. | | | | | | | | +-- Starting LBA. | | | | | | | | | | +-- Size in sectors. | | | | | | v <--+---> v <--+--> v v 0 1 2 3 4 5 6 7 8 9 A B C D E F DH DL CH CL TB DL CH CL LBA..... SIZE.... 80 01 01 00 06 0e be 94 3e000000 0c610900 1st entry 00 00 81 95 05 0e fe 7d 4a610900 724e0300 2nd entry 00 00 00 00 00 00 00 00 00000000 00000000 3rd entry 00 00 00 00 00 00 00 00 00000000 00000000 4th entry Bytes 0-3 are used by the small program in the Master Boot Record to read the first sector of an active partition into memory. The DH, DL, CH and CL above show which x86 register is loaded when the MBR program calls INT 13H AH=02H to read the active partition's boot sector. See "How It Works -- Master Boot Record". These entries define the following partitions: 1) The first partition, a primary partition DOS FAT, starts at CHS 0H,1H,1H (LBA 3EH) and ends at CHS 294H,EH,3EH with a size of 9610CH sectors. 2) The second partition, an extended partition, starts at CHS 295H,0H,1H (LBA 9614AH) and ends at CHS 37DH,EH,3EH with a size of 34E72H sectors. 3) The third and fourth table entries are unused. PARTITION TABLE RULES Keep in mind that there are NO written rules and NO industry standards on how FDISK should work but here are some basic rules that seem to be followed by most versions of FDISK: 1) In the MBR there can be 0-4 "primary" partitions, OR, 0-3 primary partitions and 0-1 extended partition entry. 2) In an extended partition there can be 0-1 "secondary" partition entries and 0-1 extended partition entries. 3) Only 1 primary partition in the MBR can be marked "active" at any given time. 4) In most versions of FDISK, the first sector of a partition will be aligned such that it is at head 0, sector 1 of a cylinder. This means that there may be unused sectors on the track(s) prior to the first sector of a partition and that there may be unused sectors following a partition table sector. For example, most new versions of FDISK start the first partition (primary or extended) at cylinder 0, head 1, sector 0. This leaves the sectors at cylinder 0, head 0, sectors 2...n as unused sectors. This same layout may be seen on the first track of an extended partition. See example 2 below. Also note that software drivers like Ontrack's Disk Manager depend on these unused sectors because these drivers will "hide" their code there (in cylinder 0, head 0, sectors 2...n). This is also a good place for boot sector virus programs to hang out. 5) The partition table entries (slots) can be used in any order. Some versions of FDISK fill the table from the bottom up and some versions of FDISK fill the table from the top down. Deleting a partition can leave an unused entry (slot) in the middle of a table. 6) And then there is the "hack" that some newer OS's (OS/2 and Linux) use in order to place a partition spanning or passed cylinder 1024 on a system that does not have a CHS translating BIOS. These systems create a partition table entry with the partition's starting and ending CHS information set to all FFH. The starting and ending LBA information is used to describe the location of the partition. The LBA can be converted back to a CHS -- most likely a CHS with more than 1024 cylinders. Since such a CHS can't be used by the system BIOS, these partitions can not be booted or accessed until the OS's kernel and hard disk device drivers are loaded. It is not known if the systems using this "hack" follow the same rules for the creation of these type of partitions. There are NO written rules as to how an OS scans the partition table entries so each OS can have a different method. For DOS, this means that different versions could assign different drive letters to the same FAT file system partitions. PARTITION NESTING What do I mean when I say the partitions are "nested" within each other? Lets look at this example: M = Master Boot Record (and any unused sectors on the same track) E = Extended partition record (and any unused sectors on the same track) pri = a primary partition (first sector is a "boot" sector) sec = a secondary partition (first sector is a "boot" sector) |<----------------the entire disk-------------->| | | |M | | | | E<---rest of 1st ext part---------->| | | | E<---rest of 2nd ext part---->| The first extended partition is described in the MBR and it occupies the entire disk following the primary partition. The second extended partition is described in the first extended partition record and it occupies the entire disk following the first secondary partition. PARTITION TABLE LINKING What do I mean when I say the partition records (tables) form a "linked" list? This means that the MBR has an entry that describes (points to) the first extended partition, the first extended partition table has an entry that describes (points to) the second extended partition table, and so on. There is, in theory, no limited to out long this linked list is. When you ask FDISK to show the DOS "logical drives" it scans the linked list looking for all of the DOS FAT type partitions that may exist. Remember that in an extended partition table, only two entries of the four can be used (rule 2 above). And one more thing... Within a partition, the layout of the file system data varies greatly. However, the first sector of a partition is expected to be a "boot" sector. A DOS FAT file system has: a boot sector, first FAT sectors, second FAT sectors, root directory sectors and finally the file data area. See "How It Works -- OS2 Boot Sector". EXAMPLE 1 A disk containing four DOS FAT partitions (C, D, E and F): |<---------------------the entire disk------------------->| | | |M<---C:---> | | | | E<---D:---><-rest of 1st ext part------------>| | | | E<---E:---><-rest of 2nd ext part->| | | | E<---------F:---------->| EXAMPLE 2 So here is an example of a disk with two primary partitions, one DOS FAT and one OS/2 HPFS, plus an extended partition with another DOS FAT: |<------------------the entire disk------------------>| | | |M | | | | | | | | E| Or in more detail ('n' is the highest cylinder, head or sector number number allowed in the indicated field of the CHS)... +-------------------------------------+ CHS=0,0,1 | Master Boot Record containing | | partition table search program and | | a partition table | | +---------------------------------+ | | | DOS FAT partition description | | points to CHS=0,1,1 | +---------------------------------+ | points to CHS=a | | OS/2 HPFS partition description | | | +---------------------------------+ | | | unused table entry | | | +---------------------------------+ | | | extended partition entry | | points to CHS=b | +---------------------------------+ | +-------------------------------------+ CHS=0,0,2 | the rest of "track 0" -- this is | : to | where the software drivers such as | : normally CHS=0,0,n | Ontrack's Disk Manager or Micro | : unused | House's EZ Drive are located. | : +-------------------------------------+ CHS=0,1,1 | Boot sector for the DOS FAT | : | partition | : a DOS FAT +-------------------------------------+ : file CHS=0,1,2 | rest of the DOS FAT partition | : system to | (FAT table, root directory and | : CHS=x-1,n,n | user data area) | : +-------------------------------------+ CHS=x,0,1 | Boot sector for the OS/2 HPFS | : | file system partition | : an OS/2 +-------------------------------------+ : HPFS file CHS=x,0,2 | rest of the OS/2 HPFS file system | : system to | partition | : CHS=y-1,n,n | | : +-------------------------------------+ CHS=y,0,1 | Partition record for the extended | | partition containing a partition | | record program (never executed) and | | a partition table | | +---------------------------------+ | | | DOS FAT partition description | | points to CHS=b+1 | +---------------------------------+ | | | unused table entry | | | +---------------------------------+ | | | unused table entry | | | +---------------------------------+ | | | unused table entry | | | +---------------------------------+ | +-------------------------------------+ CHS=y,0,2 | the rest of the first track of the | : normally to | extended partition | : unused CHS=y,0,n | | : +-------------------------------------+ CHS=y,1,1 | Boot sector for the DOS FAT | : | partition | : a DOS FAT +-------------------------------------+ : file CHS=y,1,2 | rest of the DOS FAT partition | : system to | (FAT table, root directory and | : CHS=n,n,n | user data area) | : +-------------------------------------+ EXAMPLE 3 Here is a partition record from an extended partition (the first sector of an extended partition). Note that it contains no program code. It contains only the partition table and the signature data. OFFSET 0 1 2 3 4 5 6 7 8 9 A B C D E F *0123456789ABCDEF* 000000 00000000 00000000 00000000 00000000 *................* 000010 TO 0001af SAME AS ABOVE 0001b0 00000000 00000000 00000000 00000001 *................* 0001c0 8195060e fe7d3e00 0000344e 03000000 *.....}>...4N....* 0001d0 00000000 00000000 00000000 00000000 *................* 0001e0 00000000 00000000 00000000 00000000 *................* 0001f0 00000000 00000000 00000000 000055aa *..............U.* NOTES Thanks to yue@heron.Stanford.EDU (Kenneth C. Yue) for pointing out that in V0 of this document I did not properly describe the unused sectors normally found around the partition table sectors. ----------------------------------------------------------------------- How It Works -- Master Boot Record Version 1a by Hale Landis (landis@sugs.tware.com) THE "HOW IT WORKS" SERIES This is one of several How It Works documents. The series currently includes the following: * How It Works -- CHS Translation * How It Works -- Master Boot Record * How It Works -- DOS Floppy Boot Sector * How It Works -- OS2 Boot Sector * How It Works -- Partition Tables MASTER BOOT RECORD This article is a disassembly of a Master Boot Record (MBR). The MBR is the sector at cylinder 0, head 0, sector 1 of a hard disk. An MBR is created by the FDISK program. The FDISK program of all operating systems must create a functionally similar MBR. The MBR is first of what could be many partition sectors, each one containing a four entry partition table. At the completion of your system's Power On Self Test (POST), INT 19 is called. Usually INT 19 tries to read a boot sector from the first floppy drive. If a boot sector is found on the floppy disk, the that boot sector is read into memory at location 0000:7C00 and INT 19 jumps to memory location 0000:7C00. However, if no boot sector is found on the first floppy drive, INT 19 tries to read the MBR from the first hard drive. If an MBR is found it is read into memory at location 0000:7c00 and INT 19 jumps to memory location 0000:7c00. The small program in the MBR will attempt to locate an active (bootable) partition in its partition table. If such a partition is found, the boot sector of that partition is read into memory at location 0000:7C00 and the MBR program jumps to memory location 0000:7C00. Each operating system has its own boot sector format. The small program in the boot sector must locate the first part of the operating system's kernel loader program (or perhaps the kernel itself or perhaps a "boot manager program") and read that into memory. INT 19 is also called when the CTRL-ALT-DEL keys are used. On most systems, CTRL-ALT-DEL causes an short version of the POST to be executed before INT 19 is called. ===== Where stuff is: The MBR program code starts at offset 0000. The MBR messages start at offset 008b. The partition table starts at offset 00be. The signature is at offset 00fe. Here is a summary of what this thing does: If an active partition is found, that partition's boot record is read into 0000:7c00 and the MBR code jumps to 0000:7c00 with SI pointing to the partition table entry that describes the partition being booted. The boot record program uses this data to determine the drive being booted from and the location of the partition on the disk. If no active partition table enty is found, ROM BASIC is entered via INT 18. All other errors cause a system hang, see label HANG. NOTES (VERY IMPORTANT): 1) The first byte of an active partition table entry is 80. This byte is loaded into the DL register before INT 13 is called to read the boot sector. When INT 13 is called, DL is the BIOS device number. Because of this, the boot sector read by this MBR program can only be read from BIOS device number 80 (the first hard disk). This is one of the reasons why it is usually not possible to boot from any other hard disk. 2) The MBR program uses the CHS based INT 13H AH=02H call to read the boot sector of the active partition. The location of the active partition's boot sector is in the partition table entry in CHS format. If the drive is >528MB, this CHS must be a translated CHS (or L-CHS, see my BIOS TYPES document). No addresses in LBA form are used (another reason why LBA doesn't solve the >528MB problem). ===== Here is the entire MBR record (hex dump and ascii). OFFSET 0 1 2 3 4 5 6 7 8 9 A B C D E F *0123456789ABCDEF* 000000 fa33c08e d0bc007c 8bf45007 501ffbfc *.3.....|..P.P...* 000010 bf0006b9 0001f2a5 ea1d0600 00bebe07 *................* 000020 b304803c 80740e80 3c00751c 83c610fe *...<.t..<.u.....* 000030 cb75efcd 188b148b 4c028bee 83c610fe *.u......L.......* 000040 cb741a80 3c0074f4 be8b06ac 3c00740b *.t..<.t.....<.t.* 000050 56bb0700 b40ecd10 5eebf0eb febf0500 *V.......^.......* 000060 bb007cb8 010257cd 135f730c 33c0cd13 *..|...W.._s.3...* 000070 4f75edbe a306ebd3 bec206bf fe7d813d *Ou...........}.=* 000080 55aa75c7 8bf5ea00 7c000049 6e76616c *U.u.....|..Inval* 000090 69642070 61727469 74696f6e 20746162 *id partition tab* 0000a0 6c650045 72726f72 206c6f61 64696e67 *le.Error loading* 0000b0 206f7065 72617469 6e672073 79737465 * operating syste* 0000c0 6d004d69 7373696e 67206f70 65726174 *m.Missing operat* 0000d0 696e6720 73797374 656d0000 00000000 *ing system......* 0000e0 00000000 00000000 00000000 00000000 *................* 0000f0 TO 0001af SAME AS ABOVE 0001b0 00000000 00000000 00000000 00008001 *................* 0001c0 0100060d fef83e00 00000678 0d000000 *......>....x....* 0001d0 00000000 00000000 00000000 00000000 *................* 0001e0 00000000 00000000 00000000 00000000 *................* 0001f0 00000000 00000000 00000000 000055aa *..............U.* ===== Here is the disassembly of the MBR... This sector is initially loaded into memory at 0000:7c00 but it immediately relocates itself to 0000:0600. BEGIN: NOW AT 0000:7C00, RELOCATE 0000:7C00 FA CLI disable int's 0000:7C01 33C0 XOR AX,AX set stack seg to 0000 0000:7C03 8ED0 MOV SS,AX 0000:7C05 BC007C MOV SP,7C00 set stack ptr to 7c00 0000:7C08 8BF4 MOV SI,SP SI now 7c00 0000:7C0A 50 PUSH AX 0000:7C0B 07 POP ES ES now 0000:7c00 0000:7C0C 50 PUSH AX 0000:7C0D 1F POP DS DS now 0000:7c00 0000:7C0E FB STI allow int's 0000:7C0F FC CLD clear direction 0000:7C10 BF0006 MOV DI,0600 DI now 0600 0000:7C13 B90001 MOV CX,0100 move 256 words (512 bytes) 0000:7C16 F2 REPNZ move MBR from 0000:7c00 0000:7C17 A5 MOVSW to 0000:0600 0000:7C18 EA1D060000 JMP 0000:061D jmp to NEW_LOCATION NEW_LOCATION: NOW AT 0000:0600 0000:061D BEBE07 MOV SI,07BE point to first table entry 0000:0620 B304 MOV BL,04 there are 4 table entries SEARCH_LOOP1: SEARCH FOR AN ACTIVE ENTRY 0000:0622 803C80 CMP BYTE PTR [SI],80 is this the active entry? 0000:0625 740E JZ FOUND_ACTIVE yes 0000:0627 803C00 CMP BYTE PTR [SI],00 is this an inactive entry? 0000:062A 751C JNZ NOT_ACTIVE no 0000:062C 83C610 ADD SI,+10 incr table ptr by 16 0000:062F FECB DEC BL decr count 0000:0631 75EF JNZ SEARCH_LOOP1 jmp if not end of table 0000:0633 CD18 INT 18 GO TO ROM BASIC FOUND_ACTIVE: FOUND THE ACTIVE ENTRY 0000:0635 8B14 MOV DX,[SI] set DH/DL for INT 13 call 0000:0637 8B4C02 MOV CX,[SI+02] set CH/CL for INT 13 call 0000:063A 8BEE MOV BP,SI save table ptr SEARCH_LOOP2: MAKE SURE ONLY ONE ACTIVE ENTRY 0000:063C 83C610 ADD SI,+10 incr table ptr by 16 0000:063F FECB DEC BL decr count 0000:0641 741A JZ READ_BOOT jmp if end of table 0000:0643 803C00 CMP BYTE PTR [SI],00 is this an inactive entry? 0000:0646 74F4 JZ SEARCH_LOOP2 yes NOT_ACTIVE: MORE THAN ONE ACTIVE ENTRY FOUND 0000:0648 BE8B06 MOV SI,068B display "Invld prttn tbl" DISPLAY_MSG: DISPLAY MESSAGE LOOP 0000:064B AC LODSB get char of message 0000:064C 3C00 CMP AL,00 end of message 0000:064E 740B JZ HANG yes 0000:0650 56 PUSH SI save SI 0000:0651 BB0700 MOV BX,0007 screen attributes 0000:0654 B40E MOV AH,0E output 1 char of message 0000:0656 CD10 INT 10 to the display 0000:0658 5E POP SI restore SI 0000:0659 EBF0 JMP DISPLAY_MSG do it again HANG: HANG THE SYSTEM LOOP 0000:065B EBFE JMP HANG sit and stay! READ_BOOT: READ ACTIVE PARITION BOOT RECORD 0000:065D BF0500 MOV DI,0005 INT 13 retry count INT13RTRY: INT 13 RETRY LOOP 0000:0660 BB007C MOV BX,7C00 0000:0663 B80102 MOV AX,0201 read 1 sector 0000:0666 57 PUSH DI save DI 0000:0667 CD13 INT 13 read sector into 0000:7c00 0000:0669 5F POP DI restore DI 0000:066A 730C JNB INT13OK jmp if no INT 13 0000:066C 33C0 XOR AX,AX call INT 13 and 0000:066E CD13 INT 13 do disk reset 0000:0670 4F DEC DI decr DI 0000:0671 75ED JNZ INT13RTRY if not zero, try again 0000:0673 BEA306 MOV SI,06A3 display "Errr ldng systm" 0000:0676 EBD3 JMP DISPLAY_MSG jmp to display loop INT13OK: INT 13 ERROR 0000:0678 BEC206 MOV SI,06C2 "missing op sys" 0000:067B BFFE7D MOV DI,7DFE point to signature 0000:067E 813D55AA CMP WORD PTR [DI],AA55 is signature correct? 0000:0682 75C7 JNZ DISPLAY_MSG no 0000:0684 8BF5 MOV SI,BP set SI 0000:0686 EA007C0000 JMP 0000:7C00 JUMP TO THE BOOT SECTOR WITH SI POINTING TO PART TABLE ENTRY Messages here. 0000:0680 ........ ........ ......49 6e76616c * Inval* 0000:0690 69642070 61727469 74696f6e 20746162 *id partition tab* 0000:06a0 6c650045 72726f72 206c6f61 64696e67 *le.Error loading* 0000:06b0 206f7065 72617469 6e672073 79737465 * operating syste* 0000:06c0 6d004d69 7373696e 67206f70 65726174 *m.Missing operat* 0000:06d0 696e6720 73797374 656d00.. ........ *ing system. * Data not used. 0000:06d0 ........ ........ ......00 00000000 * .....* 0000:06e0 00000000 00000000 00000000 00000000 *................* 0000:06f0 00000000 00000000 00000000 00000000 *................* 0000:0700 00000000 00000000 00000000 00000000 *................* 0000:0710 00000000 00000000 00000000 00000000 *................* 0000:0720 00000000 00000000 00000000 00000000 *................* 0000:0730 00000000 00000000 00000000 00000000 *................* 0000:0740 00000000 00000000 00000000 00000000 *................* 0000:0750 00000000 00000000 00000000 00000000 *................* 0000:0760 00000000 00000000 00000000 00000000 *................* 0000:0770 00000000 00000000 00000000 00000000 *................* 0000:0780 00000000 00000000 00000000 00000000 *................* 0000:0790 00000000 00000000 00000000 00000000 *................* 0000:07a0 00000000 00000000 00000000 00000000 *................* 0000:07b0 00000000 00000000 00000000 0000.... *............ * The partition table starts at 0000:07be. Each partition table entry is 16 bytes. This table defines a single primary partition which is also an active (bootable) partition. 0000:07b0 ........ ........ ........ ....8001 * ....* 0000:07c0 0100060d fef83e00 00000678 0d000000 *......>....x....* 0000:07d0 00000000 00000000 00000000 00000000 *................* 0000:07e0 00000000 00000000 00000000 00000000 *................* 0000:07f0 00000000 00000000 00000000 0000.... *............ * The last two bytes contain a 55AAH signature. 0000:07f0 ........ ........ ........ ....55aa *..............U.* --------------------------------------------------------------------- How It Works -- DOS Floppy Disk Boot Sector Version 1a by Hale Landis (landis@sugs.tware.com) THE "HOW IT WORKS" SERIES This is one of several How It Works documents. The series currently includes the following: * How It Works -- CHS Translation * How It Works -- Master Boot Record * How It Works -- DOS Floppy Boot Sector * How It Works -- OS2 Boot Sector * How It Works -- Partition Tables DOS FLOPPY DISK BOOT SECTOR This article is a disassembly of a floppy disk boot sector for a DOS floppy. The boot sector of a floppy disk is located at cylinder 0, head 0, sector 1. This sector is created by a floppy disk formating program, such as the DOS FORMAT program. The boot sector of a FAT hard disk partition has a similar layout and function. Basically a bootable FAT hard disk partition looks like a big floppy during the early stages of the system's boot processing. At the completion of your system's Power On Self Test (POST), INT 19 is called. Usually INT 19 tries to read a boot sector from the first floppy drive. If a boot sector is found on the floppy disk, the that boot sector is read into memory at location 0000:7C00 and INT 19 jumps to memory location 0000:7C00. However, if no boot sector is found on the first floppy drive, INT 19 tries to read the MBR from the first hard drive. If an MBR is found it is read into memory at location 0000:7c00 and INT 19 jumps to memory location 0000:7c00. The small program in the MBR will attempt to locate an active (bootable) partition in its partition table. If such a partition is found, the boot sector of that partition is read into memory at location 0000:7C00 and the MBR program jumps to memory location 0000:7C00. Each operating system has its own boot sector format. The small program in the boot sector must locate the first part of the operating system's kernel loader program (or perhaps the kernel itself or perhaps a "boot manager program") and read that into memory. INT 19 is also called when the CTRL-ALT-DEL keys are used. On most systems, CTRL-ALT-DEL causes an short version of the POST to be executed before INT 19 is called. ===== Where stuff is: The BIOS Parameter Block (BPB) starts at offset 0. The boot sector program starts at offset 3e. The messages issued by this program start at offset 19e. The DOS hidden file names start at offset 1e6. The boot sector signature is at offset 1fe. Here is a summary of what this thing does: 1) Copy Diskette Parameter Table which is pointed to by INT 1E. 2) Alter the copy of the Diskette Parameter Table. 3) Alter INT 1E to point to altered Diskette Parameter Table. 4) Do INT 13 AH=00, disk reset call. 5) Compute sector address of root directory. 6) Read first sector of root directory into 0000:0500. 7) Confirm that first two directory entries are for IO.SYS and MSDOS.SYS. 8) Read first 3 sectors of IO.SYS into 0000:0700 (or 0070:0000). 9) Leave some information in the registers and jump to IO.SYS at 0070:0000. NOTE: This program uses the CHS based INT 13H AH=02 to read the FAT root directory and to read the IO.SYS file. If the drive is >528MB, this CHS must be a translated CHS (or L-CHS, see my BIOS TYPES document). Except for internal computations no addresses in LBA form are used, another reason why LBA doesn't solve the >528MB problem. ===== Here is the entire sector in hex and ascii. OFFSET 0 1 2 3 4 5 6 7 8 9 A B C D E F *0123456789ABCDEF* 000000 eb3c904d 53444f53 352e3000 02010100 *.<.MSDOS5.0.....* 000010 02e00040 0bf00900 12000200 00000000 *...@............* 000020 00000000 0000295a 5418264e 4f204e41 *......)ZT.&NO NA* 000030 4d452020 20204641 54313220 2020fa33 *ME FAT12 .3* 000040 c08ed0bc 007c1607 bb780036 c5371e56 *.....|...x.6.7.V* 000050 1653bf3e 7cb90b00 fcf3a406 1fc645fe *.S.>|.........E.* 000060 0f8b0e18 7c884df9 894702c7 073e7cfb *....|.M..G...>|.* 000070 cd137279 33c03906 137c7408 8b0e137c *..ry3.9..|t....|* 000080 890e207c a0107cf7 26167c03 061c7c13 *.. |..|.&.|...|.* 000090 161e7c03 060e7c83 d200a350 7c891652 *..|...|....P|..R* 0000a0 7ca3497c 89164b7c b82000f7 26117c8b *|.I|..K|. ..&.|.* 0000b0 1e0b7c03 c348f7f3 0106497c 83164b7c *..|..H....I|..K|* 0000c0 00bb0005 8b16527c a1507ce8 9200721d *......R|.P|...r.* 0000d0 b001e8ac 0072168b fbb90b00 bee67df3 *.....r........}.* 0000e0 a6750a8d 7f20b90b 00f3a674 18be9e7d *.u... .....t...}* 0000f0 e85f0033 c0cd165e 1f8f048f 4402cd19 *._.3...^....D...* 000100 585858eb e88b471a 48488a1e 0d7c32ff *XXX...G.HH...|2.* 000110 f7e30306 497c1316 4b7cbb00 07b90300 *....I|..K|......* 000120 505251e8 3a0072d8 b001e854 00595a58 *PRQ.:.r....T.YZX* 000130 72bb0501 0083d200 031e0b7c e2e28a2e *r..........|....* 000140 157c8a16 247c8b1e 497ca14b 7cea0000 *.|..$|..I|.K|...* 000150 7000ac0a c07429b4 0ebb0700 cd10ebf2 *p....t).........* 000160 3b16187c 7319f736 187cfec2 88164f7c *;..|s..6.|....O|* 000170 33d2f736 1a7c8816 257ca34d 7cf8c3f9 *3..6.|..%|.M|...* 000180 c3b4028b 164d7cb1 06d2e60a 364f7c8b *.....M|.....6O|.* 000190 ca86e98a 16247c8a 36257ccd 13c30d0a *.....$|.6%|.....* 0001a0 4e6f6e2d 53797374 656d2064 69736b20 *Non-System disk * 0001b0 6f722064 69736b20 6572726f 720d0a52 *or disk error..R* 0001c0 65706c61 63652061 6e642070 72657373 *eplace and press* 0001d0 20616e79 206b6579 20776865 6e207265 * any key when re* 0001e0 6164790d 0a00494f 20202020 20205359 *ady...IO SY* 0001f0 534d5344 4f532020 20535953 000055aa *SMSDOS SYS..U.* ===== The first 62 bytes of a boot sector are known as the BIOS Parameter Block (BPB). Here is the layout of the BPB fields and the values they are assigned in this boot sector: db JMP instruction at 7c00 size 2 = eb3c db NOP instruction 7c02 1 90 db OEMname 7c03 8 'MSDOS5.0' dw bytesPerSector 7c0b 2 0200 db sectPerCluster 7c0d 1 01 dw reservedSectors 7c0e 2 0001 db numFAT 7c10 1 02 dw numRootDirEntries 7c11 2 00e0 dw numSectors 7c13 2 0b40 (ignore numSectorsHuge) db mediaType 7c15 1 f0 dw numFATsectors 7c16 2 0009 dw sectorsPerTrack 7c18 2 0012 dw numHeads 7c1a 2 0002 dd numHiddenSectors 7c1c 4 00000000 dd numSectorsHuge 7c20 4 00000000 db driveNum 7c24 1 00 db reserved 7c25 1 00 db signature 7c26 1 29 dd volumeID 7c27 4 5a541826 db volumeLabel 7c2b 11 'NO NAME ' db fileSysType 7c36 8 'FAT12 ' ===== Here is the boot sector... The first 3 bytes of the BPB are JMP and NOP instructions. 0000:7C00 EB3C JMP START 0000:7C02 90 NOP Here is the rest of the BPB. 0000:7C00 ......4d 53444f53 352e3000 02010100 * MSDOS5.0.....* 0000:7C10 02e00040 0bf00900 12000200 00000000 *...@............* 0000:7C20 00000000 0000295a 5418264e 4f204e41 *......)ZT.&NO NA* 0000:7C30 4d452020 20204641 54313220 2020.... *ME FAT12 * Now pay attention here... The 11 bytes starting at 0000:7c3e are immediately overlaid by information copied from another part of memory. That information is the Diskette Parameter Table. This data is pointed to by INT 1E. This data is: 7c3e = Step rate and head unload time. 7c3f = Head load time and DMA mode flag. 7c40 = Delay for motor turn off. 7c41 = Bytes per sector. 7c42 = Sectors per track. 7c43 = Intersector gap length. 7c44 = Data length. 7c45 = Intersector gap length during format. 7c46 = Format byte value. 7c47 = Head settling time. 7c48 = Delay until motor at normal speed. The 11 bytes starting at 0000:7c49 are also overlaid by the following data: 7c49 - 7c4c = diskette sector address (as LBA) of the data area. 7c4d - 7c4e = cylinder number to read from. 7c4f - 7c4f = sector number to read from. 7c50 - 7c53 = diskette sector address (as LBA) of the root directory. START: START OF BOOT SECTOR PROGRAM 0000:7C3E FA CLI interrupts off 0000:7C3F 33C0 XOR AX,AX set AX to zero 0000:7C41 8ED0 MOV SS,AX SS is now zero 0000:7C43 BC007C MOV SP,7C00 SP is now 7c00 0000:7C46 16 PUSH SS also set ES 0000:7C47 07 POP ES to zero The INT 1E vector is at 0000:0078. Get the address that the vector points to into the DS:SI registers. 0000:7C48 BB7800 MOV BX,0078 BX is now 78 0000:7C4B 36 SS: 0000:7C4C C537 LDS SI,[BX] DS:SI is now [0:78] 0000:7C4E 1E PUSH DS save DS:SI -- 0000:7C4F 56 PUSH SI saves param tbl addr 0000:7C50 16 PUSH SS save SS:BX -- 0000:7C51 53 PUSH BX saves INT 1E address Move the diskette param table to 0000:7c3e. 0000:7C52 BF3E7C MOV DI,7C3E DI is address of START 0000:7C55 B90B00 MOV CX,000B count is 11 0000:7C58 FC CLD clear direction 0000:7C59 F3 REPZ move the diskette param 0000:7C5A A4 MOVSB table to 0000:7c3e 0000:7C5B 06 PUSH ES also set DS 0000:7C5C 1F POP DS to zero Alter some of the diskette param table data. 0000:7C5D C645FE0F MOV BYTE PTR [DI-02],0F change head settle time at 0000:7c47 0000:7C61 8B0E187C MOV CX,[7C18] sectors per track 0000:7C65 884DF9 MOV [DI-07],CL save at 0000:7c42 Change INT 1E so that it points to the altered Diskette param table at 0000:7c3e. 0000:7C68 894702 MOV [BX+02],AX change INT 1E segment 0000:7C6B C7073E7C MOV WORD PTR [BX],7C3E change INT 1E offset Call INT 13 with AX=0000, disk reset, so that the new diskette param table is used. 0000:7C6F FB STI interrupts on 0000:7C70 CD13 INT 13 do diskette reset call 0000:7C72 7279 JB TALK jmp if any error Detemine the starting sector address of the root directory as an LBA. 0000:7C74 33C0 XOR AX,AX AX is now zero 0000:7C76 3906137C CMP [7C13],AX number sectros zero? 0000:7C7A 7408 JZ SMALL_DISK yes 0000:7C7C 8B0E137C MOV CX,[7C13] number of sectors 0000:7C80 890E207C MOV [7C20],CX save in huge num sects SMALL_DISK: 0000:7C84 A0107C MOV AL,[7C10] number of FAT tables 0000:7C87 F726167C MUL WORD PTR [7C16] number of fat sectors 0000:7C8B 03061C7C ADD AX,[7C1C] number of hidden sectors 0000:7C8F 13161E7C ADC DX,[7C1E] number of hidden sectors 0000:7C93 03060E7C ADD AX,[7C0E] number of reserved sectors 0000:7C97 83D200 ADC DX,+00 number of reserved sectors 0000:7C9A A3507C MOV [7C50],AX save start addr 0000:7C9D 8916527C MOV [7C52],DX of root dir (as LBA) 0000:7CA1 A3497C MOV [7C49],AX save start addr 0000:7CA4 89164B7C MOV [7C4B],DX of root dir (as LBA) Determine sector address of first sector in the data area as an LBA. 0000:7CA8 B82000 MOV AX,0020 size of a dir entry (32) 0000:7CAB F726117C MUL WORD PTR [7C11] number of root dir entries 0000:7CAF 8B1E0B7C MOV BX,[7C0B] bytes per sector 0000:7CB3 03C3 ADD AX,BX 0000:7CB5 48 DEC AX 0000:7CB6 F7F3 DIV BX 0000:7CB8 0106497C ADD [7C49],AX add to start addr 0000:7CBC 83164B7C00 ADC WORD PTR [7C4B],+00 of root dir (as LBA) Read the first root dir sector into 0000:0500. 0000:7CC1 BB0005 MOV BX,0500 addr to read into 0000:7CC4 8B16527C MOV DX,[7C52] get start of address 0000:7CC8 A1507C MOV AX,[7C50] of root dir (as LBA) 0000:7CCB E89200 CALL CONVERT call conversion routine 0000:7CCE 721D JB TALK jmp is any error 0000:7CD0 B001 MOV AL,01 read 1 sector 0000:7CD2 E8AC00 CALL READ_SECTORS read 1st root dir sector 0000:7CD5 7216 JB TALK jmp if any error 0000:7CD7 8BFB MOV DI,BX addr of 1st dir entry 0000:7CD9 B90B00 MOV CX,000B count is 11 0000:7CDC BEE67D MOV SI,7DE6 addr of file names 0000:7CDF F3 REPZ is this "IO.SYS"? 0000:7CE0 A6 CMPSB 0000:7CE1 750A JNZ TALK no 0000:7CE3 8D7F20 LEA DI,[BX+20] addr of next dir entry 0000:7CE6 B90B00 MOV CX,000B count is 11 0000:7CE9 F3 REPZ is this "MSDOS.SYS"? 0000:7CEA A6 CMPSB 0000:7CEB 7418 JZ FOUND_FILES they are equal TALK: Display "Non-System disk..." message, wait for user to hit a key, restore the INT 1E vector and then call INT 19 to start boot processing all over again. 0000:7CED BE9E7D MOV SI,7D9E "Non-System disk..." 0000:7CF0 E85F00 CALL MSG_LOOP display message 0000:7CF3 33C0 XOR AX,AX INT 16 function 0000:7CF5 CD16 INT 16 read keyboard 0000:7CF7 5E POP SI get INT 1E vector's 0000:7CF8 1F POP DS address 0000:7CF9 8F04 POP [SI] restore the INT 1E 0000:7CFB 8F4402 POP [SI+02] vector's data 0000:7CFE CD19 INT 19 CALL INT 19 to try again SETUP_TALK: 0000:7D00 58 POP AX pop junk off stack 0000:7D01 58 POP AX pop junk off stack 0000:7D02 58 POP AX pop junk off stack 0000:7D03 EBE8 JMP TALK now talk to the user FOUND_FILES: Compute the sector address of the first sector of IO.SYS. 0000:7D05 8B471A MOV AX,[BX+1A] get starting cluster num 0000:7D08 48 DEC AX subtract 1 0000:7D09 48 DEC AX subtract 1 0000:7D0A 8A1E0D7C MOV BL,[7C0D] sectors per cluster 0000:7D0E 32FF XOR BH,BH 0000:7D10 F7E3 MUL BX multiply 0000:7D12 0306497C ADD AX,[7C49] add start addr of 0000:7D16 13164B7C ADC DX,[7C4B] root dir (as LBA) Read IO.SYS into memory at 0000:0700. IO.SYS is 3 sectors long. 0000:7D1A BB0007 MOV BX,0700 address to read into 0000:7D1D B90300 MOV CX,0003 read 3 sectors READ_LOOP: Read the first 3 sectors of IO.SYS (IO.SYS is much longer than 3 sectors). 0000:7D20 50 PUSH AX save AX 0000:7D21 52 PUSH DX save DX 0000:7D22 51 PUSH CX save CX 0000:7D23 E83A00 CALL CONVERT call conversion routine 0000:7D26 72D8 JB SETUP_TALK jmp if error 0000:7D28 B001 MOV AL,01 read one sector 0000:7D2A E85400 CALL READ_SECTORS read one sector 0000:7D2D 59 POP CX restore CX 0000:7D2E 5A POP DX restore DX 0000:7D2F 58 POP AX restore AX 0000:7D30 72BB JB TALK jmp if any INT 13 error 0000:7D32 050100 ADD AX,0001 add one to the sector addr 0000:7D35 83D200 ADC DX,+00 add one to the sector addr 0000:7D38 031E0B7C ADD BX,[7C0B] incr mem addr by sect size 0000:7D3C E2E2 LOOP READ_LOOP read next sector Leave information in the AX, BX, CX and DX registers for IO.SYS to use. Finally, jump to IO.SYS at 0070:0000. 0000:7D3E 8A2E157C MOV CH,[7C15] media type 0000:7D42 8A16247C MOV DL,[7C24] drive number 0000:7D46 8B1E497C MOV BX,[7C49] get start addr of 0000:7D4A A14B7C MOV AX,[7C4B] root dir (as LBA) 0000:7D4D EA00007000 JMP 0070:0000 JUMP TO 0070:0000 MSG_LOOP: This routine displays a message using INT 10 one character at a time. The message address is in DS:SI. 0000:7D52 AC LODSB get message character 0000:7D53 0AC0 OR AL,AL end of message? 0000:7D55 7429 JZ RETURN jmp if yes 0000:7D57 B40E MOV AH,0E display one character 0000:7D59 BB0700 MOV BX,0007 video attrbiutes 0000:7D5C CD10 INT 10 display one character 0000:7D5E EBF2 JMP MSG_LOOP do again CONVERT: This routine converts a sector address (an LBA) to a CHS address. The LBA is in DX:AX. 0000:7D60 3B16187C CMP DX,[7C18] hi part of LBA > sectPerTrk? 0000:7D64 7319 JNB SET_CARRY jmp if yes 0000:7D66 F736187C DIV WORD PTR [7C18] div by sectors per track 0000:7D6A FEC2 INC DL add 1 to sector number 0000:7D6C 88164F7C MOV [7C4F],DL save sector number 0000:7D70 33D2 XOR DX,DX zero DX 0000:7D72 F7361A7C DIV WORD PTR [7C1A] div number of heads 0000:7D76 8816257C MOV [7C25],DL save head number 0000:7D7A A34D7C MOV [7C4D],AX save cylinder number 0000:7D7D F8 CLC clear carry 0000:7D7E C3 RET return SET_CARRY: 0000:7D7F F9 STC set carry RETURN: 0000:7D80 C3 RET return READ_SECTORS: The caller of this routine supplies: AL = number of sectors to read ES:BX = memory location to read into and CHS address to read from in memory locations 7c25 and 7C4d-7c4f. 0000:7D81 B402 MOV AH,02 INT 13 read sectors 0000:7D83 8B164D7C MOV DX,[7C4D] get cylinder number 0000:7D87 B106 MOV CL,06 shift count 0000:7D89 D2E6 SHL DH,CL shift upper cyl left 6 bits 0000:7D8B 0A364F7C OR DH,[7C4F] or in sector number 0000:7D8F 8BCA MOV CX,DX move to CX 0000:7D91 86E9 XCHG CH,CL CH=cyl lo, CL=cyl hi + sect 0000:7D93 8A16247C MOV DL,[7C24] drive number 0000:7D97 8A36257C MOV DH,[7C25] head number 0000:7D9B CD13 INT 13 read sectors 0000:7D9D C3 RET return Data not used. 0000:7D90 ca86e98a 16247c8a 36257ccd 13c3.... *.....$|.6%|... * Messages here. 0000:7D90 ........ ........ ........ ....0d0a * ..* 0000:7Da0 4e6f6e2d 53797374 656d2064 69736b20 *Non-System disk * 0000:7Db0 6f722064 69736b20 6572726f 720d0a52 *or disk error..R* 0000:7Dc0 65706c61 63652061 6e642070 72657373 *eplace and press* 0000:7Dd0 20616e79 206b6579 20776865 6e207265 * any key when re* 0000:7De0 6164790d 0a00.... ........ ........ *ady... * MS DOS hidden file names (first two root directory entries). 0000:7De0 ........ ....494f 20202020 20205359 * IO SY* 0000:7Df0 534d5344 4f532020 20535953 000055aa *SMSDOS SYS..U.* The last two bytes contain a 55AAH signature. 0000:7Df0 ........ ........ ........ ....55aa * U.* --------------------------------------------------------------------- How It Works -- OS2 Boot Sector Version 1a by Hale Landis (landis@sugs.tware.com) THE "HOW IT WORKS" SERIES This is one of several How It Works documents. The series currently includes the following: * How It Works -- CHS Translation * How It Works -- Master Boot Record * How It Works -- DOS Floppy Boot Sector * How It Works -- OS2 Boot Sector * How It Works -- Partition Tables OS2 BOOT SECTOR Note: I'll leave it to someone else to provide you with a disassembly of an OS/2 HPFS boot sector, or a Linux boot sector, or a WinNT boot sector, etc. This article is a disassembly of a floppy or hard disk boot sector for OS/2. Apparently OS/2 uses the same boot sector for both environments. Basically a bootable FAT hard disk partition looks like a big floppy during the early stages of the system's boot processing. This sector is at cylinder 0, head 0, sector 1 of a floppy or it is the first sector within a FAT hard disk partition. OS/2 floppy disk and hard disk boot sectors are created by the OS/2 FORMAT program. At the completion of your system's Power On Self Test (POST), INT 19 is called. Usually INT 19 tries to read a boot sector from the first floppy drive. If a boot sector is found on the floppy disk, the that boot sector is read into memory at location 0000:7C00 and INT 19 jumps to memory location 0000:7C00. However, if no boot sector is found on the first floppy drive, INT 19 tries to read the MBR from the first hard drive. If an MBR is found it is read into memory at location 0000:7c00 and INT 19 jumps to memory location 0000:7c00. The small program in the MBR will attempt to locate an active (bootable) partition in its partition table. If such a partition is found, the boot sector of that partition is read into memory at location 0000:7C00 and the MBR program jumps to memory location 0000:7C00. Each operating system has its own boot sector format. The small program in the boot sector must locate the first part of the operating system's kernel loader program (or perhaps the kernel itself or perhaps a "boot manager program") and read that into memory. INT 19 is also called when the CTRL-ALT-DEL keys are used. On most systems, CTRL-ALT-DEL causes an short version of the POST to be executed before INT 19 is called. ===== Where stuff is: The BIOS Parameter Block (BPB) starts at offset 0. The boot sector program starts at offset 46. The messages issued by this program start at offset 198. The OS/2 boot loader file name starts at offset 1d5. The boot sector signature is at offset 1fe. Here is a summary of what this thing does: 1) If booting from a hard disk partition, skip to step 6. 2) Copy Diskette Parameter Table which is pointed to by INT 1E to the top of memory. 3) Alter the copy of the Diskette Parameter Table. 4) Alter INT 1E to point to altered Diskette Parameter Table at the top of memory. 5) Do INT 13 AH=00, disk reset call so that the altered Diskette Parameter Table is used. 6) Compute sector address of the root directory. 7) Read the entire root directory into memory starting at location 1000:0000. 8) Search the root directory entires for the file OS2BOOT. 9) Read the OS2BOOT file into memory at 0800:0000. 10) Do a far return to enter the OS2BOOT program at 0800:0000. NOTES: This program uses the CHS based INT 13H AH=02 to read the FAT root directory and to read the OS2BOOT file. If the drive is >528MB, this CHS must be a translated CHS (or L-CHS, see my BIOS TYPES document). Except for internal computations no addresses in LBA form are used, another reason why LBA doesn't solve the >528MB problem. ===== Here is the entire sector in hex and ascii. OFFSET 0 1 2 3 4 5 6 7 8 9 A B C D E F *0123456789ABCDEF* 000000 eb449049 424d2032 302e3000 02100100 *.D.IBM 20.0.....* 000010 02000200 00f8d800 3e000e00 3e000000 *........>...>...* 000020 06780d00 80002900 1c0c234e 4f204e41 *.x....)...#NO NA* 000030 4d452020 20204641 54202020 20200000 *ME FAT ..* 000040 00100000 0000fa33 db8ed3bc ff7bfbba *.......3.....{..* 000050 c0078eda 803e2400 00753d1e b840008e *.....>$..u=..@..* 000060 c026ff0e 1300cd12 c1e0068e c033ff33 *.&...........3.3* 000070 c08ed8c5 367800fc b90b00f3 a41fa118 *....6x..........* 000080 0026a204 001e33c0 8ed8a378 008c067a *.&....3....x...z* 000090 001f8a16 2400cd13 a0100098 f7261600 *....$........&..* 0000a0 03060e00 5091b820 00f72611 008b1e0b *....P.. ..&.....* 0000b0 0003c348 f7f35003 c1a33e00 b800108e *...H..P...>.....* 0000c0 c033ff59 890e4400 58a34200 33d2e873 *.3.Y..D.X.B.3..s* 0000d0 0033db8b 0e11008b fb51b90b 00bed501 *.3.......Q......* 0000e0 f3a65974 0583c320 e2ede335 268b471c *..Yt... ...5&.G.* 0000f0 268b571e f7360b00 fec08ac8 268b571a *&.W..6......&.W.* 000100 4a4aa00d 0032e4f7 e203063e 0083d200 *JJ...2.....>....* 000110 bb00088e c333ff06 57e82800 8d360b00 *.....3..W.(..6..* 000120 cbbe9801 eb03bead 01e80900 bec201e8 *................* 000130 0300fbeb feac0ac0 7409b40e bb0700cd *........t.......* 000140 10ebf2c3 50525103 061c0013 161e00f7 *....PRQ.........* 000150 361800fe c28ada33 d2f7361a 008afa8b *6......3..6.....* 000160 d0a11800 2ac34050 b402b106 d2e60af3 *....*.@P........* 000170 8bca86e9 8a162400 8af78bdf cd1372a6 *......$.......r.* 000180 5b598bc3 f7260b00 03f85a58 03c383d2 *[Y...&....ZX....* 000190 002acb7f afc31200 4f532f32 20212120 *.*......OS/2 !! * 0001a0 53595330 31343735 0d0a0012 004f532f *SYS01475.....OS/* 0001b0 32202121 20535953 30323032 350d0a00 *2 !! SYS02025...* 0001c0 12004f53 2f322021 21205359 53303230 *..OS/2 !! SYS020* 0001d0 32370d0a 004f5332 424f4f54 20202020 *27...OS2BOOT * 0001e0 00000000 00000000 00000000 00000000 *................* 0001f0 00000000 00000000 00000000 000055aa *..............U.* ===== The first 62 bytes of a boot sector are known as the BIOS Parameter Block (BPB). Here is the layout of the BPB fields and the values they are assigned in this boot sector: db JMP instruction at 7c00 size 2 = eb44 db NOP instruction 7c02 1 90 db OEMname 7c03 8 'IBM 20.0' dw bytesPerSector 7c0b 2 0200 db sectPerCluster 7c0d 1 01 dw reservedSectors 7c0e 2 0001 db numFAT 7c10 1 02 dw numRootDirEntries 7c11 2 0200 dw numSectors 7c13 2 0000 (use numSectorsHuge) db mediaType 7c15 1 f8 dw numFATsectors 7c16 2 00d8 dw sectorsPerTrack 7c18 2 003e dw numHeads 7c1a 2 000e dd numHiddenSectors 7c1c 4 00000000 dd numSectorsHuge 7c20 4 000d7806 db driveNum 7c24 1 80 db reserved 7c25 1 00 db signature 7c26 1 29 dd volumeID 7c27 4 001c0c23 db volumeLabel 7c2b 11 'NO NAME ' db fileSysType 7c36 8 'FAT ' ===== Here is the boot sector... The first 3 bytes of the BPB are JMP and NOP instructions. 0000:7C00 EB44 JMP START 0000:7C02 90 NOP Here is the rest of the BPB. 0000:7C00 eb449049 424d2032 302e3000 02100100 *.D.IBM 20.0.....* 0000:7C10 02000200 00f8d800 3e000e00 3e000000 *........>...>...* 0000:7C20 06780d00 80002900 1c0c234e 4f204e41 *.x....)...#NO NA* 0000:7C30 4d452020 20204641 54202020 20200000 *ME FAT ..* Additional data areas. 0000:7C30 ........ ........ ........ ....0000 * ..* 0000:7C40 00100000 0000.... ........ ........ *...... * Note: 0000:7c3e (DS:003e) = number of sectors in the FATs and root dir. 0000:7c42 (DS:0042) = number of sectors in the FAT. 0000:7c44 (DS:0044) = number of sectors in the root dir. START: START OF BOOT SECTOR PROGRAM 0000:7C46 FA CLI interrupts off 0000:7C47 33DB XOR BX,BX zero BX 0000:7C49 8ED3 MOV SS,BX SS now zero 0000:7C4B BCFF7B MOV SP,7BFF SP now 7bff 0000:7C4E FB STI interrupts on 0000:7C4F BAC007 MOV DX,07C0 set DX to 0000:7C52 8EDA MOV DS,DX 07c0 Are we booting from a floppy or a hard disk partition? 0000:7C54 803E240000 CMP BYTE PTR [0024],00 is driveNum in BPB 00? 0000:7C59 753D JNZ NOT_FLOPPY jmp if not zero We are booting from a floppy. The Diskette Parameter Table must be copied and altered... Diskette Parameter Table is pointed to by INT 1E. This program moves this table to high memory, alters the table, and changes INT 1E to point to the altered table. This table contains the following data: ????:0000 = Step rate and head unload time. ????:0001 = Head load time and DMA mode flag. ????:0002 = Delay for motor turn off. ????:0003 = Bytes per sector. ????:0004 = Sectors per track. ????:0005 = Intersector gap length. ????:0006 = Data length. ????:0007 = Intersector gap length during format. ????:0008 = Format byte value. ????:0009 = Head settling time. ????:000a = Delay until motor at normal speed. Compute a valid high memory address. 0000:7C5B 1E PUSH DS save DS 0000:7C5C B84000 MOV AX,0040 set ES 0000:7C5F 8EC0 MOV ES,AX to 0040 (BIOS data area) 0000:7C61 26 ES: reduce system memory 0000:7C62 FF0E1300 DEC WORD PTR [0013] size by 1024 0000:7C66 CD12 INT 12 get system memory size 0000:7C68 C1E06 SHL AX,06 shift AX (mult by 64) 0000:7C6B 8EC0 MOV ES,AX move to ES 0000:7C6D 33FF XOR DI,DI zero DI Move the diskette param table to high memory. 0000:7C6F 33C0 XOR AX,AX zero AX 0000:7C71 8ED8 MOV DS,AX DS now zero 0000:7C73 C5367800 LDS SI,[0078] DS:SI = INT 1E vector 0000:7C77 FC CLD clear direction 0000:7C78 B90B00 MOV CX,000B count is 11 0000:7C7B F3 REPZ copy diskette param table 0000:7C7C A4 MOVSB to top of memory Alter the number of sectors per track in the diskette param table in high memory. 0000:7C7D 1F POP DS restore DS 0000:7C7E A11800 MOV AX,[0018] get sectorsPerTrack from BPB 0000:7C81 26 ES: alter sectors per track 0000:7C82 A20400 MOV [0004],AL in diskette param table Change INT 1E to point to altered diskette param table and do a INT 13 disk reset call. 0000:7C85 1E PUSH DS save DS 0000:7C86 33C0 XOR AX,AX AX now zero 0000:7C88 8ED8 MOV DS,AX DS no zero 0000:7C8A A37800 MOV [0078],AX alter INT 1E vector 0000:7C8D 8C067A00 MOV [007A],ES to point to altered diskette param table 0000:7C91 1F POP DS restore DS 0000:7C92 8A162400 MOV DL,[0024] driveNum from BPB 0000:7C96 CD13 INT 13 diskette reset NOT_FLOPPY: Compute the location and the size of the root directory. Read the entire root directory into memory. 0000:7C98 A01000 MOV AL,[0010] get numFAT 0000:7C9B 98 CBW make into a word 0000:7C9C F7261600 MUL WORD PTR [0016] mult by numFatSectors 0000:7CA0 03060E00 ADD AX,[000E] add reservedSectors 0000:7CA4 50 PUSH AX save 0000:7CA5 91 XCHG CX,AX move to CX 0000:7CA6 B82000 MOV AX,0020 dir entry size 0000:7CA9 F7261100 MUL WORD PTR [0011] mult by numRootDirEntries 0000:7CAD 8B1E0B00 MOV BX,[000B] get bytesPerSector 0000:7CB1 03C3 ADD AX,BX add 0000:7CB3 48 DEC AX subtract 1 0000:7CB4 F7F3 DIV BX div by bytesPerSector 0000:7CB6 50 PUSH AX save number of dir sectors 0000:7CB7 03C1 ADD AX,CX add number of fat sectors 0000:7CB9 A33E00 MOV [003E],AX save 0000:7CBC B80010 MOV AX,1000 AX is now 1000 0000:7CBF 8EC0 MOV ES,AX ES is now 1000 0000:7CC1 33FF XOR DI,DI DI is now zero 0000:7CC3 59 POP CX get number dir sectors 0000:7CC4 890E4400 MOV [0044],CX save 0000:7CC8 58 POP AX get number fat sectors 0000:7CC9 A34200 MOV [0042],AX save 0000:7CCC 33D2 XOR DX,DX DX now zero 0000:7CCE E87300 CALL READ_SECTOR read 1st sect of root dir 0000:7CD1 33DB XOR BX,BX BX is now zero 0000:7CD3 8B0E1100 MOV CX,[0011] number of root dir entries DIR_SEARCH: SEARCH FOR OS2BOOT. Search the root directory for the file name OS2BOOT. 0000:7CD7 8BFB MOV DI,BX DI is dir entry addr 0000:7CD9 51 PUSH CX save CX 0000:7CDA B90B00 MOV CX,000B count is 11 0000:7CDD BED501 MOV SI,01D5 addr of "OS2BOOT" 0000:7CE0 F3 REPZ is 1st dir entry 0000:7CE1 A6 CMPSB for "OS2BOOT"? 0000:7CE2 59 POP CX restore CX 0000:7CE3 7405 JZ FOUND_OS2BOOT jmp if OS2BOOT 0000:7CE5 83C320 ADD BX,+20 incr to next dir entry 0000:7CE8 E2ED LOOP DIR_SEARCH try again FOUND_OS2BOOT: FOUND OS2BOOT. OS2BOOT was found. Get the starting cluster number and convert to a sector address. Read OS2BOOT into memory and finally do a far return to enter the OS2BOOT program. 0000:7CEA E335 JCXZ FAILED1 JMP if CX zero. 0000:7CEC 26 ES: get the szie of 0000:7CED 8B471C MOV AX,[BX+1C] the OS2BOOT file 0000:7CF0 26 ES: from the OS2BOOT 0000:7CF1 8B571E MOV DX,[BX+1E] directory entry 0000:7CF4 F7360B00 DIV WORD PTR [000B] div by bytesPerSect 0000:7CF8 FEC0 INC AL add 1 0000:7CFA 8AC8 MOV CL,AL num sectors OS2BOOT 0000:7CFC 26 ES: get the starting 0000:7CFD 8B571A MOV DX,[BX+1A] cluster number 0000:7D00 4A DEC DX subtract 1 0000:7D01 4A DEC DX subtract 1 0000:7D02 A00D00 MOV AL,[000D] sectorsPerCluster 0000:7D05 32E4 XOR AH,AH mutiply 0000:7D07 F7E2 MUL DX to get LBA 0000:7D09 03063E00 ADD AX,[003E] add number of FAT sectors 0000:7D0D 83D200 ADC DX,+00 to LBA 0000:7D10 BB0008 MOV BX,0800 set ES 0000:7D13 8EC3 MOV ES,BX to 0800 0000:7D15 33FF XOR DI,DI set ES:DI to entry point 0000:7D17 06 PUSH ES address of 0000:7D18 57 PUSH DI OS2BOOT 0000:7D19 E82800 CALL READ_SECTOR read OS2BOOT into memory 0000:7D1C 8D360B00 LEA SI,[000B] set DS:SI 0000:7D20 CB RETF "far return" to OS2BOOT FAILED1: OS2BOOT WAS NOT FOUND. 0000:7D21 BE9801 MOV SI,0198 "SYS01475" message 0000:7D24 EB03 JMP FAILED3 FAILED2: ERROR FROM INT 13. 0000:7D26 BEAD01 MOV SI,01AD "SYS02025" message FAILED3: OUTPUT ERROR MESSAGES. 0000:7D29 E80900 CALL MSG_LOOP display message 0000:7D2C BEC201 MOV SI,01C2 "SYS02027" message 0000:7D2F E80300 CALL MSG_LOOP display message 0000:7D32 FB STI interrupts on HANG: HANG THE SYSTEM! 0000:7D33 EBFE JMP HANG sit and stay! MSG_LOOP: DISPLAY AN ERROR MESSAGE. Routine to display the message text pointed to by SI. 0000:7D35 AC LODSB get next char of message 0000:7D36 0AC0 OR AL,AL end of message? 0000:7D38 7409 JZ RETURN jmp if yes 0000:7D3A B40E MOV AH,0E write 1 char 0000:7D3C BB0700 MOV BX,0007 video attributes 0000:7D3F CD10 INT 10 INT 10 to write 1 char 0000:7D41 EBF2 JMP MSG_LOOP do again RETURN: 0000:7D43 C3 RET return READ_SECTOR: ROUTINE TO READ SECTORS. Read sectors into memory. Read multiple sectors but don't read across a track boundary. The caller supplies the following: DX:AX = sector address to read (as LBA) CX = number of sectors to read ES:DI = memory address to read into 0000:7D44 50 PUSH AX save lower part of LBA 0000:7D45 52 PUSH DX save upper part of LBA 0000:7D46 51 PUSH CX save number of sect to read 0000:7D47 03061C00 ADD AX,[001C] add numHiddenSectors 0000:7D4B 13161E00 ADC DX,[001E] to LBA 0000:7D4F F7361800 DIV WORD PTR [0018] div by sectorsPerTrack 0000:7D53 FEC2 INC DL add 1 to sector number 0000:7D55 8ADA MOV BL,DL save sector number 0000:7D57 33D2 XOR DX,DX zero upper part of LBA 0000:7D59 F7361A00 DIV WORD PTR [001A] div by numHeads 0000:7D5D 8AFA MOV BH,DL save head number 0000:7D5F 8BD0 MOV DX,AX save cylinder number 0000:7D61 A11800 MOV AX,[0018] sectorsPerTrack 0000:7D64 2AC3 SUB AL,BL sub sector number 0000:7D66 40 INC AX add 1 0000:7D67 50 PUSH AX save number of sector to rea d 0000:7D68 B402 MOV AH,02 INT 13 read sectors 0000:7D6A B106 MOV CL,06 shift count 0000:7D6C D2E6 SHL DH,CL shift high cyl left 0000:7D6E 0AF3 OR DH,BL or in sector number 0000:7D70 8BCA MOV CX,DX move cyl/sect to CX 0000:7D72 86E9 XCHG CH,CL swap cyl/sect 0000:7D74 8A162400 MOV DL,[0024] driveNum 0000:7D78 8AF7 MOV DH,BH head number 0000:7D7A 8BDF MOV BX,DI memory addr to read into 0000:7D7C CD13 INT 13 INT 13 read sectors call 0000:7D7E 72A6 JB FAILED2 jmp if any error 0000:7D80 5B POP BX get number of sectors read 0000:7D81 59 POP CX restore CX 0000:7D82 8BC3 MOV AX,BX number of sector to AX 0000:7D84 F7260B00 MUL WORD PTR [000B] multiply by sector size 0000:7D88 03F8 ADD DI,AX add to memory address 0000:7D8A 5A POP DX restore upper part of LBA 0000:7D8B 58 POP AX resotre lower part of LBA 0000:7D8C 03C3 ADD AX,BX add number of sector just 0000:7D8E 83D200 ADC DX,+00 read to LBA 0000:7D91 2ACB SUB CL,BL decr requested num of sect 0000:7D93 7FAF JG READ_SECTOR jmp if not zero 0000:7D95 C3 RET return Data not used. 0000:7D90 ........ ....1200 ........ ........ * .. * Messages here. 0000:7D90 ........ ........ 4f532f32 20212120 * OS/2 !! * 0000:7Da0 53595330 31343735 0d0a0012 004f532f *SYS01475.....OS/* 0000:7Db0 32202121 20535953 30323032 350d0a00 *2 !! SYS02025...* 0000:7Dc0 12004f53 2f322021 21205359 53303230 *..OS/2 !! SYS020* 0000:7Dd0 32370d0a 00...... ........ ........ *27... * OS/2 loader file name. 0000:7Dd0 ........ ..4f5332 424f4f54 20202020 * OS2BOOT * Data not used. 0000:7De0 00000000 00000000 00000000 00000000 *................* 0000:7Df0 00000000 00000000 00000000 0000.... *.............. * The last two bytes contain a 55AAH signature. 0000:7Df0 ........ ........ ........ ....55aa * U.* ------------------------(end docs here)-------------------------- -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From shap@viper.cis.upenn.edu Tue May 9 16:26:16 1995 id QAA12092; Tue, 9 May 1995 16:26:15 -0600 (1.36.108.7/16.2) id AA10178; Tue, 9 May 1995 18:21:04 -0400 Date: Tue, 9 May 1995 18:21:04 -0400 From: shap@viper.cis.upenn.edu (Jonathan Shapiro) To: erich@uruk.org Cc: multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu In-Reply-To: <199505092106.OAA00435@uruk.org> (erich@uruk.org) Subject: Re: Multiboot Proposal 0.2 ...[my comment on preferring linear addressing deleted]... > The IDE spec allows drives to implement different numbers of sectors > per track at different places on the disk and do sector > remapping... ??? I'm not sure I see how my using linear addressing will cause this problem. I'm planning on using linear addressing by converting between a logical block number (starting at 0, and represented by a long) and cylinder/head/sector addresses as used by the ... BIOS. I think I was the one confused. >From your comments about wanting to work around the mutually incompatible remapping strategies used by different controllers, I guessed (apparently I misunderstood) that you planned to use some canonical remapping and bypass the controller's remapping altogether. If in fact you did this, your linear sector number and the BIOS linear sector number would be likely to disagree, leading to the incompatibility issue I raised. At the risk of saying something redundant (I haven't read the previous parts of this thread carefully), a quasi-portable approach just occurred to me. Write a byte in your MBR somewhere describing the remapping policy used by the controller. Sector 0 is probably addressable at the same place in all schemes. You can then use the byte to decide what remapping strategy to use to invert the damage done by your controller. Jonathan From erich@uruk.org Tue May 9 18:45:43 1995 id SAA12220; Tue, 9 May 1995 18:45:41 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: shap@viper.cis.upenn.edu (Jonathan Shapiro) cc: multiboot@schirf.cs.utah.edu, goel@cs.columbia.edu Subject: Re: Multiboot Proposal 0.2 In-reply-to: Your message of "Tue, 09 May 1995 18:21:04 EDT." <199505092221.PAA00605@uruk.org> Date: Tue, 09 May 1995 17:40:30 -0700 Sender: erich@uruk.org shap@viper.cis.upenn.edu (Jonathan Shapiro) writes: > At the risk of saying something redundant (I haven't read the previous > parts of this thread carefully), a quasi-portable approach just > occurred to me. Write a byte in your MBR somewhere describing the > remapping policy used by the controller. Sector 0 is probably > addressable at the same place in all schemes. You can then use the > byte to decide what remapping strategy to use to invert the damage > done by your controller. Certainly for basic operation of most of the free OS's, the C/H/S numbers aren't used for anything. I don't know about VSTa, but for FreeBSD/NetBSD/ Linux/the Mach-based OS's, linear block addressing is used. I've been thinking a little about why partitions and such are supposed to be aligned to cylinder/head boundaries, and the best guess I can make is that this is a somewhat artificial result of the fact that in general the MBR/stage1 boot of most (well, all I've ever seen) OS's work with these BIOS calls and the BIOS call can't read between cylinder/head boundaries (it returns a read error). Those that I've been working on automatically break up the reads to fall on the translated cylinder/head boundaries, so that wouldn't be a problem here. FAT & FFS filesystems don't seem to care about which particular cylinder/head/sector translation being used. So, it might be possible when switching geometries (changing the translation mode on a SCSI controller or changing SCSI controllers) to only change the partition table cylinder/head/sector sets to correspond to the new geometry, and the rest would fall into place. I have to think a little bit on this and run some tests, but if a bootloader with the "pure-linear-mapping" function was used as a multi-OS boot, then it would be a framework that could free up translation issues, except for maintining the partition tables for other OS's. Well, at least this is a possible solution for Intel PC's. I still don't know much about other architectures and what kind of partition schemes are used. I'll blissfully let that be somebody else's problem. :-) -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From <@punt.demon.co.uk:dave@humbug.demon.co.uk> Wed May 10 04:43:06 1995 id EAA20211; Wed, 10 May 1995 04:43:05 -0600 10 May 95 10:58 GMT-60:00 10 May 95 10:57 GMT-60:00 id m0s96iL-0003jdC; Wed, 10 May 95 09:01 BST From: Dave Hudson Subject: Re: Multiboot Proposal 0.2 To: erich@uruk.org Date: Wed, 10 May 1995 09:01:48 +0100 (BST) Cc: multiboot@schirf.cs.utah.edu In-Reply-To: <199505100040.RAA00918@uruk.org> from "erich@uruk.org" at May 9, 95 05:40:30 pm X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 3054 Hi, erich@uruk.org wrote: > > Certainly for basic operation of most of the free OS's, the C/H/S numbers > aren't used for anything. I don't know about VSTa, but for FreeBSD/NetBSD/ > Linux/the Mach-based OS's, linear block addressing is used. VSTa's linear block based too. > I've been thinking a little about why partitions and such are supposed to > be aligned to cylinder/head boundaries, and the best guess I can make is > that this is a somewhat artificial result of the fact that in general > the MBR/stage1 boot of most (well, all I've ever seen) OS's work with these > BIOS calls and the BIOS call can't read between cylinder/head boundaries > (it returns a read error). Those that I've been working on automatically > break up the reads to fall on the translated cylinder/head boundaries, so > that wouldn't be a problem here. I think this is largely historical - up to about 1991 a lot of drives only had a couple of kBytes of sector buffer space on-board (some had just 512 bytes), and working with the physical mappings gave some important performance benefits. I know some larger Unix systems used to place duplicate superblocks around a partition so that they were on different physical heads/cylinders to try and improve the chances of recovering from defective sectors. > FAT & FFS filesystems don't seem to care about which particular > cylinder/head/sector translation being used. DOS takes exception to this though if your FAT fs C/H/S parameters don't match those supplied by the BIOS - I forget the exact error message, but it's something like "No System". Once it's been booted though (say off a floppy) everything's fine and you can still use the fs, just not boot it. I don't know how *BSD handles this, but Linux has an ioctl() to get the disk geometry, and this is how I ensured consistency when I wrote mkdosfs. Generally translation's not a problem below 512 MBytes (real computer ones, not disk drive vendor ones :-)) - I've always found it to be the simple: (((C * num_heads) + H) * num_sectors_per_track) + S > So, it might be possible when switching geometries (changing the > translation mode on a SCSI controller or changing SCSI controllers) to > only change the partition table cylinder/head/sector sets to correspond > to the new geometry, and the rest would fall into place. FWIW it's not just SCSI that has this problem - I know of two different ways BIOS writers handled IDE disks larger than 512 MBytes (without going to LBA). > I have to think a little bit on this and run some tests, but if a bootloader > with the "pure-linear-mapping" function was used as a multi-OS boot, then > it would be a framework that could free up translation issues, except for > maintining the partition tables for other OS's. I think this is the important bit - I don't know of any fs used on Intel/PC hardware (I don't know any other architectures well enough to comment) where the C/H/S mapping is important for reading (I can't think of a good reason for a boot loader to be writing). Regards, Dave From erich@uruk.org Wed May 10 08:43:20 1995 id IAA20974; Wed, 10 May 1995 08:43:18 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Dave Hudson cc: multiboot@schirf.cs.utah.edu Subject: Re: Multiboot Proposal 0.2 In-reply-to: Your message of "Wed, 10 May 1995 09:01:48 BST." Date: Wed, 10 May 1995 07:38:16 -0700 Sender: erich@uruk.org Dave Hudson writes: > erich@uruk.org wrote: > > > FAT & FFS filesystems don't seem to care about which particular > > cylinder/head/sector translation being used. > > DOS takes exception to this though if your FAT fs C/H/S parameters don't > match those supplied by the BIOS - I forget the exact error message, but > it's something like "No System". Once it's been booted though (say off a > floppy) everything's fine and you can still use the fs, just not boot it. I > don't know how *BSD handles this, but Linux has an ioctl() to get the disk > geometry, and this is how I ensured consistency when I wrote mkdosfs. I've read the DOS & OS/2 stage 1 boot-loaders plus some others, and they all use C/H/S parameters directly. That means if they're trying to access anything not on cylinder/head zero, it would fail. The DOS boot-loader looks where it thinks the FAT directory is, then searches for the files "IO.SYS" and "MSDOS.SYS", so it makes sense that it would say "No System". I think that once it is operating, it uses calculated linear addressing just like all of the free OS's do. > I think this is the important bit - I don't know of any fs used on Intel/PC > hardware (I don't know any other architectures well enough to comment) where > the C/H/S mapping is important for reading (I can't think of a good reason > for a boot loader to be writing). Well, some MBR replacements that are also multi-OS chain-boot programs set the "Active" partition in the MBR. This is pretty straightforward since it is the first block. I agree that standard FS writing is going a bit far though... it would certainly add a lot of code and bloat. The issue I worry about is if an OS would get confused writing to a filesystem where the extents (linear start and end) are correct, but the C/H/S translation was different than creation time. I tend to think not, since you can back up pretty much every filesystem I've heard of simply by reading the data linearly off of the partition and writing it onto another partition of potentially different configuration. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From baford Wed May 10 15:18:11 1995 id PAA21740; Wed, 10 May 1995 15:18:11 -0600 To: erich@uruk.org cc: multiboot@schirf Subject: Re: Proposal-0.2 In-reply-to: Your message of "Sun, 07 May 95 22:15:10 PDT." <199505080515.WAA25534@uruk.org> Date: Wed, 10 May 95 15:18:11 MDT From: Bryan Ford (Sorry for not being very responsive lately; I've been rather busy. But it's nice to see a lot of things are happening anyway... :-)) >First of all, there is a presumption of an "a.out" style symbol table. >I don't know very much about debugging info, but this seems somewhat >restrictive unless the "syms.count, syms.pointer" model is equivalent >to the others in use. While relocations and external definitions and references and such vary widely in what different object file formats support and how they represent them, "plain" non-relocatable symbols are fairly straightforward and generally represented in about the same way (they're basically just name-value pairs). So I figured we could probably just pick one symbol table format and have boot loaders convert from other formats if they want to load symbol tables from kernel images in those other formats. Since the proposed standard only specifies support for ELF and "augmented" a.out, those are the only two we really have to worry about for now. Alternatively, the standard could separately support both a.out and ELF symbol tables (and those of other formats, if necessary, later) by reserving one bit in the multiboot_info.flags word for each format, and a corresponding set of words in the structure itself. Or just have one bit, and a separate "symbol table format" word. Assuming we decide to pick a particular "standard" symbol table format, maybe that should ELF instead of a.out, since that's the "desired" executable file format specified the standard (the a.out support is just a backward compatibility kludge). What does everyone think? BTW, one thing to note is that the symbol table support as I specified it in 0.2 is screwed up, because it provides no way of finding out where the symbol table stops and the string table begins. So at least that will certainly change in the next revision. :-) >> * EAX must contain the magic value 0x2BADB002; the presence of this value >> indicates to the OS that it was loaded by a MultiBoot-compliant boot >> loader (e.g. as opposed to another type of boot loader that the OS can >> also be loaded from). >> >> * EBX must contain the 32-bit physical address of the multiboot_info >> structure provided by the boot loader (see below). > >... > >> Boot Information Format >> ~~~~~~~~~~~~~~~~~~~~~~~ >> Upon entry to the OS, the EAX register contains the physical address of >> a 'multiboot_info' data structure, through which the boot loader >> communicates vital information to the OS. The OS can use or ignore any >> parts of the structure as it chooses; all information passed by the boot >> loader is advisory only. > >This seems to be a typo. Is EAX the physical address, or EBX ? Oops, yes - just a typo. EBX contains the address. >> ISSUES >> ~~~~~~ >> Should the boot loader zero the kernel's bss segment? > >I don't think it really matters much each way as long as we make a decision. >The only real advantage I can see of the kernel zeroing it's own bss is if >you're loading it in low memory and it would overlap the boot-loader, like >it does with some current ones. It still has to deal with the possibility that the loaded text and data may overlap with itself, though... Also, it has occurred to me that what to do with the bss segment is really an executable format issue, not a boot interface issue - what we should probably be doing is just saying, "kernel images must be in executable format X, and must be loaded however executable images in that format are usually loaded." Currently 'X' is either ELF or a.out, and both specify that bss segments are zeroed. (ELF even potentially allows for multiple "bss" segments, although I don't know if that feature is ever used in practice.) >> Should the PIC be guaranteed to be programmed for the normal BIOS/DOS >> interrupt vectors on entry? > >For low-level boot, the PIC is already set up. For other situations, it >can be a pain. There are some advantages to presuming it is pre-set-up, >though, in that you can have re-programmed the interrupt vectors for >something special... but I don't know how valuable this is. Agreed. Also, I've been thinking that the logical conclusion of this all-encompassing statement that "nothing is initialized" would basically prevent the OS from ever making BIOS callbacks of any kind, because there is no guarantee that the boot loader hasn't mucked with some arbitrary machine rendering the BIOS unusable. I'm changing the specification so that only "processor registers" are undefined, whereas other machine state like the PIC and other hardware components are initialized to "sensible" BIOS-compatible values, presumably what the BIOS initialized them to. Bryan From baford Wed May 10 17:29:23 1995 id RAA22136; Wed, 10 May 1995 17:29:22 -0600 To: erich@uruk.org cc: multiboot@schirf Subject: Re: Multiboot Proposal 0.2 In-reply-to: Your message of "Mon, 08 May 95 18:48:15 PDT." <199505090148.SAA28121@uruk.org> Date: Wed, 10 May 95 17:29:22 MDT From: Bryan Ford >Well, a few things I forgot to add to my comments on "Proposal-0.2" is: > >1) How does the kernel know from which drive it was booted? > >2) How does the kernel know from which partition it was booted? Good points. :-) Sounds like most of the issues are getting resolved just fine. However... >I already wanted to get away from the BSD naming scheme, because it >precludes using this booting system for anything that doesn't have >a BSD disklabel, quite annoying. It turns out that the FreeBSD >people solved a similar problem, so FreeBSD-current (and the lastest >snapshot) now has code to read generic DOS-style partitions and to >distinguish how they lie on the disk. > >Their scheme is based on the concept of "slices". An article explaining >the concept from one of the FreeBSD newsgroups is included at the end of >this message. I'm still a little unclear on the terminology for DOS >extended partitions, so I'll get that part clarified. Yes, this is good - or at least a whole lot better than what BSD was doing before. :-) Personally I prefer simple single-level partitioning like Linux does (i.e. just use DOS partitions), but there are lots of compatibility issues that have to be accounted for... >I suggest passing the "root device" name as a string. The "slice" >naming scheme is sufficient to define precisely what partition to go >to on what drive, including naming funny root devices like CD-ROMs >(i.e. a root device for booting that is different from where the file >came from). I think specifying the "root device" is already well taken care of - presumably it can be part of the kernel command line. In fact, Linux already understands a 'root=...' command-line option that does exactly this. The root device is specified as a string, and interpreted in an OS-specific way. I don't think boot loaders should generally have a concept of a "root device"; that's in the domain of the OS being loaded. On the other hand, a boot loader certainly has a concept of a "boot device", and it's commonly desirable for the OS's root device to default to the boot loader's boot device if no root device has been specified explicitly. Therefore it makes sense to define some standard protocol by which the boot loader can indicate the boot device to the OS, which the OS can use or ignore as appropriate. Unfortunately, I think it is inappropriate to select some particular Unix-like string-based naming scheme for the specification of the boot device. Not only are the different PC-based Unices (free and otherwise) totally inconsistent with each other in these names, they often have no relation to BIOS device numbers, which is what the boot loader is generally more concerned with. For example, some partition on BIOS disk 0x80 could become /dev/wd0a on NetBSD, /dev/wd0s0a on FreeBSD, or /dev/hda1 on Linux; or it may just as easily become /dev/sd0a, /dev/sd0s0a, or /dev/sda1 if that disk turns out to be a SCSI disk which the boot loader was obliviously using through a SCSI BIOS interface. The situation gets even more complicated if we start thinking about network booting. Basically, I think the best we can do on this front is to make the "boot device specification", as passed from the boot loader to the OS, as close as possible to what the boot loader itself uses, and as independent as possible from the OS's device naming scheme. So, for example, a typical boot loader that uses the BIOS disk reading functions would pass the boot device as a BIOS device number (0x80, 0x81, ...) and the OS is responsible for figuring out what device that number corresponds to. For network boot, the boot loader might pass an I/O port number that indicates where the network card is, and the OS can figure out what device is at that address. (Or, more likely perhaps, the OS might simply use bootp to get its root device parameters in that case, and ignore everything the boot loader provides.) Here's my first whack at a boot device specification feature; tell me what you think. + If bit 3 in the multiboot_info.flags word is set, then the 'boot_device' + field is valid, and indicates which BIOS disk device the boot loader loaded + the OS from. If the OS was not loaded from a BIOS disk, then this field + must not be present (bit 3 must be clear). The OS may use this field as a + hint for determining its own "root" device, but is not required to. The + boot_device field is layed out in four one-byte subfields as follows: + + +-------+-------+-------+-------+ + | drive | part1 | part2 | part3 | + +-------+-------+-------+-------+ + + The first byte contains the BIOS drive number as understood by the BIOS + INT 0x13 low-level disk interface: e.g. 0x00 for the first floppy disk or + 0x80 for the first hard disk. + + The three remaining bytes specify the boot partition. 'part1' specifies + the "top-level" partition number, 'part2' specifies a "sub-partition" in + the top-level partition, etc. Partition numbers always start from zero. + Unused partition bytes must be set to 0xFF. For example, if the disk is + partitioned using a simple one-level DOS partitioning scheme, then 'part1' + contains the DOS partition number, and 'part2' and 'part3' are both zero. + As another example, if a disk is partitioned first into DOS partitions, and + then one of those DOS partitions is subdivided into several BSD partitions + using BSD's "disklabel" strategy, then 'part1' contains the DOS partition + number, 'part2' contains the BSD sub-partition within that DOS partition, + and 'part3' is 0xFF. + + DOS extended partitions are indicated as partition numbers starting from 4 + and increasing, rather than as nested sub-partitions, even though the + underlying disk layout of extended partitions is hierarchical in nature. + For example, if the boot loader boots from the second extended partition + on a disk partitioned in conventional DOS style, then 'part1' will be 5, + and 'part2' and 'part3' will both be 0xFF. + + The boot loader is expected to provide only as much boot device information + as it conveniently knows about (i.e. the information it has to collect + anyway for its own functioning). For example, a boot loader that operates + by interpreting a particular file system and loading a file from it, such + as the traditional BSD/Mach boot loader, has to know about partitioning + anyway, and so it should supply full drive and partition information to the + OS. However, a boot loader that boots directly from the master boot record + and has no direct knowledge of partitions, such as LILO, may simply supply + the drive number and set all the 'part' subfields to 0xFF, even if the disk + is in fact partitioned. The OS should be prepared to handle incomplete (or + completely nonexistent) boot device information, and provide the user a way + to override it if necessary. + Bryan From baford Wed May 10 17:32:47 1995 id RAA22157; Wed, 10 May 1995 17:32:47 -0600 To: multiboot@schirf Subject: Disk partition interpretation libraries Reply-to: baford@cs.utah.edu Date: Wed, 10 May 95 17:32:46 MDT From: Bryan Ford BTW, I've placed copies of both VSTa's and FreeBSD's disk partition interpretation libraries in kahlua.cs.utah.edu:/private/boot/misc, for whoever might want to take a look without downloading the full respective source trees... Other than that, I'll leave it to the rest of you to bash things out. :-) Bryan From mycroft@gnu.ai.mit.edu Thu May 11 15:03:50 1995 id PAA01790; Thu, 11 May 1995 15:03:48 -0600 Date: Thu, 11 May 1995 16:56:47 -0400 From: "Charles M. Hannum" To: multiboot@schirf.cs.utah.edu Subject: Various I looked over the last proposal and the mailing list archives, and I have several comments, complaints, and pseudo-random thoughts. This isn't very well organized, but I've already spent too much time writing it. As I see it, the issues that have been brought up here are: 1) The state of the CPU when the kernel is started. The current proposal for this seems adequate, and there doesn't seem to be any disagreement. 2) The format of the loaded images. It makes sense to require an extra piece of info for a.out images; in particular, the lack of a record of the link address has been an annoyance for many years. But what do I get out of being able to read ELF images? Do any of the relevant systems currently use ELF kernels? How do I even test this as the boot program changes over time? Is there an implicit assumption that everyone is going to be moving to ELF? 3) The method of loading other modules. How does the boot program know which extra modules to load for a given kernel? 4) The method of passing info into the kernel. I don't really care whether the `command line' is a single string or an array of strings. But what about the boot device? We already have a hard time determining what NetBSD device number corresponds to a particular BIOS device number, and passing in less information is not going to help. (This is also related to number 5, below.) (And just what is the format of the `command line', anyway? Is this just a free-for-all?) 5) The method of finding the images on the disk. Currently, my boot block allows one to type (for example) `sd(1,a)/foobie/bletch' at the boot prompt. Ignoring for a moment the slightly different syntax, the first part is essentially a NetBSD device name, and the last part is a path within the file system that is presumed to reside on that device. The boot program uses (roughly) the same conventions as the kernel for locating the corresponding partition, parsing the file system, and finding the named image. Now, I'll be the first to admit that this is currently somewhat limited. Notably, the boot program only knows how to read FFS file systems. Clearly this should (and will) be expanded. Some other systems are only able to load images which have been preprogrammed, and for which a list of block numbers has been inserted into the boot program. How do you plan to reconcile these two vastly different approaches? Is there, in fact, going to be any standardization as to how the images are located and read? (If not, what's the point?) What about the file system format? Does it even make sense for systems without a common file system format to share a boot program? There are some other issues that haven't been brought up: 6) Size of the boot program. For a boot program to do all of what's in `Proposal 0.2', and still be able to read my file system and boot an arbitrary image, it's going to be too large for the current NetBSD boot strategy. I would have to split it into two portions, as is done on some other systems (e.g. SunOS's /boot); mostly likely, one smaller portion that loads the other larger portion by reading a series of specific blocks off the disk. What does MultiBoot say about this, and about the interaction between the two portions? Do they both need to be MultiBoot compliant? 7) Portability of the boot program. One would expect the MultiBoot boot program to be portable between systems that use it. Yet, how can this be accomplished? For example, Linux is unlikely to adopt the NetBSD kernel's include files, so a portable boot program can't use those to get constants and structures. Does this mean all of the information has to be duplicated? From my perspective, as someone who already does regular maintenance on 100+ MB of source code, that seems like a lose. Ultimately, the boot program and the kernel should be able to share most of this information. 8) Portability to other architectures. How is this model (ignoring the details of the CPU state on startup) portable to other architectures? For example, there are several systems that can run (or at least crawl) on SPARCs now; do they need a similar standard? What I'm not clear on is what people expect to gain from this standard. Just having two cute data structures doesn't solve any real problems. What real problems does this standard solve? The last draft seems to specifically sidestep the issue of how the images are located and read, and yet this is almost certainly the most important difference between the boot strategies currently employed by the various systems. At some point, the NetBSD/i386 boot program is going to have to be split into two portions anyway. Given that, it seems to me that the best approach for NetBSD to conform to `MultiBoot', as it's currently specified, would be to have the MultiBoot portion just load the secondary boot program, which then provides the current boot prompt, can read an arbitrary image, etc., and can convert the MultiBoot info to whatever format the kernel would prefer. On the other hand, if I'm going to do that, I could almost use LILO for the first stage boot. So where does MultiBoot actually gain me anything? Note that I currently do almost exactly that on several systems which need to have both NetBSD and Linux; use LILO to load either the Linux kernel or the NetBSD boot program. This hasn't ever been a problem. From baford Fri May 12 10:58:26 1995 id KAA10817; Fri, 12 May 1995 10:58:25 -0600 To: "Charles M. Hannum" cc: multiboot@schirf Subject: Re: Various In-reply-to: Your message of "Thu, 11 May 95 16:56:47 EDT." <199505112056.QAA14692@duality.gnu.ai.mit.edu> Date: Fri, 12 May 95 10:58:25 MDT From: Bryan Ford >I looked over the last proposal and the mailing list archives, and I >have several comments, complaints, and pseudo-random thoughts. This >isn't very well organized, but I've already spent too much time >writing it. Thanks; you brought up a lot of good points - many of them revealed a need for clarifications in the document, or for technical revisions. >2) The format of the loaded images. > >It makes sense to require an extra piece of info for a.out images; in >particular, the lack of a record of the link address has been an >annoyance for many years. But what do I get out of being able to read >ELF images? Do any of the relevant systems currently use ELF kernels? >How do I even test this as the boot program changes over time? Is >there an implicit assumption that everyone is going to be moving to >ELF? In general, ELF seems to be becoming the standard executable file format for Unix-like operating systems on the i386 - most of the commercial Unices already use it, Linux is moving to it, the Hurd is moving to it, etc. It's undoubtably a huge improvement over a.out, and it doesn't seem like there are any other major competitors (except possibly COFF). (In fact, as it turns out, I've found that it generally takes significantly less code to load an ELF executable than it does to load an a.out...) Basically, since a.out is likely to remain important for free Unix-like OS's for a while yet, but there's also a fairly big push toward ELF, it seems like the only real option for now is to support both. (However, if you're faced with a decision of making a boot loader "not quite" MultiBoot compliant by not supporting ELF, or else making the boot loader not compliant at all, partial support would be better than none.) >3) The method of loading other modules. > >How does the boot program know which extra modules to load for a given >kernel? Different boot programs will do this in different ways - e.g. by reading a file that lists the boot modules to load, or with additional command-line options to the boot loader, or whatever. Basically, this is part of the boot loader's job and is outside the scope of this standard. >4) The method of passing info into the kernel. > >I don't really care whether the `command line' is a single string or >an array of strings. But what about the boot device? We already have >a hard time determining what NetBSD device number corresponds to a >particular BIOS device number, and passing in less information is not >going to help. (This is also related to number 5, below.) What information, specifically, is still missing? (Note that a few messages ago I proposed adding a boot_device word to the boot_info structure that indicates the boot drive and partition.) >(And just what is the format of the `command line', anyway? Is this >just a free-for-all?) It's indended to be just an arbitrary string as far as the boot loader is concerned, passed from the user, through the boot loader, to the OS. Again, how the user specifies that string to the boot loader is an aspect of the boot loader's user interface, and therefore outside the scope of this standard. >5) The method of finding the images on the disk. > >Currently, my boot block allows one to type (for example) >`sd(1,a)/foobie/bletch' at the boot prompt. Ignoring for a moment the >slightly different syntax, the first part is essentially a NetBSD >device name, and the last part is a path within the file system that >is presumed to reside on that device. The boot program uses (roughly) >the same conventions as the kernel for locating the corresponding >partition, parsing the file system, and finding the named image. > >Now, I'll be the first to admit that this is currently somewhat >limited. Notably, the boot program only knows how to read FFS file >systems. Clearly this should (and will) be expanded. > >Some other systems are only able to load images which have been >preprogrammed, and for which a list of block numbers has been inserted >into the boot program. > >How do you plan to reconcile these two vastly different approaches? >Is there, in fact, going to be any standardization as to how the >images are located and read? (If not, what's the point?) What about >the file system format? Does it even make sense for systems without a >common file system format to share a boot program? These are all aspects of the implementation of the boot loader, which is outside the scope of this standard. Many different types of boot loaders are possible, each possibly working in a different subset of the available OS environments. See below for why such a standard is still useful despite the other incompatibilities between these environments. >There are some other issues that haven't been brought up: > >6) Size of the boot program. > >For a boot program to do all of what's in `Proposal 0.2', and still be >able to read my file system and boot an arbitrary image, it's going to >be too large for the current NetBSD boot strategy. I would have to >split it into two portions, as is done on some other systems >(e.g. SunOS's /boot); mostly likely, one smaller portion that loads >the other larger portion by reading a series of specific blocks off >the disk. > >What does MultiBoot say about this, and about the interaction between >the two portions? Do they both need to be MultiBoot compliant? No - only the interface between the "final" stage of the boot loader and the OS image needs to be MultiBoot compliant. The boot loader itself can be written in as many stages as is convenient, and those intermediate stages can work in whatever way is most appropriate. I'll clarify this in the document. >7) Portability of the boot program. > >One would expect the MultiBoot boot program to be portable between >systems that use it. Yet, how can this be accomplished? For example, >Linux is unlikely to adopt the NetBSD kernel's include files, so a >portable boot program can't use those to get constants and structures. Portability of the boot program itself is not addressed by this proposal, mainly because there are so many different ways boot programs can work. It is to be expected that most boot loaders will only be able to be compiled, installed, and run in certain environments - possibly a small subset of the "MultiBoot-compliant" operating systems. However, there is enough overlap that this standard remains useful. For example, both Linux and VSTa currently have DOS-based boot loaders, but they currently can't load anything but Linux and VSTa, respectively. If at least one of those DOS-based boot loaders becomes MultiBoot compliant, then it will be possible to boot any MultiBoot-compliant OS image from DOS. This is still useful even though the boot loader itself can probably only be compiled under DOS. Similarly with network boot: Currently NetBSD, FreeBSD, and Linux all have different, incompatible network boot processes, each supporting a different (small) subset of the available network cards. this standard should make it possible to create just one boot PROM for a particular network card, and have it be able to load any MultiBoot-compliant OS over the network. Finally, even with disk-based boot loaders, where the differences in file system format tend to limit boot loader portability to a significant degree, there is use in having a boot interface standard. For example, even though Mach, NetBSD, and FreeBSD all use the same file system format, and even though all of their disk-based boot loaders are derived from a common ancestor (the Mach boot loader), none of them can load each other's boot images! In this case, the MultiBoot standard could smooth over the differences in the executable file formats and such, and make these boot loaders compatible with each other once again. In the future, it may be a good idea to try to standardize other OS features as well, in order to allow boot loaders themselves to be more portable: for example, specify a standard file system ioctl (or whatever) to build a block list for a file, so a LILO-like boot loader could be portable to all OS's that support that API. However, that'll have to be a separate step. And even then, I think it'll still remain useful to have a variety of different boot loaders available that use different techniques to find and load the kernel: for example, file system interpretation and block-list methods both have advantages and disadvantages; the user should be able to decide which to use in a particular situation. >8) Portability to other architectures. > >How is this model (ignoring the details of the CPU state on startup) >portable to other architectures? For example, there are several >systems that can run (or at least crawl) on SPARCs now; do they need a >similar standard? Only if they don't already have some kind of reasonably powerful "de facto" boot standard, as I suspect a lot of those do, since there isn't nearly the proliferation of operating systems on workstations as there is on PC's. However, if such a standard is needed on a particular architecture, MultiBoot could certainly be adapted appropriately. Bryan From erich@uruk.org Wed Jun 14 08:43:02 1995 id IAA14370; Wed, 14 Jun 1995 08:43:00 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Multiboot compatible bootloader Date: Wed, 14 Jun 1995 07:45:08 -0700 Sender: erich@uruk.org Hi all. In just under a week, I'm going to release a preliminary version of a boot-loader which will conform to the Multiboot proposal put out here (perhaps plus some commentary... we'll see). I just solved the last major problem (would you believe that machines with an Award BIOS lock up on some BIOS calls if you have any of the protected-mode 32-bit extended bits set in the base pointer register, argh!!). My working name is "flexboot", but my pet name for it is "grub", for GRand Unified Bootloader (pretentious, but I hope justified). Major features: - menu driven interface (with support for serial) - presets for multiple OSes in configuration file - alter any presets (or generate new one completely) before use - both filesystem support and block-lists - as in Multiboot proposal, arbitrary numbers of modules loaded for boot-images supporting the "multiboot format" - chain-loading of DOS, OS/2, Windows NT, other boot-loaders, etc. Initially, the chain-loading, serial interface, and filesystems other than FFS (and possibly FAT) will not be implemented (or not be fully flushed out), but should come soon (ext2fs filesystem is a priority). The menu interface will probably be somewhat unfinished as well. A strong desire will be to make a DOS executable with these feature, and hopefully shared source code, as well. Network booting is also a consideration. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From baford Wed Jun 21 10:45:31 1995 id KAA13453; Wed, 21 Jun 1995 10:45:30 -0600 X-Mailer: exmh version 1.6 4/21/95 To: erich@uruk.org cc: multiboot@schirf Subject: Re: Multiboot compatible bootloader In-reply-to: Your message of "Wed, 14 Jun 95 07:45:08 PDT." <199506141445.HAA00880@uruk.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 21 Jun 95 10:45:30 MDT From: Bryan Ford >Hi all. > >In just under a week, I'm going to release a preliminary version of a >boot-loader which will conform to the Multiboot proposal put out here >(perhaps plus some commentary... we'll see). Great! (BTW, I assume you mean it conforms to the current version of the proposal? The boot stuff in the current mach4 distribution still reflects the original version; I haven't had a chance to update it yet. I guess now would be a good time, so we'll have something for GRUB to load. :-)) >I just solved the last >major problem (would you believe that machines with an Award BIOS lock >up on some BIOS calls if you have any of the protected-mode 32-bit >extended bits set in the base pointer register, argh!!). Interesting... Can you say exactly which calls do this, or are there "a whole bunch" and we should just in general avoid making any BIOS calls with the high 16 bits of EBP nonzero? >My working name is "flexboot", but my pet name for it is "grub", for >GRand Unified Bootloader (pretentious, but I hope justified). I like it. :-) Bryan From erich@uruk.org Wed Jun 21 13:48:21 1995 id NAA14020; Wed, 21 Jun 1995 13:48:18 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Bryan Ford cc: multiboot@schirf.cs.utah.edu Subject: Re: Multiboot compatible bootloader In-reply-to: Your message of "Wed, 21 Jun 1995 10:45:30 MDT." <199506211645.KAA13453@schirf.cs.utah.edu> Date: Wed, 21 Jun 1995 12:49:49 -0700 Sender: erich@uruk.org Bryan Ford writes: > >Hi all. > > > >In just under a week, I'm going to release a preliminary version of a > >boot-loader which will conform to the Multiboot proposal put out here > >(perhaps plus some commentary... we'll see). Actually, it will still be a few more days (grumble), but friday evening I'll have a package on my FTP site. > (BTW, I assume you mean it conforms to the current version of the > proposal? The boot stuff in the current mach4 distribution still reflects > the original version; I haven't had a chance to update it yet. > I guess now would be a good time, so we'll have something for GRUB > to load. :-)) It will conform to the version "0.2", the current version. I'm also waffling on whether to make it able to load a FreeBSD kernel using their current mechanism, but I may pass on that. > >I just solved the last > >major problem (would you believe that machines with an Award BIOS lock > >up on some BIOS calls if you have any of the protected-mode 32-bit > >extended bits set in the base pointer register, argh!!). > > Interesting... Can you say exactly which calls do this, or are there > "a whole bunch" and we should just in general avoid making any BIOS > calls with the high 16 bits of EBP nonzero? Both machines that have Award BIOS that I've tested it on are Pentium machines using BIOS version 4.50G and PCI SCSI adapters (different ones). Actually, there was only one call that I observed this behavior in (out of about 7 calls that I use), the "get BIOS disk parameters" call, int 0x13 ah=8, but only when querying a fixed disk drive. The floppy drives return with the correct data with no apparent problem. It may well have something to do with either the SCSI BIOS hooked in at boot time, or the PCI BIOS. I really don't know. I think at this point, it is just safer to mask out the high 16-bits of EBP. It seems a blanket (and working so far) solution to the problem. At some point, I may be tempted to track it down by disassembling the BIOS or somesuch, but not now. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From erich@uruk.org Fri Jun 23 19:54:35 1995 id TAA04257; Fri, 23 Jun 1995 19:54:33 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: goel@cs.columbia.edu Subject: GRUB snapshot... Date: Fri, 23 Jun 1995 18:55:55 -0700 Sender: erich@uruk.org Well (excuses, excuses), I'm in the middle of a job change right now, so I haven't been making any progress on GRUB. In case any are interested, I put a snapshot on my anonymous FTP site at URL "ftp://uruk.org/private/grub/grub-950623.tar.gz". The lower-level code for everything but the UI and menu-config-file interfaces has been written and tested (the only filesystem included is the BSD FFS, though it will be very easy to add any others given the structure I put in place), though note the licenses for a few functions partially or completely stolen from various *BSD-style bootloaders have NOT been set up correctly, so please note this is a private release! A testing UI exists now, but it is mostly a mock-up, used to exercise the code. It is somewhat sparsely documented right now, concentrated most heavily as comments in the 2 assembly source files. The "docs" subdirectory contains a few useful background documents, and will contain a document on the bootloader itself when it is closer to being finished. Shantanu, I didn't remember if you were on this e-mail list, but in the "shared_src" directory, the header file "pc_slice.h" contains the appropriate definitions for PC-style partition blocks and BSD-style disklabels (and their partitions) inside a PC-partition. The source file "disk_io.c" and main function "open_partition" is appropriate to look at. The format of the input data is: (int) "current_drive", actually one byte representing the BIOS disk device. (int) "current_partition", formatted as 3 bytes, right-justified, with the left-most the first level (i.e. PC-style) partition, the next as the BSD-style disklabel (possibly) present in that partition. Values of 255 indicate an unused slot, with everything else reserved. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From roland@gnu.ai.mit.edu Tue Oct 24 16:16:11 1995 id QAA14172; Tue, 24 Oct 1995 16:16:09 -0600 Date: Tue, 24 Oct 1995 18:13:42 -0400 From: Roland McGrath To: multiboot@schirf.cs.utah.edu Subject: loading files from disk vs locating files on disk X-Nsa-Fodder: radar Croatian PLO class struggle Noriega genetic arrangements Firstly, where are the archives of the multiboot mailing list? Second, in developing the Hurd I have come across what I think is a major problem with the multiboot specification. The spec calls for the boot loader to read from disk into RAM the contents of the second and later files listed in the boot script. This implies the need for sufficient real RAM to hold the entire contents of all those files, since they are uninterpreted (as opposed to the first file, the kernel, which loaded as specified by its object file format). In the case of the Hurd servers, if you don't have a whole lot (a WHOLE lot) of memory and don't mind waiting for it to be filled at each boot, this in effect requires that the server binaries not contain debugging symbols. I have some more complex situations where it is even more of a screw. I think the solution is that instead of interpreting the filesystem to load the files into memory, the boot loader should simply locate all their blocks on the disk through the filesystem structure, and then give the kernel for each file a disk name (or number or whatever) and a list of (logical) block numbers on that disk where the contents reside. Comments? Is anything happening on this booting issue at all these days? The discussion died down months ago. Erich Boleyn was working on a new boot loader based on the multiboot spec, but noone has heard from him in months and he does not reply to my mail any more. From multiboot-request Fri Oct 27 09:46:07 1995 id JAA09480; Fri, 27 Oct 1995 09:46:07 -0600 id JAA27532; Fri, 27 Oct 1995 09:43:31 -0600 id JAA02057; Fri, 27 Oct 1995 09:40:34 -0600 To: Roland McGrath cc: multiboot@schirf Subject: Re: loading files from disk vs locating files on disk In-reply-to: Your message of "Tue, 24 Oct 95 18:13:42 EDT." <199510242213.SAA14527@churchy.gnu.ai.mit.edu> Date: Fri, 27 Oct 95 09:40:34 MDT From: Bryan Ford >Firstly, where are the archives of the multiboot mailing list? kahlua.cs.utah.edu:/private/boot/multiboot-archive >Second, in developing the Hurd I have come across what I think is a major >problem with the multiboot specification. The spec calls for the boot >loader to read from disk into RAM the contents of the second and later >files listed in the boot script. This implies the need for sufficient real >RAM to hold the entire contents of all those files, since they are >uninterpreted (as opposed to the first file, the kernel, which loaded as >specified by its object file format). > >In the case of the Hurd servers, if you don't have a whole lot (a WHOLE >lot) of memory and don't mind waiting for it to be filled at each boot, >this in effect requires that the server binaries not contain debugging >symbols. I have some more complex situations where it is even more of a screw Yes, I can certainly see how this could be a problem. What kind of file sizes are we talking about anyway (with and without debug symbols)? >I think the solution is that instead of interpreting the filesystem to load >the files into memory, the boot loader should simply locate all their >blocks on the disk through the filesystem structure, and then give the >kernel for each file a disk name (or number or whatever) and a list of >(logical) block numbers on that disk where the contents reside. The problem is that this would only work when the kernel is booted from a BIOS-accessible disk device, _and_ when the boot loader is actually in a position to find and provide a block list. This would be no problem for LILO, but wouldn't work for a network boot loader, for example, or for a DOS-based boot loader such as VSTa's (without some really horrible kludging around in the FAT filesystem). In general, the whole purpose of boot modules is to provide the OS with whatever it needs to get going _before_ it has access to external I/O devices and such, which implies that they need to be in main memory. If the OS would have to access I/O devices in order to make use of a boot module, then there would be no point in having the boot loader know about boot modules at all, since the OS could just as easily find whatever data it needs on the root filesystem or whatever itself. As for debugging symbols, it seems to me that in production systems the boot modules would generally be stripped anyway, so it's only really a problem on development systems; and in that case the developer can decide whether to leave debugging stuff in (and pay the price) or not, and how much to leave in. BTW, are you actually running out of memory now with debugging symbols in, or is it just taking too long to load? >Is anything happening on this booting issue at all these days? The >discussion died down months ago. Erich Boleyn was working >on a new boot loader based on the multiboot spec, but noone has heard from >him in months and he does not reply to my mail any more. It's still alive; it's just that everyone is busy doing other things so it's on the back burner at the moment. :-) However, as a bit of advance news, we're just about to release a new version of Mach4 which (finally) includes Shantanu's device drivers and seems pretty stable now; and the kernel is also mostly updated to the latest MultiBoot spec, along with the Linux and BSD boot adaptors. (The remaining differences brought up issues about possible changes that may need to be made to the spec; I'll present them to the list shortly.) I haven't heard anything from Erich Boleyn either. If he remains silent, one of us will probably try to bash his prototype boot loader into a useable form as soon as we have a chance. Bryan From multiboot-request Mon Oct 30 11:31:15 1995 id LAA06267; Mon, 30 Oct 1995 11:31:13 -0700 Date: Mon, 30 Oct 1995 13:27:55 -0500 From: Roland McGrath To: baford@snake.cs.utah.edu Cc: multiboot@schirf.cs.utah.edu Subject: Re: loading files from disk vs locating files on disk In-Reply-To: Bryan Ford's message of Fri, 27 October 1995 09:40:34 MDT <199510271540.JAA02057@snake.cs.utah.edu> X-Shopping-List: (1) Awkward breath cushions (2) Exuberant basic rooster brides (3) Migrant bogs > Yes, I can certainly see how this could be a problem. > What kind of file sizes are we talking about anyway > (with and without debug symbols)? Well, the statically linked ufs is about 400kb vs about 13MB. And don't even get me started on the dynamically linked filesystem and all the shared libs it uses. Let's see, it looks like ext2fs uses eleven: bash# ldd /hurd/ext2fs libdiskfs.so => /lib//libdiskfs.so (0x00018000) libpager.so => /lib//libpager.so (0x00038000) libioserver.so => /lib//libioserver.so (0x0003d000) libfshelp.so => /lib//libfshelp.so (0x0003f000) libports.so => /lib//libports.so (0x00042000) libthreads.so => /lib//libthreads.so (0x00049000) libihash.so => /lib//libihash.so (0x0004f000) libshouldbeinlibc.so => /lib//libshouldbeinlibc.so (0x00051000) libc.so => /lib//libc.so (0x00056000) libmachuser.so => /lib/libmachuser.so (0x0012f000) libhurduser.so => /lib/libhurduser.so (0x00142000) Unstripped: 4112002 libdiskfs.so 776548 libpager.so 118411 libioserver.so 277126 libfshelp.so 1213887 libports.so 213109 libthreads.so 18429 libihash.so 108802 libshouldbeinlibc.so 11652379 libc.so 2749175 libmachuser.so 5488803 libhurduser.so 26728671 total and stripped: 148752 libdiskfs.so 23360 libpager.so 4384 libioserver.so 12196 libfshelp.so 22040 libports.so 23912 libthreads.so 5876 libihash.so 17496 libshouldbeinlibc.so 780388 libc.so 100548 libmachuser.so 125900 libhurduser.so 1264852 total So with everything stripped, you could probably boot with only 2 or 3 MB of RAM. Not that I think it is reasonable to say everything must be stripped or you can't boot. > The problem is that this would only work when the kernel is booted from > a BIOS-accessible disk device, _and_ when the boot loader is actually > in a position to find and provide a block list. That is the only time this information would be relevant. > This would be no problem > for LILO, but wouldn't work for a network boot loader, for example, or > for a DOS-based boot loader such as VSTa's (without some really horrible > kludging around in the FAT filesystem). The DOS loader would locate the blocks on disk through the MSDOS filesystem just as our boot loader will locate them through the ufs or ext2fs filesystem. If the file is gotten over the network, then I don't think this applies. > In general, the whole purpose of boot modules is to provide the OS > with whatever it needs to get going _before_ it has access to external > I/O devices and such, which implies that they need to be in main memory. I don't believe that implication necessarily follows. In the case of Mach, all it needs in main memory to "get going" is the kernel image. The device drivers are in the kernel, so it can read the disk or access the ethernet for whatever else it wants to do. > If the OS would have to access I/O devices in order to make use of a > boot module, then there would be no point in having the boot loader know > about boot modules at all, since the OS could just as easily find > whatever data it needs on the root filesystem or whatever itself. This logic is what leads to the Mach "bundled bootstrap program" scheme that we are trying to replace with something better. Access to i/o devices and access to filesystem information are separate issues. The Mach kernel has access to i/o devices; but it does not have access to any filesystem data, since it contains no code to understand any filesystem format. > As for debugging symbols, it seems to me that in production systems the > boot modules would generally be stripped anyway, so it's only really a > problem on development systems; and in that case the developer can > decide whether to leave debugging stuff in (and pay the price) or not, > and how much to leave in. This is not acceptable. It is not such an unlikely thing that a user will want debugging symbols in the C library. And then the poor schmuck CANNOT BOOT AT ALL because he lacks sufficient RAM to hold the debugging symbols in core for the three seconds before the Hurd is completely booted and they are thrown away. The GNU system will have a booting scheme wherein it is possible to have debugging symbols in binaries and not need infinite RAM to boot. If the multiboot spec cannot meet this requirement, we will have to do something different. > BTW, are you actually running out of memory now with debugging symbols in, > or is it just taking too long to load? I am not booting native now, so I have disk-backed VM by the time I "boot". But if I were booting a dynamically linked Hurd bootstrap filesystem program native on a machine with less than 30 or 40 MB of RAM, I would run out. The purpose of multiple boot modules is that the kernel need not know everything about how to load all the other programs needed to boot. Even if the kernel needs to know something more than a physical memory address to read from, it is still of considerable use that it need to know less than everything about it. For boot modules that are coming off the disk, what Mach really needs to know is something it can translate into a Mach device name, and locations to read from. It can do the rest. For boot modules that are coming over the network, Mach cannot do the protocols itself, and so needs the modules in core. For network booting of the hurd, I expect the boot module needed in core would be a tiny hurd filesystem server implementing IP and TFTP or BOOTP or NFS or whatever using just Mach, sufficient to load the servers. Only this tiny program and ld.so would be needed in core to get going, and from there ld.so would make filesystem RPCs to the tiny server to load just the pieces of the binaries it actually needs to run them. The boot script might specify other files whose locations need to be found by the boot loader and told to the boot modules; just as for files on disk this gives device and locations, for network booting it might give IP address and nfs file handle or somesuch. From multiboot-request Mon Oct 30 12:31:00 1995 id MAA06350; Mon, 30 Oct 1995 12:30:58 -0700 To: Roland McGrath Cc: baford@snake.cs.utah.edu, multiboot@schirf.cs.utah.edu Subject: Re: loading files from disk vs locating files on disk In-Reply-To: Your message of "Mon, 30 Oct 1995 13:27:55 EST." <199510301827.NAA16526@churchy.gnu.ai.mit.edu> Date: Mon, 30 Oct 1995 11:27:01 -0800 From: Andrew Valencia [Roland McGrath writes:] >Well, the statically linked ufs is about 400kb vs about 13MB. And don't >even get me started on the dynamically linked filesystem and all the shared >libs it uses... I'm not sure where this path leads. Given full debugging symbols, doesn't one really want source code to correlate the line number information against? And once you have that, multiple windows? I also have the impression that your boot environment will attach to shared libraries, or was the shared library aspect just to draw out what else must be pulled into the static image? The design philosophy the Hurd appears to have chosen is to pull some OS-level services (filesystem ones, in this case) into the boot functionality. I'm not clear how it was decided that the current "cut" is enough, or whether an iteration of this will pull the source into a position to be accessed from the boot environment. And what would follow from there. FWIW, VSTa has neither filesystems nor disk drivers in the kernel. All it really asks of a boot environment is that a contiguous run of memory be filled with an initial contents. The current DOS loader concatenates the various inital servers itself, but it would be just as easy to do this when building a new OS, and have the loader be nothing more than "load these blocks and jump". If multiboot must indeed grow to encompass the Hurd model, I hope it can be done with strong modularity so that the memory impact and complexity can be avoided for environments which choose not to use these features. Is there any way the filesystem+driver function could be embedded inside the loaded module? The obvious way would create yet another level of boot load, an obvious disadvantage. :-( "My opinions only" Andy Valencia From multiboot-request Mon Oct 30 12:43:33 1995 id MAA06369; Mon, 30 Oct 1995 12:43:32 -0700 Date: Mon, 30 Oct 1995 14:40:28 -0500 From: Roland McGrath To: vandys@cisco.com Cc: baford@snake.cs.utah.edu, multiboot@schirf.cs.utah.edu Subject: Re: loading files from disk vs locating files on disk In-Reply-To: Andrew Valencia's message of Mon, 30 October 1995 11:27:01 -0800 <199510301927.LAA18923@puli.cisco.com> X-Windows: Japan's secret weapon. > [Roland McGrath writes:] > > >Well, the statically linked ufs is about 400kb vs about 13MB. And don't > >even get me started on the dynamically linked filesystem and all the shared > >libs it uses... > > I'm not sure where this path leads. Given full debugging symbols, doesn't > one really want source code to correlate the line number information > against? And once you have that, multiple windows? I also have the > impression that your boot environment will attach to shared libraries, or > was the shared library aspect just to draw out what else must be pulled into > the static image? I don't want the debugging symbols loaded at all! That is my point. The current multiboot spec insists on loading them into core if it is going to know of the existence of the server binary at all. That is the problem. > The design philosophy the Hurd appears to have chosen is to pull some > OS-level services (filesystem ones, in this case) into the boot > functionality. I'm not clear how it was decided that the current "cut" is > enough, or whether an iteration of this will pull the source into a position > to be accessed from the boot environment. And what would follow from there. I don't really understand what you are saying, and I suspect you don't understand the intention of Hurd booting (but I can't tell). > FWIW, VSTa has neither filesystems nor disk drivers in the kernel. All it > really asks of a boot environment is that a contiguous run of memory be > filled with an initial contents. The current DOS loader concatenates the > various inital servers itself, but it would be just as easy to do this when > building a new OS, and have the loader be nothing more than "load these > blocks and jump". That is what the multiboot spec now describes. The problem is that it presumes that for every file that is used in the initial server set up is necessarily reasonable to read that entire file into core. With any type of file other than stripped, statically linked, fairly small programs, this presumption loses. > If multiboot must indeed grow to encompass the Hurd model, I hope it can be > done with strong modularity so that the memory impact and complexity can be > avoided for environments which choose not to use these features. Is there > any way the filesystem+driver function could be embedded inside the loaded > module? The obvious way would create yet another level of boot load, an > obvious disadvantage. :-( I don't want to say anything about any "Hurd model", because I have no idea what you think that phrase means. The model of booting that I want is that a boot loader can read whatever filesystem formats your boot filesystem might be in, and the kernel need not be able to do so, so the boot loader deals with the grokking filesystem formats part of loading other servers. As I have identified, there is a serious practical problem in tying the grokking of filesystem formats to a single "grok filesystem formats and read whole files into core" operation. From multiboot-request Mon Oct 30 13:13:13 1995 id NAA06452; Mon, 30 Oct 1995 13:13:12 -0700 (1.36.108.7/16.2) id AA29572; Mon, 30 Oct 1995 15:13:08 -0500 Date: Mon, 30 Oct 1995 15:13:08 -0500 From: shap@viper.cis.upenn.edu (Jonathan Shapiro) To: roland@gnu.ai.mit.edu Cc: baford@snake.cs.utah.edu, multiboot@schirf.cs.utah.edu In-Reply-To: <199510301827.NAA16526@churchy.gnu.ai.mit.edu> (message from Roland McGrath on Mon, 30 Oct 1995 13:27:55 -0500) Subject: Re: loading files from disk vs locating files on disk > As for debugging symbols, it seems to me that in production systems the > boot modules would generally be stripped anyway, so it's only really a > problem on development systems; and in that case the developer can > decide whether to leave debugging stuff in (and pay the price) or not, > and how much to leave in. This is not acceptable. It is not such an unlikely thing that a user will want debugging symbols in the C library. If I understand matters properly, it seems that two distinct questions are being convolved here. The first is loading the code necessary to get things running. The second is having a mechanism that allows symbols to be loaded for the benefit of the user. Not having looked at the multiboot spec, I'm butting my nose in anyway :-) I can see two reasonable solutions to the debugging info issue: 1. Teach the boot loader enough about the file formats in question that it can load only the code/data out of library files. Since the boot loader already groks the native binary format in order to load the kernel, this doesn't seem to me to involve all that much more work. This would suggest a boot loader that knew about three sorts of objects in the boot image: o executables/librarys, at the address(es) specified inside them. o Uninterpreted files, at an address specified in the load image header. o Ignored files, that exist in the image, but should be ignored by the bootstrap loader. Approximately this approach is taken by the HP IPL loader, and works pretty well. I think this would be a worthwhile facility for the multiboot system to provide in any case. 2. Seperate the symbol tables into distinct files, and teach the debugger about those. For example, the C library symbols could be in libc.sym, which would not be loaded. Debuggers generally have a mechanism to load symbols from a seperate file anyway, particularly where shared libraries are concerned. This was actually the whole point of one of the /proc calls - it provided a mechanism for the debugger to open the file that backed the region, thereby getting access to the symbol table that had not been loaded by the kernel. Either approach reduces the problem from building a block list to providing a file name. If you are concerned about *kernel* debugging, neither of these is quite adequate. A third possibility: Back when we did debugger work at AT&T, we wrote a utility that hauled in a COFF file, grotted through all of the symbol information, and performed a grand unification on the types and the line numbers, eliminating redundant entries. We found that the resulting symbol tables for C programs were on the order of 10% of the size of the originals, and even less for C++ programs (more redundancy). The idea would work equally well on a.out and ELF files. Having such a utility would seem a good thing in general, and might effectively eliminate the memory crunch. Jonathan From multiboot-request Mon Oct 30 13:29:48 1995 id NAA06465; Mon, 30 Oct 1995 13:29:46 -0700 Date: Mon, 30 Oct 1995 15:26:49 -0500 From: Roland McGrath To: shap@viper.cis.upenn.edu Cc: baford@snake.cs.utah.edu, multiboot@schirf.cs.utah.edu Subject: Re: loading files from disk vs locating files on disk In-Reply-To: Jonathan Shapiro's message of Mon, 30 October 1995 15:13:08 -0500 <199510302003.PAA05098@geech.gnu.ai.mit.edu> X-Nsa-Fodder: Cocaine KGB FSF Panama Croatian BATF Legion of Doom > If I understand matters properly, it seems that two distinct questions > are being convolved here. The first is loading the code necessary to > get things running. The second is having a mechanism that allows > symbols to be loaded for the benefit of the user. Not really. I am only concerned with loading the code necessary to get things running. I have no desire for multiboot to say anything on the subject of debugging symbols. I simply want it to be possible to use the current object file arrangements, wherein the most natural thing is for the disk file to contain miscellaneous information (such as the debugging symbols) which are never loaded into core in normal operation. > 1. Teach the boot loader enough about the file formats in question > that it can load only the code/data out of library files. Since > the boot loader already groks the native binary format in order to > load the kernel, this doesn't seem to me to involve all that much > more work. Yes and no. If the boot loader has a notion of a file whose contents are interpreted as an object file, but those contents are handed off to the kernel instead of loaded into core, then the protocol must be extended so it can adequately describe what each section loaded from the file is supposed to be (i.e. in the case of ELF, the information in the program headers along with where each segment was loaded into core). > This would suggest a boot loader that knew about three sorts of > objects in the boot image: What does "boot image" mean? > o executables/librarys, at the address(es) specified inside them. > o Uninterpreted files, at an address specified in the load image > header. > o Ignored files, that exist in the image, but should be ignored by > the bootstrap loader. I don't understand what it means to "exist in the image" but not be loaded into core. > 2. Seperate the symbol tables into distinct files, and teach the > debugger about those. For example, the C library symbols could be > in libc.sym, which would not be loaded. This is already doable with gdb, but it is an unacceptable inconvenience. In the case of libraries, it is not only the debugger which needs symbol table information which should not be loaded into core at runtime; also the linker needs this information to link against the library. > Either approach reduces the problem from building a block list to > providing a file name. This makes no sense to me whatsoever. Please explain. > Back when we did debugger work at AT&T, we wrote a utility that hauled > in a COFF file, grotted through all of the symbol information, and > performed a grand unification on the types and the line numbers, > eliminating redundant entries. We found that the resulting symbol > tables for C programs were on the order of 10% of the size of the > originals, and even less for C++ programs (more redundancy). The idea > would work equally well on a.out and ELF files. I believe GNU ld already does this when possible. > Having such a utility would seem a good thing in general, and might > effectively eliminate the memory crunch. This avoids the issue rather than addressing it. I already know many techniques for kludging around this limitation of the multiboot spec. I want a boot spec that is better so kludges are not required. From multiboot-request Mon Oct 30 14:49:15 1995 id OAA06550; Mon, 30 Oct 1995 14:49:14 -0700 id OAA29784; Mon, 30 Oct 1995 14:46:26 -0700 id OAA28762; Mon, 30 Oct 1995 14:43:21 -0700 To: Roland McGrath cc: multiboot@schirf Subject: Re: loading files from disk vs locating files on disk In-reply-to: Your message of "Mon, 30 Oct 95 13:27:55 EST." <199510301827.NAA16526@churchy.gnu.ai.mit.edu> Date: Mon, 30 Oct 95 14:43:21 MST From: Bryan Ford >> Yes, I can certainly see how this could be a problem. >> What kind of file sizes are we talking about anyway >> (with and without debug symbols)? > >Well, the statically linked ufs is about 400kb vs about 13MB. And don't >even get me started on the dynamically linked filesystem and all the shared >libs it uses. Let's see, it looks like ext2fs uses eleven: OK, I get the idea. :-) Thanks! Now, let's see if we can cut through some of this tangle... Because of the sizes of the initial Hurd servers and libraries, it appears to be completely impractical to have the boot loader load these into RAM before starting the OS; therefore they must be loaded into RAM sometime later, when the Mach kernel is in control of the machine. This means that the kernel's disk device driver must be used (as opposed to the boot loader's, or the BIOS's) to do this loading. Given this, the only question is who should do the filesystem interpretation necessary to find these files on disk: the boot loader, or something later. If the kernel's disk driver must be used to load the initial Hurd servers, I don't see much advantage in using the boot loader to do the file system interpretation, and I see lots of disadvantages, some of which I mentioned in my previous message. Therefore, for the Hurd, I think the best solution is for something _after_ the boot loader (and the MultiBoot interface) to do both the disk device access and the filesystem interpretation necessary to load the first Hurd servers. And lo and behold, we already have such a program: the Mach bootstrap program, which already knows how to interpret three different types of filesystems and can load both ELF and a.out executables. We can simply enhance it by adding the little scripting language you currently use in the boot-from-Lites code, and the kernel can remain as small as it is now. (In fact, we'll be able to remove a few kludges currently in the kernel.) As far as MultiBoot is concerned, there will only be one boot module: the bootstrap program. Fitting it into RAM should not be a problem, and it could even be unloaded after bootup if the Hurd takes over the job of being the default pager. I know we were hoping to get rid of the Mach bootstrap program, but just because of the sheer size of the initial Hurd services I think we'll still need it. And this way the kernel's bootstrap support can remain relatively simple: e.g. it doesn't need to know how to deal with multiple boot modules, where and when to load them, what arguments to start them with, etc. The MultiBoot boot module scheme still serves a useful purpose in cleanly separating the bootstrap program from the kernel, and might be even more useful in the future if Mach's device drivers become dynamically loadable (either Linux-style or VSTa style). >> In general, the whole purpose of boot modules is to provide the OS >> with whatever it needs to get going _before_ it has access to external >> I/O devices and such, which implies that they need to be in main memory. > >I don't believe that implication necessarily follows. In the case of Mach, >all it needs in main memory to "get going" is the kernel image. The device >drivers are in the kernel, so it can read the disk or access the ethernet >for whatever else it wants to do. The device drivers happen to be in the kernel in Mach, but that may not always be the case, and already isn't in the case of VSTa as Andy has already pointed out. If the device drivers aren't in the kernel, they would need to be in RAM-loaded boot modules (or _something_ the boot loader loads into RAM anyway). Bryan From multiboot-request Mon Oct 30 15:10:39 1995 id PAA06587; Mon, 30 Oct 1995 15:10:38 -0700 Date: Mon, 30 Oct 1995 17:07:46 -0500 From: Roland McGrath To: baford@snake.cs.utah.edu Cc: multiboot@schirf.cs.utah.edu Subject: Re: loading files from disk vs locating files on disk In-Reply-To: Bryan Ford's message of Mon, 30 October 1995 14:43:21 MST <199510302143.OAA28762@snake.cs.utah.edu> X-Antipastobozoticataclysm: When George Bush projectile vomits the antipasto on the Japanese. Since the boot loader already has to have a tiny filesystem, it seems preferable to have just that one instead of two tiny filesystems. But I guess if they just share the source code, there is no real waste. Your solution is good in several other ways. OK, I like that. Getting rid of the Mach "bootstrap" program is not an end goal. This makes the boot script concept more fuzzy for me. I can envision several workable possibilities: 1. The boot loader is like the *BSD boot loader and loads just one file, the "kernel boot image". For monolithic systems, this is just the kernel. For Mach it is the kernel with the bootstrap program wrapped inside as was done before. The Mach bootstrap program is told a file name for a boot script and it reads that file and does what it likes. 2. The boot loader understands a boot script file specifying multiple modules, as we have talked about. The boot script specifies the Mach kernel module, and the bootstrap program module with file names, and the boot loader reads these into core. The Mach bootstrap program then reads some other boot script file to decide what to do (like start hurd servers). 3. Like #2, but a single file is used and the boot loader knows to ignore the second half and the Mach bootstrap program knows to ignore the first half. #1 seems fine enough to me. And in that case what's the point of all this multiboot stuff anyhow? The Mach kernel's startup code should just figure out what address it's been loaded at, and where the flags the user typed are, from the *BSD or CMU boot loader, or LILO, or DOS, or whatever else. From multiboot-request Mon Oct 30 16:05:01 1995 id QAA06663; Mon, 30 Oct 1995 16:05:00 -0700 id QAA01901; Mon, 30 Oct 1995 16:02:12 -0700 id PAA28917; Mon, 30 Oct 1995 15:59:07 -0700 To: Roland McGrath cc: multiboot@schirf Subject: Re: loading files from disk vs locating files on disk In-reply-to: Your message of "Mon, 30 Oct 95 17:07:46 EST." <199510302207.RAA17290@churchy.gnu.ai.mit.edu> Date: Mon, 30 Oct 95 15:59:07 MST From: Bryan Ford >This makes the boot script concept more fuzzy for me. I can envision >several workable possibilities: Right - basically, along with the two tiny filesystem interpreters, we also have two points at which the system needs to know _what_ to load. The first is before the MultiBoot interface, and therefore is the boot loader's problem. For example, LILO already uses a little script file called lilo.conf; it could be the same even if it used MultiBoot instead of the 16-bit Linux boot scheme. The script file not only specifies what to load, but also provides default command lines, supports multiple OSes or OS versions, etc. Thus, because the first script does a lot more than just determine what to load next, I think in general it'll still be needed regardless of what happens later. The second-level script, on the other hand, is specific to Mach, and basically specifies Mach-specific actions needed to load the initial Mach "personality". As such I think it is quite different from the first-level script, and deserves to be separate, in the same way that /etc/lilo.conf is separate from /etc/inittab or /etc/rc or /etc/netstart or... It's unfortunate that we have to add "yet another" startup script file, but given that multiple very-different OS's can be booted from MultiBoot, and multiple somewhat-less-different Mach-based OS's can run under Mach, I think this separation is reasonable. (In other words, I'm voting for #2 on your list.) >#1 seems fine enough to me. And in that case what's the point of all this >multiboot stuff anyhow? The Mach kernel's startup code should just figure >out what address it's been loaded at, and where the flags the user typed >are, from the *BSD or CMU boot loader, or LILO, or DOS, or whatever else. MultiBoot is more than just a way to load boot modules: it's intended to clean up and unify the existing bootstrap kludges so that every OS doesn't need to know about every possible boot loader it should be bootable from, and so every OS doesn't have to "own" its own suite of boot loaders. To paraphrase someone we know well, MultiBoot doesn't solve all the world's problems; only some of them. It provides a boot module facility as a cleaner substitute for the increasingly common existing practice of cramming a bunch of little files together into one big image that the boot loader loads into RAM all in one shot, which are then separated out again when the OS starts up. (This behavior is already common to Mach, VSTa, and (sort of) Linux, which I think is sufficient grounds for considering it "common practice".) The boot module mechanism may not be useful in certain situations, and is not guaranteed to be able to get any OS to a particular stage in its boot process without additional help. (e.g. in most OS's, MultiBoot is sufficient to load everything needed to mount the root filesystem; in Hurd that's not the case, but that isn't necessarily a catastrophic deficiency.) The boot module mechanism merely provides a cleaner solution to an existing technique. Bryan From multiboot-request Mon Oct 30 16:40:27 1995 id QAA06724; Mon, 30 Oct 1995 16:40:27 -0700 id QAA02760; Mon, 30 Oct 1995 16:37:38 -0700 id QAA29086; Mon, 30 Oct 1995 16:34:33 -0700 To: multiboot@schirf Subject: GRUB uploaded to kahlua Reply-to: baford@cs.utah.edu Date: Mon, 30 Oct 95 16:34:33 MST From: Bryan Ford BTW, I've placed a copy of Erich's grub-950623.tar.gz snapshot on kahlua.cs.utah.edu:/private/boot, where all the other MultiBoot-related stuff is, for whoever is interested. Bryan From multiboot-request Tue Oct 31 10:53:35 1995 id KAA15638; Tue, 31 Oct 1995 10:53:33 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Multiboot stuff & long absence... Date: Tue, 31 Oct 1995 09:52:24 -0800 Sender: erich@uruk.org Hi all. I just read the list of messages (I haven't even been reading e-mail for a while, very busy with a new job), and I'll send along my comments soon (though Bryan made some good points already). Just saying that I'm still alive... -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Tue Oct 31 12:19:51 1995 id MAA15770; Tue, 31 Oct 1995 12:19:50 -0700 id MAA23228; Tue, 31 Oct 1995 12:16:58 -0700 id MAA08063; Tue, 31 Oct 1995 12:13:52 -0700 To: erich@uruk.org cc: multiboot@schirf Subject: Re: Multiboot stuff & long absence... In-reply-to: Your message of "Tue, 31 Oct 95 09:52:24 PST." <199510311752.JAA19249@uruk.org> Date: Tue, 31 Oct 95 12:13:51 MST From: Bryan Ford >Hi all. Hey, you've risen from the dead! Welcome back. :-) Bryan From multiboot-request Mon Nov 13 17:58:14 1995 id RAA16589; Mon, 13 Nov 1995 17:58:10 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu Subject: Getting back to work (so to speak)... Date: Mon, 13 Nov 1995 16:56:00 -0800 Sender: erich@uruk.org I had gone off into space for a while, as I was in a (what was to me) major legal hassle with Intel over the emulation project that I was gearing up for (I got a job working in the x86 main architecture group). In short, I can continue (with actual progress, I hope :-) working on the bootloader, shared library and emulation projects, as long as I don't write or directly facilitate portions to emulate x86 binaries on other companies' hardware (of course, running Mac binaries on a PC with the HURD is just fine by them). Enough on that topic. I read the e-mail messages that have been flying around, and I agree with Bryan Ford that "a better bootloader" is still a good idea, but I can see Roland's point that it isn't a complete solution for the HURD. It looks like I can continue pretty much from where I was (it was a little farther than the version Bryan uploaded, I had added a simple "block-list" filesystem, and was half-way through a FAT filesystem which I hope to finish this weekend sometime), as the multiboot loader simply passes along a lot of uninterpreted command-lines which the OS can do as it feels like anyway. More later... -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Tue Nov 14 11:12:13 1995 id LAA25448; Tue, 14 Nov 1995 11:12:11 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu Subject: On the Multiboot stuff... In-reply-to: Your message of "Mon, 13 Nov 1995 16:56:00 PST." <199511140056.QAA03567@uruk.org> Date: Tue, 14 Nov 1995 10:10:13 -0800 Sender: erich@uruk.org In my last message I said I could see Roland's point about the HURD being too big to load into core at one shot, but I'm wondering (now that I'm thinking about it :-)... I saw the message with the sizes of the libraries graphically demonstrated by Roland (they are BIG with debug symbols), and so that makes me wonder why we need to load the symbols at all? It seems to me there are several options available: (1) Have an option to the multiboot loader to recognize enough about the various executable types to leave out the symbol tables (since they are generally at the end of the file anyway) when loading them, say via a default switch that can be explicitly toggled to load the whole file. Then when the system comes up, since it knows where the files came from in the filesystem, they can be backed in VM from their original images on disk (which have symbols). This has the distinct advantage of being faster than loading the whole thing plus needing a lot less RAM for the booting stage, and the result is the same once the filesystem comes on line. It also keeps the ability to "boot" to a state where the filsystem or disk code is not alive yet. (2) GDB has the ability to use separate symbol tables from the executable. Use stripped libraries plus either a new feature for a set of "symbol" files or a duplicate unstripped set of shared libraries. I think #1 is a good possibility, and it seems easy enough for me to add. What do people think? (if nothing else, then I can do it as an experiment, as I still think it is the more elegant approach in the long run, but priorities may differ) -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Tue Nov 14 12:37:29 1995 id MAA25610; Tue, 14 Nov 1995 12:37:23 -0700 Date: Tue, 14 Nov 1995 13:51:51 -0500 From: mib@gnu.ai.mit.edu (Michael I. Bushnell, p/BSG) To: erich@uruk.org CC: multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu In-reply-to: erich@uruk.org's message of Tue, 14 Nov 1995 10:10:13 -0800 <199511141810.KAA05219@uruk.org> Subject: Re: On the Multiboot stuff... X-Geek-Code: (V2.1) GCS/J/M/MU/P/S/O>AT d- H-- s-: g+++ p0 !au a- w++ v+++(*) C++$ UB++++$ P--- L 3- E++ N++ K++++ W-- M- V-- po-- Y+(--) t++ 5+ j++ R- G'''' tv+ b+++ !D B-- e+ u++(*) h* f? r n y++ X-Zippy-Says: I smell like a wet reducing clinic on Columbus Day! There are some real problems in implementing the filesystem too, to have it "know" that the loaded pages are part of the disk. In part, this is because the pages (for the filesystem itself, at least) are not actually backed by the filesystem's pagers. This makes a whole new special case to deal with. It's really quite hard, in the long run, and not really worth it at the moment. In my opinion, the most important things for a boot loader are: 1) Be able to load the kernel and at least two initial tasks; 2) Be able to deal with a variety of exec formats; 3) Be able to deal with a variety of filesystem formats and disk partitioning schemes; 4) Be able to deal with a variety of hardware types; and 5) Be able to load all the kernels out there--that is, start things in real mode or protected mode; 16 bit or 32 bit mode, etc. Michael From multiboot-request Tue Nov 14 12:43:54 1995 id MAA25620; Tue, 14 Nov 1995 12:43:51 -0700 Date: Tue, 14 Nov 1995 14:39:00 -0500 From: Roland McGrath To: erich@uruk.org Cc: multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu Subject: Re: On the Multiboot stuff... In-Reply-To: erich@uruk.org's message of Tue, 14 November 1995 10:10:13 -0800 <199511141810.KAA05219@uruk.org> X-Zippy-Says: I'm having fun HITCHHIKING to CINCINNATI or FAR ROCKAWAY!! > (2) GDB has the ability to use separate symbol tables from the executable. > Use stripped libraries plus either a new feature for a set of "symbol" > files or a duplicate unstripped set of shared libraries. This is inconvenient and I already rejected it. But there is a more general issue. It happens to be the case that the only practical problem on any machine anybody in the discussion has would be binaries with debugging info. But for shared libraries, the really ideal thing would be never to read the whole file into core at all, but only the necessary pages as happens in the system normally once booted. > (1) Have an option to the multiboot loader to recognize enough about the > various executable types to leave out the symbol tables (since they > are generally at the end of the file anyway) when loading them, say > via a default switch that can be explicitly toggled to load the whole > file. > Then when the system comes up, since it knows where the files came > from in the filesystem, they can be backed in VM from their original > images on disk (which have symbols). > This has the distinct advantage of being faster than loading the whole > thing plus needing a lot less RAM for the booting stage, and the result > is the same once the filesystem comes on line. It also keeps the > ability to "boot" to a state where the filsystem or disk code is not > alive yet. > > I think #1 is a good possibility, and it seems easy enough for me to add. > > What do people think? > > (if nothing else, then I can do it as an experiment, as I still think it > is the more elegant approach in the long run, but priorities may differ) I have come to agree with Bryan that having yet another stage (the Mach bootstrap program) in this case is worth the complexity. There are many reasons. There are two things that would be avoided by a single-script scheme as we had previously planned: 1. The Mach `bootstrap' program would go away, with its yet other three tiny read-only filesystem implementations. It would be nice for maintenance and general sanity if there were just the one set of tiny filesystems, in the boot loader. 2. There would be just one boot script, which is (hopefully) less confusing than two scripts. However, compared to one simplification for developers and one simplification for OS installers, the two-script scheme I am now pursuing has several very nice advantages. Basically, it is really really nice that the thing reading the files (executables and shared libraries) to load the servers is a Mach user task and not only that, but one that already implements a disk pager. To recap, the recent change is that instead of the same boot script read by the boot loader to load the kernel containing the specifications to load the Hurd servers, that script loads only Mach and the Mach bootstrap/default pager program, pretty much just how things work now. I am now hacking the Mach bootstrap program to interpret yet another boot script of its own, which will specify loading of the Hurd servers. Since this script is read only by bootstrap and not by the boot loader, its syntax need not be so constrained. That is one advantage. I will probably extend the current syntax only a little right now, but the door is open for this syntax being completely redone for whatever goals there might be, without the tight constraints of keeping the boot loader exceedingly simple. The major advantage is that the program reading the server binaries is now the bootstrap program. Since it already contains a disk pager and filesystem interpreter, it will be easy to extend it so that the boot script can specify creation of memory objects backed by the contents of given files. Since it is a (simple) cooperating server program running on Mach, it is very easy for the Hurd to interact with. We can add RPCs to allow the Hurd filesystem server to retrieve the receive rights and all associated state for these memory objects, so it can gracefully take over backing the memory for the text of the servers and their shared libraries. Likewise a fancier Hurd default pager can take over from the simple one; once it is no longer serving any ports, the bootstrap task can die. I am hacking bootstrap right now to handle boot scripts. I expect to have it basically working by tomorrow. Extending it to provide file-backed memory objects and use them for loading will depend on how complicated it is to implement, and is not an immediate priority for Hurd booting. From multiboot-request Tue Nov 14 12:54:11 1995 id MAA25655; Tue, 14 Nov 1995 12:54:11 -0700 id MAA06934; Tue, 14 Nov 1995 12:50:00 -0700 id MAA17458; Tue, 14 Nov 1995 12:49:59 -0700 From: Jay Lepreau To: multiboot@schirf cc: hurd-dev@gnu.ai.mit.edu Subject: Re: On the Multiboot stuff... In-reply-to: Roland McGrath's message of Tue, 14 Nov 95 14:39:00 EST Date: Tue, 14 Nov 95 12:49:58 MST The Sun boot ROM code primarily consists of a Forth interpreter, and I believe this has been the case for a long time. Since there are so many varying needs, maybe that's the way to go: arbitrary extensibility by Turing-completeness. (As opposed to a fixed loader implementation and a configuration language, both of which are bound to need change.) This is certainly in the GNU tradition and is also a Rising Trend. Have you folks considered such an option? From multiboot-request Tue Nov 14 13:08:59 1995 id NAA25708; Tue, 14 Nov 1995 13:08:57 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Roland McGrath cc: multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu Subject: Re: On the Multiboot stuff... In-reply-to: Your message of "Tue, 14 Nov 1995 14:39:00 EST." <199511141939.OAA29903@churchy.gnu.ai.mit.edu> Date: Tue, 14 Nov 1995 12:06:44 -0800 Sender: erich@uruk.org Roland McGrath writes: ...[lots of my comments deleted]... > I have come to agree with Bryan that having yet another stage (the Mach > bootstrap program) in this case is worth the complexity. There are many > reasons. ...[lots of reasons deleted]... > Since it is a (simple) cooperating server program running on Mach, it is > very easy for the Hurd to interact with. We can add RPCs to allow the Hurd > filesystem server to retrieve the receive rights and all associated state > for these memory objects, so it can gracefully take over backing the memory > for the text of the servers and their shared libraries. Likewise a fancier > Hurd default pager can take over from the simple one; once it is no longer > serving any ports, the bootstrap task can die. > > I am hacking bootstrap right now to handle boot scripts. I expect to have > it basically working by tomorrow. Extending it to provide file-backed > memory objects and use them for loading will depend on how complicated it > is to implement, and is not an immediate priority for Hurd booting. I like it. It solves a lot of problems and easily allows HURD-specific extensions that might have been wierd to put into a bootloader. I particularly like the idea of being able to "shift" backing of objects to the filesystem server at some point. I also agree that this is an optimization that doesn't need to happen real soon. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Wed Nov 15 08:44:56 1995 id IAA04616; Wed, 15 Nov 1995 08:44:51 -0700 Date: Wed, 15 Nov 1995 10:40:36 -0500 From: mib@gnu.ai.mit.edu (Michael I. Bushnell, p/BSG) To: roland@gnu.ai.mit.edu CC: erich@uruk.org, multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu In-reply-to: Roland McGrath's message of Tue, 14 Nov 1995 14:39:00 -0500 <199511141939.OAA29903@churchy.gnu.ai.mit.edu> Subject: Re: On the Multiboot stuff... X-Geek-Code: (V2.1) GCS/J/M/MU/P/S/O>AT d- H-- s-: g+++ p0 !au a- w++ v+++(*) C++$ UB++++$ P--- L 3- E++ N++ K++++ W-- M- V-- po-- Y+(--) t++ 5+ j++ R- G'''' tv+ b+++ !D B-- e+ u++(*) h* f? r n y++ X-Tom-Swiftie: "I like amputations," Tom said disarmingly. Date: Tue, 14 Nov 1995 14:39:00 -0500 From: Roland McGrath Cc: multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu There are two things that would be avoided by a single-script scheme as we had previously planned: 1. The Mach `bootstrap' program would go away, with its yet other three tiny read-only filesystem implementations. It would be nice for maintenance and general sanity if there were just the one set of tiny filesystems, in the boot loader. 2. There would be just one boot script, which is (hopefully) less confusing than two scripts. Damn good reasons, if you ask me. Basically, it is really really nice that the thing reading the files (executables and shared libraries) to load the servers is a Mach user task and not only that, but one that already implements a disk pager. We need to make sure, first, that we already know that this will help. The issues in the filesystem are hard to deal with, and unlikely to be much easier with this solution than any other. Can you please come in, and we can make sure there's a good implementation first? Michael From multiboot-request Wed Nov 15 08:45:50 1995 id IAA04622; Wed, 15 Nov 1995 08:45:48 -0700 Date: Wed, 15 Nov 1995 10:41:40 -0500 From: mib@gnu.ai.mit.edu (Michael I. Bushnell, p/BSG) To: lepreau@cs.utah.edu CC: multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu In-reply-to: Jay Lepreau's message of Tue, 14 Nov 95 12:49:58 MST <199511141949.MAA17458@mancos.cs.utah.edu> Subject: Re: On the Multiboot stuff... X-Geek-Code: (V2.1) GCS/J/M/MU/P/S/O>AT d- H-- s-: g+++ p0 !au a- w++ v+++(*) C++$ UB++++$ P--- L 3- E++ N++ K++++ W-- M- V-- po-- Y+(--) t++ 5+ j++ R- G'''' tv+ b+++ !D B-- e+ u++(*) h* f? r n y++ X-Windows: Putting new limits on productivity. From: Jay Lepreau cc: hurd-dev@gnu.ai.mit.edu Date: Tue, 14 Nov 95 12:49:58 MST The Sun boot ROM code primarily consists of a Forth interpreter, and I believe this has been the case for a long time. Since there are so many varying needs, maybe that's the way to go: arbitrary extensibility by Turing-completeness. (As opposed to a fixed loader implementation and a configuration language, both of which are bound to need change.) This is certainly in the GNU tradition and is also a Rising Trend. Have you folks considered such an option? Blech. Eeeeuuuuu. Sigh. Michael From multiboot-request Wed Nov 15 08:57:43 1995 id IAA04647; Wed, 15 Nov 1995 08:57:42 -0700 id m0tFk9K-0003wXC; Wed, 15 Nov 95 07:53 PST X-Authentication-Warning: localhost.tfs.com: Host localhost didn't use HELO protocol To: mib@gnu.ai.mit.edu (Michael I. Bushnell, p/BSG) cc: lepreau@cs.utah.edu, multiboot@schirf.cs.utah.edu, hurd-dev@gnu.ai.mit.edu Subject: Re: On the Multiboot stuff... In-reply-to: Your message of "Wed, 15 Nov 1995 10:41:40 EST." <199511151541.KAA11218@duality.gnu.ai.mit.edu> Date: Wed, 15 Nov 1995 16:53:22 +0100 From: Poul-Henning Kamp > The Sun boot ROM code primarily consists of a Forth > interpreter, and I believe this has been the case > for a long time. > > Have you folks considered such an option? > We might have to actually, as this is the path lined up for PCI in the future as well... -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@ref.tfs.com TRW Financial Systems, Inc. Future will arrive by its own means, progress not so. From multiboot-request Thu Nov 16 09:08:12 1995 id JAA14303; Thu, 16 Nov 1995 09:08:11 -0700 From: more@power.globalnews.com Date: Thu, 16 Nov 1995 15:31:32 GMT To: jfree@caldera.com Reply-To: error2@power.globalnews.com (Do Not Reply. Bounce-backs only) Subject: 1441 CHRP SPEC PUBLISHED - AS THE POWERPC PLATFORM Sender: jfree@caldera.com CHRP SPEC PUBLISHED - AS THE POWERPC PLATFORM (November 15th 1995) It's here - IBM, Motorola and Apple have published the first complete version of what used to be called the Common Hardware Reference Platform, but has now been shortened to the 'PowerPC Platform'. Readers with understanding bookshops can pop down the road and order the 300 page tome, published by Morgan Kaufman. The new document is described by Todd Moore, architecture program manager with IBM's RS/6000 Division, as being much tighter that the PReP specification that it supersedes; "PReP talked much more about recommendations... which lead to confusion" The new platform replaces many of these recommendations with mandatory requirements. In addition the new document specifies a register model for the machine and specifies the way that some registers should be used. So what's new since the general architecture White Paper was published in April? Moore picks out work done on Open Firmware; a new set of 'runtime abstraction services' and support for multiboot as particularly noteworthy. The runtime abstracts define a skinny set of APIs that will help isolate the operating system from differences that manufacturers introduce into their individual implementations of the architecture. OSs will be able to rely on these APIs for standardised power management support, to handle the system clock, PCI configuration, L2 caching flushing, error handling and exceptions and a few other things. This is a completely new addition, not present in PReP. On the Open Firmware side, Moore says that the "bindings" have been more closely specified that previously - defining more exactly how adapters and boot devices should function. Should there be a problem using existing PCI/OpenFirmware cards in the news machines "it shouldn't be a problem" he says, though recommending that manufacturers test them out. One big change over PReP is the formalised support for booting multiple operating systems. The design team has completely rewritten the format of the data held in the machine's non-volatile RAM specifically to allow a choice of multiple operating system boot options. On the subject of how expensive the machines will be to build, Moore is adamant that the additional costs above and beyond PReP are minimal. There is the price of the mandatory ROM socket to take the Macintosh ROM, but other than that he reckons that the additional bits needed to support Mac I/O probably come to around $10-$12. When will the first machines appear? The standard answer from the PowerPC triumvirate is the second half of 1996, however it is worth remembering that FirePower Systems have been making noises about shipping a machine within 90 days of CHRP's announcement. The company hasn't been so vocal about this time scale lately, and we couldn't reach them as we were going to press, however, if the timetable still stands, the first machine would appear on the 11th of January. Make a note in your diary. Another company that should prove quick off the mark is Mac OS licensee Power Computing Corp which says it has been working on a CHRP project for the last six months (see story below). (c) PowerPC News - free by mailing add@power.globalnews.com From multiboot-request Fri Nov 24 15:21:29 1995 id PAA25982; Fri, 24 Nov 1995 15:21:20 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: baford@cs.utah.edu Subject: Multiboot & GRUB... Date: Fri, 24 Nov 1995 14:18:52 -0800 Sender: erich@uruk.org Hi all. I'm just about done getting my first version of the GRUB bootloader working. A more detailed announcement will follow when I've wrung out the bugs in a day or two, but I will say that in addition to supporting native NetBSD and FreeBSD executable formats, a reasonable subset of i386-elf (i.e. most anything you're ever likely to try booting) is recognized directly and bootable with any of the the NetBSD, FreeBSD, or "multiboot" boot information methodologies. The UI has you set of a lot of things by hand right now, but I figure this level of manual control is good for us OS geeks, and a snazzy menu UI will be added for the masses at a later time anyway. In the meantime, I've noticed that the "boot-proposal-0.3" had a few problems that make the "a.out kludge" unusable. The root of the problem is the assumption that the text and data segments are contiguous in memory as well as the file. Most of the BSD boot-images now require that the data segment is loaded at a 4K page boundary, but it is uncertain to me if this should always be done, and when. Therefore, I have slightly modified the "boot-proposal-0.3", changing the "a.out kludge" parameters and adding an optional "don't align an a.out" flag bit. It is optional since a bootloader might understand it correctly and work fine anyway. The diffs are included at the end of this message. [For Bryan Ford: The "multiboot" adapter you set up for Mach4 isn't going to work in it's current form. It is connected right onto the kernel, but when a bsd image is created, the parts added on check for the MULTIBOOT_MAGIC at some point in the file before the beginning of the kernel's text section. I would suggest putting the "multiboot" structure at the beginning of the bsd&linux boot adapters.] ---------------------(start diff of "boot-proposal-0.3")----------------- diff -ruN boot-proposal-0.3.orig boot-proposal-0.3 --- boot-proposal-0.3.orig Fri Nov 24 12:32:28 1995 +++ boot-proposal-0.3 Fri Nov 24 14:00:26 1995 @@ -178,9 +178,9 @@ segment after the "real" executable header. It _must_ be contained completely within the first 8192 bytes of the executable file, and must be longword (32-bit) aligned. These rules allow the boot loader to find and -synchronize with the text segment in the a.out file without knowing -beforehand the details of the a.out variant. The layout of the header is -as follows: +synchronize with the text and data segments in the a.out file without +knowing beforehand the details of the a.out variant. The layout of the +header is as follows: +-------------------+ 0 | magic: 0x1BADB002 | (required) @@ -188,9 +188,10 @@ +-------------------+ 8 | header_addr | (present if flags[16] is set) 12 | load_addr | (present if flags[16] is set) -16 | load_end_addr | (present if flags[16] is set) -20 | bss_end_addr | (present if flags[16] is set) -24 | entry_addr | (present if flags[16] is set) +16 | text_end_addr | (present if flags[16] is set) +20 | data_end_addr | (present if flags[16] is set) +24 | bss_end_addr | (present if flags[16] is set) +28 | entry_addr | (present if flags[16] is set) +-------------------+ All fields are in little-endian byte order, of course. The first field is @@ -224,8 +225,15 @@ support other executable formats, such as particular a.out variants, but are not required to. +If bit 17 in the flags word is set, then the text and data segments must +be contiguous in memory. Many a.out variants presume the data segment +must be aligned on a page (4KB) boundary. A compliant boot loader should +either recognize this option or directly support the a.out format in +question. + All of the address fields enabled by flag bit 16 are physical addresses. -The meaning of each is as follows: +The presumption is made that the text and data segments are contiguous +in the OS image. The meaning of each is as follows: header_addr Contains the address corresponding to the beginning of the @@ -240,11 +248,17 @@ (header_addr - load_addr). load_addr must be less than or equal to header_addr. -load_end_addr - Contains the physical address of the end of the data segment. - (load_end_addr - load_addr) specifies how much data to load. This - implies that the text and data segments must be consecutive in the - OS image; this is true for existing a.out executable formats. +text_end_addr + Contains the ending physical address of the text segment. + *NOTE*: The data segment is likely to be page aligned, which + implies that the starting address is *not* text_end, but + the next page (4KB) boundary. + +data_end_addr + Contains the ending physical address of the data segment. + *NOTE*: The data segment is likely to be page aligned, which + implies that the length of the data segment is *not* + (data_end - text_end), but (data_end - PAGE_ALIGN(text_end)). bss_end_addr Contains the physical address of the end of the bss segment. The @@ -252,7 +266,7 @@ occupies to avoid placing boot modules and other data relevant to the OS in that area. -entry +entry_addr The physical address to which the boot loader should jump in order to start running the OS. ----------------------(end diff of "boot-proposal-0.3")------------------ -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Mon Nov 27 00:54:13 1995 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: hurd-dev@gnu.ai.mit.edu, mach4-users@cs.utah.edu, roland@gnu.ai.mit.edu Subject: GRUB version 0.0 available Date: Sun, 26 Nov 1995 23:51:22 -0800 Sender: erich@uruk.org Hi all. I've got a workable (gasp!) version of my new bootloader going, and it is solid enough to have other people experiment with it now, I think. The readme is included at the end for your perusal, but I'd like to stress that this is basically a private release to members of these lists for testing purposes. After the licenses have been fixed up, etc. people can have at it as they please. It is available via anonymous ftp from: "ftp://uruk.org/private/grub/grub-951126.tar.gz" It (in theory) can boot images in "a.out" of various flavors and ELF formats. The limitations on ELF are currently that the "program header" exist inside the first 8Kbytes of the file, which is pretty uniformly true. It only supports booting in 32-bit mode, so chain-loading other OS's like DOS, Windows NT, Linux, etc. are not supported yet. [Roland: I tried it out on NetBSD, and it sort of tried to come up, but I'd like you or someone else who has NetBSD to try it out] -------------------------(start GRUB README)------------------------- GRUB -- GRand Unified Boot-loader version 0.0 by Erich Boleyn This is a relatively private release, of the 0.0 version (i.e. pre-alpha), in other words, caveat emptor! The licenses in the files are *not* correctly set up yet, so please respect the possibility that I didn't write the code and haven't included the correct license yet. If you just want to try booting things, get a floppy, and execute your OS's equivalent of (this is for FreeBSD): dd if=stage1/stage1 of=/dev/fd0d bs=512 count=1 dd if=stage2/stage2 of=/dev/fd0d bs=512 seek=1 count=20 This will mess up the filesystem on the floppy (if there was any), but will allow you to try out the boot-loader pretty painlessly. The UI is pretty straightforward. I'll add a snazzy end-user-friendly menu thingy later on, but this is the most useful kind of thing for now, though some might argue I made it *too* manually driven. NOTE: You can type filenames like: "hd(0,a)/blah" even though you've set root filesystem to something else. It allows files to be grabbed from anywhere you want to boot a working system. NOTE: The information the main boot-loader uses to figure out where it is is based on the boot drive (passed in a well-known register by the BIOS), and embedded data near the beginning of "asm.S". In particular, the value of "install_partition" is what is used to determine where on the disk it should go to by default. The format of this variable is exactly the same as the partition parts of the "boot device" described in the "boot-proposal-0.4" document. It is currently set up to look for the "a" partition in the first BSD slice of the booting disk. I have tested the UI, filesystem, and booting code out reasonably, but there's likely to be problems. A very small list is included in the file "BUGS". The "TODO" is too big and vague to have included at the moment. The FFS filesystem is complete and apparently working well. No menus or other snazzy things are supported yet. It is completely manually driven, and manually installed (i.e. use "dd" to put it on your disk). The boot-information-formats that are supported are: - FreeBSD (boots correctly) - NetBSD (comes through device recognition, then quickly reboots before I can see what is going on, (grumble)) - multiboot (described in the "docs/boot-proposal-0.4") Executable types supported are: - multiboot header: If you choose the "multiboot" type and have a "multiboot" header with the "a.out-kludge" option selected, then it should be recognized, but I haven't tested it. This is only supported with the "multiboot" boot-information-format. If an unrecognized multiboot flag is found, then it gives up. (the current Mach4-UK02p21 does *not* have a correct multiboot header as given by the bsd boot adaptor) - ELF: All loadable program segments will be placed in memory at the requested addresses, and extra space in segments zeroed out. Debug sections are currently ignored. The only ELF image I have tried to boot is a Mach i386-gnu build, which partially worked, but I haven't figured out if it is a bug in my code or Mach's. It at least is getting recognizable error codes from Mach's bsd boot-adapter. The "segtype=" notation for listing the segments loaded stands for: s = starting address of the segment, l = disk size of the segment (limited by a memory size parameter), and z = extra memory size not from the disk (this is zeroed). It decides on the text or data segments based on which one contains the entry address... not that it really matters. - a.out (FreeBSD and NetBSD variants): Marrying these together in a consistent fashion is debatable, so I currently treat them exactly as their original boot-loaders do (including ancillary formats not normally used). Debugging symbols are loaded. Here is a description of what executable pieces are included with the source code: - "stage1/stage1": This is the first stage boot-loader, and it fits into one sector on a floppy or hard disk. It is currently set up to load one contiguous set of sectors starting at logical sector 1 (the second sector) and going for 32 sectors, loading it at address 0x8000 (32Kbytes). Typing "make floppy" in this directory on a FreeBSD system will put it in the first sector. It is set up to work with a "stage2" directly right now (the load address is the key issue here). You can patch the binary directly, or look at the assembly source to note where pretty much at the end just before the partition table a set of numbers controlling this stuff resides. If you need to - "ffs_stage1_5/ffs_stage1_5": This is a sort of middleman program, that is set up to load the file "hd(0,a)/grub/grub" at 0x8000 and start it. It is linked at 0x2000 (i.e. if you want to load it with stage1, you have to change where stage1 loads things). NOTE: This doesn't work currently. I've included it mostly so that people can get a feel for the code if they want. - "stage2/stage2": This is the main boot-loader. It is linked at 0x8000. --------------------------(end GRUB README)-------------------------- -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Mon Nov 27 10:58:01 1995 id KAA21256; Mon, 27 Nov 1995 10:57:59 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: mach4-users@cs.utah.edu Subject: Mach multiboot Date: Mon, 27 Nov 1995 09:55:06 -0800 Sender: erich@uruk.org Scratch the statement I made about needing a different boot-adaptor for multiboot, I just found the multiboot-specific images for Mach4: kernel: "kernel/kernel" module-1: "bootstrap/bootstrap" They appear to be set up correctly, so I'll try booting them tonight. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Wed Nov 29 02:40:19 1995 id CAA02772; Wed, 29 Nov 1995 02:40:17 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: GRUB: newer version Date: Wed, 29 Nov 1995 01:37:14 -0800 Sender: erich@uruk.org Any of those interested in the bootloader I'm working on, I just fixed a few major bugs not listed in the "BUGS" file, and put another snapshot in "ftp://uruk.org/private/grub/" as "grub-951128.tar.gz". They prevented loading executables using the "a.out"-kludge method, plus I fixed a bug in the print routine. Every executable type I've tried to load so far (freebsd native, netbsd native, Mach NMAGIC, Mach a.out-kludge, and a few elf programs) now seem to load correctly (to the level that I can test), but Mach a.out-kludge and Mach elf still don't work... but I'm not really sure if it's my code or Mach's. Anyway, check every week or so for updates. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Thu Nov 30 11:52:20 1995 id LAA19919; Thu, 30 Nov 1995 11:52:18 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: mach4-users@cs.utah.edu Subject: Mach4 multiboot changes & new GRUB Date: Thu, 30 Nov 1995 10:49:02 -0800 Sender: erich@uruk.org The changes I sent for the multiboot proposal a while ago don't really need to be made, so I took them out of my local versions, and now the only thing I can see is that the boot-proposal-0.3 needs some improved language for the part describing a symbol table. The "boot-proposal-0.4" in the GRUB "docs" directory now has that one paragraph changed in it, but is otherwise identical. The GRUB bootloader now boots correctly (at least as of early this morning) with nearly all the executable types available to me (NetBSD reboots itself apparently after looking at my FreeBSD and/or GNU HURD filesystem layouts and choking on it... there's a really quick message that passes by before it dies and it clears the screen, and the ELF version of Mach4 with the bsd boot adapter hasn't been tried fo the last day or so, but it got far enough along to suggest it still might not work, though other ELF images do work... which suggests the problem lies in how the bsd boot- adapter works (Bryan might be able to comment here)). The new version is at: "ftp://uruk.org/private/grub/grub-951130.tar.gz". Multimodule booting is now working for both a.out and ELF formats, and a set of changed files for Mach4-UK02p21 are available at: "ftp://uruk.org/private/grub/UK02p21-mb-changes.tar.gz" The two caveats if you want to boot Mach4 as separate modules "kernel" and "bootstrap" using multiboot (and/or just use the changes listed above): - The top-of-memory parameter has been wired to 16MB in "model_dep.c", but will be fixed soon (it's easy to find and change to what you need). - The root partition must be specified on the kernel command-line in a manner described in the README file included with the changes above. This will also be fixed. Have fun. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Mon Dec 4 17:23:07 1995 id RAA27097; Mon, 4 Dec 1995 17:23:04 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: GRUB comments? Date: Mon, 04 Dec 1995 16:19:19 -0800 Sender: erich@uruk.org I've noticed that a bunch of people have FTP'ed the pre-alpha version of GRUB that I made available. I have a bunch of avenues available to continue development, and while I'd like to see them all explored, some people might need to use some features more critically than others... ...so I'm interested in hearing people's comments/feedback/requests on what's there now and what they'd like to see. As a general guide, my current rough outline is (in this order): (1) add more filesystem & device support - add FAT & ext2fs - add "gzip"-style uncompression (automatic, just give it the name of the compressed file) (?) - add network boot support (2) add menu-interface for pre-set options (3) add booting/chain-loading of 16-bit modules/OSes (4) make a version which runs from DOS (i.e. you could read files from a CD-ROM under DOS for booting) -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Mon Dec 4 18:12:01 1995 id SAA27178; Mon, 4 Dec 1995 18:11:59 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Re: GRUB comments? In-reply-to: Your message of "Mon, 04 Dec 1995 16:19:19 PST." <199512050019.QAA26113@uruk.org> Date: Mon, 04 Dec 1995 17:08:19 -0800 Sender: erich@uruk.org In my last message, I wrote: ...[feature wanna-be list mostly deleted]... > - add "gzip"-style uncompression (automatic, just give it the name of > the compressed file) There's 3 basic avenues I can see for this kind of thing (it doesn't add much to the size of the bootloader if it is done right) for boot modules, and the kernel pretty clearly would always be needed to be uncompressed as it is loaded (how else do you run it): (1) Continue to laod boot-modules as they are, as just a block of bytes with no decompression done. (2) Decompress all "gzip"-compressed files automatically, (make it a feature of the filesystem, basically). (3) Add a "decompression" flag to the file-name notation, so that it can be done selectively. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Thu Dec 7 14:49:59 1995 id OAA26030; Thu, 7 Dec 1995 14:49:58 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu, mach4-users@cs.utah.edu Subject: Error in GRUB README Date: Thu, 07 Dec 1995 13:45:35 -0800 Sender: erich@uruk.org Hi all. For those using/interested in using the bootloader I'm working on, I made a serious goof in the README file in the part explaining how to just use it off of a floppy. The mistake was that at the end of the second "dd" command, you should take out the "count=20" completely, as this would only copy 10Kbytes of the file onto the disk, and the file is about 13Kbytes long. dd if=stage2/stage2 of=/dev/fd0d bs=512 seek=1 count=20 ^^^^^^^^ I won't be ready to release another snapshot for about two weeks, but that one will have FAT filesystem and automatic "gzip"-style decompression for kernels. I may have Linux booting and the ext2fs filesystem as well, but I'm really not sure on that one (the Linux booting is NOT using the specially compressed image, but the full-size kernel... which after running through "gzip" is smaller anyway). -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Dec 29 14:37:37 1995 id OAA09525; Fri, 29 Dec 1995 14:37:35 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: ELF symbol table... Date: Fri, 29 Dec 1995 13:38:34 -0800 Sender: erich@uruk.org In thinking about how to add loading of an ELF symbol table to the multiboot proposal (and GRUB, my bootloader), it seems almost easier to me to make it load all of the normally ignored sections, which would be the most flexible option. An example would be using a custom section type. For those who know the ELF format, the idea would be to load the section header into memory, then fill in the "memory location" entry for each of the normally ignored sections, then pass this to the kernel. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Dec 29 15:56:03 1995 id PAA09731; Fri, 29 Dec 1995 15:56:02 -0700 Date: Fri, 29 Dec 1995 17:55:42 -0500 From: Roland McGrath To: erich@uruk.org Cc: multiboot@schirf.cs.utah.edu Subject: Re: ELF symbol table... In-Reply-To: erich@uruk.org's message of Fri, 29 December 1995 13:38:34 -0800 <199512292138.NAA10042@uruk.org> X-Windows: Dissatisfaction guaranteed. I know the ELF format too well, but I don't understand what you are proposing. > In thinking about how to add loading of an ELF symbol table to the > multiboot proposal (and GRUB, my bootloader), it seems almost easier > to me to make it load all of the normally ignored sections, which > would be the most flexible option. An example would be using a custom > section type. I don't understand. Are you suggesting reading into core all section contents for sections that are not covered by any PT_LOAD program header? > For those who know the ELF format, the idea would be to load the section > header into memory, then fill in the "memory location" entry for each > of the normally ignored sections, then pass this to the kernel. Sorry, this doesn't make anything any clearer for me. From multiboot-request Fri Dec 29 16:37:56 1995 id QAA09840; Fri, 29 Dec 1995 16:37:54 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Roland McGrath cc: multiboot@schirf.cs.utah.edu Subject: Re: ELF symbol table... In-reply-to: Your message of "Fri, 29 Dec 1995 17:55:42 EST." <199512292255.RAA16449@churchy.gnu.ai.mit.edu> Date: Fri, 29 Dec 1995 15:38:47 -0800 Sender: erich@uruk.org My explanation was lacking in specifics, so here they are... Roland McGrath writes: > > In thinking about how to add loading of an ELF symbol table to the > > multiboot proposal (and GRUB, my bootloader), it seems almost easier > > to me to make it load all of the normally ignored sections, which > > would be the most flexible option. An example would be using a custom > > section type. > > I don't understand. Are you suggesting reading into core all section > contents for sections that are not covered by any PT_LOAD program header? Yes. > > For those who know the ELF format, the idea would be to load the section > > header into memory, then fill in the "memory location" entry for each > > of the normally ignored sections, then pass this to the kernel. > > Sorry, this doesn't make anything any clearer for me. Load the section header table referred to by the ELF header, and for each section header entry whose "sh_flags" bit SHF_ALLOC is zero, load the corresponding section into memory, then set the "sh_addr" value to this memory address (normally for such sections it would be zero). It is allowed for a section header table to not be present, and in these cases the bootloader wouldn't load anything. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Dec 29 17:08:43 1995 id RAA09894; Fri, 29 Dec 1995 17:08:37 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: roland@gnu.ai.mit.edu cc: multiboot@schirf.cs.utah.edu Subject: Re: ELF symbol table... In-reply-to: Your message of "Fri, 29 Dec 1995 15:38:47 PST." <199512292338.PAA10313@uruk.org> Date: Fri, 29 Dec 1995 16:09:29 -0800 Sender: erich@uruk.org Erich Boleyn writes: > It is allowed for a section header table to not be present, and in these > cases the bootloader wouldn't load anything. My sloppy speech again. The program header table is always present, so it is used to load the program's executable code/data (the sections that would be marked with the SHF_ALLOC bit set to 1 if a section header table was present) in all cases. If a section header table exists, then it will try to load the sections with the SHF_ALLOC bit set to 0 plus the table itself, as previously described. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Dec 29 18:20:59 1995 id SAA10351; Fri, 29 Dec 1995 18:20:58 -0700 Date: Fri, 29 Dec 1995 20:20:35 -0500 From: Roland McGrath To: erich@uruk.org Cc: multiboot@schirf.cs.utah.edu Subject: Re: ELF symbol table... In-Reply-To: erich@uruk.org's message of Fri, 29 December 1995 15:38:47 -0800 <199512292338.PAA10313@uruk.org> X-Windows: It could happen to you. I see. With full -g info, the extra sections could be quite large. In the case of Mach, it is probably only useful for the boot loader to pre-load the symbol tables of kernel modules (at present only one kernel module is supported), not of the user modules (at present only the bootstrap/default-pager program). So perhaps the boot loader should have configuration options to load only some symbol tables to save core and time at boot. From multiboot-request Sat Dec 30 10:43:29 1995 id KAA18033; Sat, 30 Dec 1995 10:43:27 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Roland McGrath cc: multiboot@schirf.cs.utah.edu Subject: Re: ELF symbol table... In-reply-to: Your message of "Fri, 29 Dec 1995 20:20:35 EST." <199512300120.UAA16862@churchy.gnu.ai.mit.edu> Date: Sat, 30 Dec 1995 09:44:19 -0800 Sender: erich@uruk.org Roland McGrath writes: > I see. With full -g info, the extra sections could be quite large. Yes. > In the > case of Mach, it is probably only useful for the boot loader to pre-load > the symbol tables of kernel modules (at present only one kernel module is > supported), not of the user modules (at present only the > bootstrap/default-pager program). So perhaps the boot loader should have > configuration options to load only some symbol tables to save core and time > at boot. Right now, the bootloader loads the entirety of the user modules anyway. The usefulness of only interpreting the "kernel module" is that it allows any file to be loaded into memory. It seems possible to add a "if recognized format, perform naive strip" flag. For a.out it would zero the symbols value in the a.out header and ignore anything beyond the end of the data area. For ELF, the simplest thing would be to only load items covered in the program header table and zero out the references to the section header table that are in the ELF header. Adding a kernel symbol-table flag (use same flag as above) is pretty easy, though. Both this and the decompression option (for user modules) seems to imply I need yet more notation to cover those cases. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Mon Jan 8 16:06:38 1996 id QAA07648; Mon, 8 Jan 1996 16:06:37 -0700 id QAA06597; Mon, 8 Jan 1996 16:05:51 -0700 id PAA00962; Mon, 8 Jan 1996 15:59:57 -0700 To: erich@uruk.org cc: multiboot@schirf Subject: Re: ELF symbol table... In-reply-to: Your message of "Fri, 29 Dec 95 13:38:34 PST." <199512292138.NAA10042@uruk.org> Date: Mon, 08 Jan 96 15:59:57 MST From: Bryan Ford Well, sorry about my long absence; I'm just now trying to get caught up with everything that's been happening during the holiday season... Erich, it looks like you've been doing some great stuff. I grabbed grub-951210 and scanned through it (although I haven't had a chance to try it myself yet), and it looks really good. Just what we need. I also looked at your patches to UK02p21, and we'll integrate them as soon as one minor issue is worked out: the top-of-memory problem which you already know about. :-) Basically, as I was updating Mach to be "mostly" compatible with MultiBoot-0.3, I discovered what seems like sort of a catch-22 situation. When the OS first starts running, it needs to find out how much physical memory is available in order to set up kernel page tables and such. However, finding out how much physical memory is available requires a BIOS call (or else a potentially unreliable manual memory probe), and it's difficult to make BIOS calls before the OS's basic VM system is set up properly. Because of the lack of this one small piece of information, it appears necessary to implement essentially two different BIOS call mechanisms in the OS: one to perform that single memory size call in real mode before the VM system is set up, the other to perform "normal" BIOS calls for other purposes after the VM system is up. Therefore, my question is, given that any boot loader under the sun could trivially perform that BIOS call and pass the memory size to the OS, should we just add a physical memory size parameter to the MultiBoot standard in order to simplify OS's that boot from MultiBoot boot loaders? I certainly don't want us to start sliding down the slippery slope of making the boot loader pass more and more "useful but non-critical" information to the OS; other OS's may be structured so as to be able to make that initial memory size determination more easily (or not rely on having it until later), and I even have missed a simple, clean solution possible in Mach. However, I believe that Linux, for one, also wants to set up its kernel page tables very early (i.e. before BIOS calls would be straightforward to perform), and given the nature of typical x86 OS's I suspect this is the case in other systems as well. If this particular lack of information is going to be the cause of major grief, then I think we should just include it in MultiBoot as a special exception to the rule. So, what do others think? Specifically, will this also be a problem in ? (I know it isn't in FreeBSD since FreeBSD finds memory size using a manual probe anyway, but I don't know about the others...) Do you think the additional practical convenience of knowing physical memory size from the start is worth a minor kludge in the standard? Or am I just being lazy? :-) Thanks! Bryan From multiboot-request Mon Jan 8 16:30:30 1996 id QAA07710; Mon, 8 Jan 1996 16:30:30 -0700 id QAA07144; Mon, 8 Jan 1996 16:29:43 -0700 id QAA01060; Mon, 8 Jan 1996 16:23:50 -0700 To: erich@uruk.org cc: multiboot@schirf Subject: Re: ELF symbol table... In-reply-to: Your message of "Fri, 29 Dec 95 13:38:34 PST." <199512292138.NAA10042@uruk.org> Date: Mon, 08 Jan 96 16:23:49 MST From: Bryan Ford >In thinking about how to add loading of an ELF symbol table to the >multiboot proposal (and GRUB, my bootloader), it seems almost easier >to me to make it load all of the normally ignored sections, which >would be the most flexible option. An example would be using a custom >section type. As far as I'm concerned that should be fine. It suffers from the obvious potential memory efficiency problems, but on the other hand, how much of a typical ELF executable is devoted to data that is neither loaded code/data/whatever nor debug information? (In other words, how much space do those extra non-loaded, non-debug sections really take in normal executables, given that you're loading the code, data, and debug sections anyway?) Anyway, go ahead and work out a debug symbol table provision for ELF however you think is best, and add it to the spec; we can then bash on it further if necessary. BTW, I put your Proposal-0.4 (with minor grammatical fixes, e.g. "it's" -> "its" :-) ) into the MultiBoot FTP directory. Thanks a lot! Bryan From multiboot-request Sun Jan 14 21:16:40 1996 id VAA04343; Sun, 14 Jan 1996 21:16:35 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Re: ELF symbol table... Date: Sun, 14 Jan 1996 20:17:09 -0800 Sender: erich@uruk.org I've been gone since the 8th, and my modem did something mysterious in my absence, so e-mail generally did not get received unless it was sent as of the 10th or 11th. If I missed anything in that time period (I got Bryan's two messages on the 8th, and Andy's reply on the 10th). Bryan Ford writes: > Well, sorry about my long absence; > I'm just now trying to get caught up with everything > that's been happening during the holiday season... I understand how that goes (see above). ...[on the memory probe issue]... As Andy said in his reply, the bootstrapping issue is relatively easy, as we can read the NVRAM settings... or I can just pass it in the bootloader as a good starting value to use... since I need to get the information anyway for purposes of loading modules into RAM, passing on the information for the low-level bootstrap seems reasonable. I'll add the code back in for my next snapshot, which will include the ELF symbol table stuff. > So, what do others think? Specifically, will this also be a problem in > ? (I know it isn't in FreeBSD since FreeBSD finds memory > size using a manual probe anyway, but I don't know about the others...) > Do you think the additional practical convenience of knowing physical > memory size from the start is worth a minor kludge in the standard? > Or am I just being lazy? :-) Unfortunately, this only works for up to 65MB of RAM (saturating 16-bit value in kilobytes). FreeBSD's memory probe only occurs on the amount of RAM detected, currently. For some of the newer and bigger systems, neither a memory probe (holes in memory used by memory mapped devices above 65MB), nor the 65MB limit, is acceptable. It turns out that there is a newer BIOS interface that gives you a complete memory map of the machine, including relatively arbitrary amounts of RAM installed. I have a copy of the documentation for this interface. It is sufficiently hairy that you'd not want to put it in a bootloader, anyway. For other reasons, I wanted to see a real BIOS interface in Mach4 (a BIOS disk interface for losers like me who don't want to set up an IDE disk just for testing OSes :-). More on some of this stuff later... Anyway, this is all fine to run after the VM system has been past it's first initialization. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Jan 19 10:13:01 1996 id KAA20831; Fri, 19 Jan 1996 10:12:58 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Zack Weinberg: Mach4 bootloader: Four stages? Date: Fri, 19 Jan 1996 09:13:29 -0800 Sender: erich@uruk.org I'm forwarding this to the list, as it appears appropriate (and the guy asked)... ------- Forwarded Message Date: Thu, 18 Jan 1996 14:45:39 -0500 (EST) From: Zack Weinberg To: Bryan Ford cc: erich@uruk.org Subject: Mach4 bootloader: Four stages? In-Reply-To: <199601171942.MAA18863@snake.cs.utah.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > >I have skills appropriate to either of these projects. Could you let me > >know what is currently being done on them and how I might help? > > Great! Well, if you haven't seen it already, check out the MultiBoot > standard project and the GRUB boot loader - both are available by FTP [...] > FreeBSD, and Linux. (Mach4 already has it.) Contact me or ask on the > list if you have any questions. OK, I've caught up on the mailing list and I've skimmed the GRUB code... I can't really help much with the OS- and architecture- specific code, because I don't have anything to test it on. I could easily flesh out the menuing code, which is +/- undone as of the 950623 snapshot (surely there's a later one?) It seems to me that some effort should be made to make GRUB architecture-independent as far as possible. Mach already runs on multiple systems; Linux I know is being ported to PowerPC and Alpha machines; I'm sure similar things are being done to {Free,Net}BSD and VSTa. The GRUB design, however, relies on a *lot* of assumptions about the system, which AFAIK are only true of IBM PC's: the pCHS/lCHS/LBA mess, the format of a Master Boot Record, and the _existence_ of the BIOS, for example. Also, the Hurd-specific module-loading problems and the earlier debate about available lists suggest, to me anyway, that the module-loading part of the MultiBoot spec is bogus. Module loading, IMO, is a task that should be carried out by the OS, or an OS-specific boot stage if necessary, _not_ an OS-independent bootloader. I'd like to suggest a four-stage sequence for booting, which deals with both these problems: Stage 1: Architecture specific. (This corresponds to Erich's stage 1, and the MBR code on a PC.) Takes control of the computer from whatever's in ROM -- theoretically, this could even *be* the ROM. Needs to fit in the MBR or equivalent space; performs setup to make the other stages happy, finds stage 1 in some fixed location and loads it in. This could perhaps also be the part contained in a 'switch to other OS' executable. Stage 2: Filesystem specific. (Erich's stage 1.5.) Interprets the filesystem, or accesses the network, to find stage 3 and its config file in a standard location (/boot/grub.3, /boot/grub.conf or something like that -- reserve 'grub' for the reconfiguration executable in case we need one) and loads it. Also provides file-access services to stage 3. Stage 3: Generic. (Erich's stage 2.) Displays the boot menu, allows the user to select their favorite OS and pass parameters to it -- as before. Then calls on stage 2 to locate and load the stage 4 module associated with that OS (see below), and passes it the OS's parameters. Stage 4: OS-specific. There's one of these for every operating system; it interprets the parameters handed it, reads its associated kernel and modules into memory, and hands off control. (Naming suggestion: /boot/grub.4..) E.g. for Linux this can be a stupid program that loads and uncompresses /vmlinuz and throws the parameters at it, but for the Hurd it's the module loader roland and mib were discussing. This mechanism can also be used to coerce 'unfriendly' OS's like the Microsoft ones into loading w/o having to deal with chaining: we simply write a stage 4 that puts the machine back into the state they expect and then mimics their standard boot code. For example, DOS v5 can be loaded by a stage 4 that reverts to 16-bit mode, reads the first three sectors of IO.SYS, and hands off control. Questions: Should stage 4 be allowed to use the stage 2 file access facilities to read the disk/network? Should stage 3 communicate with stage 4 by the MultiBoot protocol, or some other method? My opinion on these two is: #1: yes. If we allow stage 4 to use stage 2's file access facilities, then only stage 2 needs to worry about talking to devices; this reduces considerably both the size of stage 4 and the amount of assembly language needed there. #2: sort of. We need to extend it a bit so that stage 3 can specify modules to be loaded to stage 4, and tell it where to find stage 2's file functions. With those changes, however, it should work just fine. zw feel free to post this to the multiboot list; and yes, I'd like to be added. ------- End of Forwarded Message -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Jan 19 11:18:07 1996 id LAA20958; Fri, 19 Jan 1996 11:17:58 -0700 Date: Fri, 19 Jan 1996 13:13:44 -0500 From: Roland McGrath To: erich@uruk.org Cc: multiboot@schirf.cs.utah.edu Subject: Re: Zack Weinberg: Mach4 bootloader: Four stages? In-Reply-To: erich@uruk.org's message of Fri, 19 January 1996 09:13:29 -0800 <199601191713.JAA03008@uruk.org> X-Zippy-Says: I HAVE to buy a new ``DODGE MISER'' and two dozen JORDACHE JEANS because my viewscreen is ``USER-FRIENDLY''!! Sounds good to me, though it really has to do with the design of GRUB much more than with the Multiboot interface standard. Everything up through stage 3 is an implementation detail (and his implementation structure sounds good for GRUB). The purpose of the concept of this multiboot standard is to avoid the miriad separate "stage 4" programs he suggests. But that is still a good structure to deal with non-"standard" OS's like DOS et al. GRUB can have one "stage 4" for booting multiboot-compliant kernels, and other emulation "stage 4"s for booting other OS's as they expect to be booted. On the other hand, perhaps "stage 3" should talk to "stage 4" via the multiboot protocol, and then "stage 4" is the "kernel", which might be an emulation loader for another non-multiboot kernel. The issue for multiboot is if it does in fact encode i386-specific things. It should be generalized for other machines. But it need not be greatly similar on all machines, as long as it is useful and sensical on each. Having a multiboot magic number and machine-type spec in a known place is sufficient similarity for any future possibility of a multiboot-compatible loader that groks kernels for multiple machine types. From multiboot-request Wed Jan 24 18:05:39 1996 id SAA09861; Wed, 24 Jan 1996 18:05:34 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Zack Weinberg cc: multiboot@schirf.cs.utah.edu Subject: Re: Mach4 bootloader: Four stages? In-reply-to: Your message of "Thu, 18 Jan 1996 14:45:39 EST." Date: Wed, 24 Jan 1996 17:04:52 -0800 Sender: erich@uruk.org Sorry for taking so long to respond... First, I forwarded your message to the "multiboot" e-mail list. My response follows. Zack Weinberg writes: > OK, I've caught up on the mailing list and I've skimmed the GRUB code... > > I can't really help much with the OS- and architecture- specific code, > because I don't have anything to test it on. I could easily flesh out the > menuing code, which is +/- undone as of the 950623 snapshot (surely > there's a later one?) There's a much later one. Look at my anonymous FTP site on my main machine "uruk.org", in "/private/grub/grub-951210.tar.gz". > It seems to me that some effort should be made to make GRUB > architecture-independent as far as possible. Mach already runs on > multiple systems; Linux I know is being ported to PowerPC and Alpha > machines; I'm sure similar things are being done to {Free,Net}BSD and > VSTa. The GRUB design, however, relies on a *lot* of assumptions about > the system, which AFAIK are only true of IBM PC's: the pCHS/lCHS/LBA > mess, the format of a Master Boot Record, and the _existence_ of the BIOS, > for example. I very much agree that GRUB is specific to the IBM/Intel PC architecture. Some of it's operation depends on this. I not only don't own a PowerPC or Alpha box, I don't know anyone who does who also has the technical specs on the boot-process (no-one has stepped forward to help with these other architectures, even in spirit). I suppose this is somewhat beside the point, but unless we understand what really is common or not, a lot of work would be wasted on speculation that might cause problems with those architectures anyway. I think some of you guys at Utah know the info on the PA-RISC bootup process. Providing at least general details of the process would be helpful here. -> begin aside Here is the place to note that I am an Intel employee, and I do have easy access to much of this data about PC's (all of it is public anyway). I'm willing to help work toward a multi-architecture booting scheme that is consistent, but the likelihood is that I will not own any hardware that isn't an IBM/Intel PC arch box for quite a while (Intel's SMP P6 boxes have a LOT of bang for the buck)... and neither will I have access to such hardware. -> end aside > Also, the Hurd-specific module-loading problems and the earlier debate > about available lists suggest, to me anyway, that the module-loading part > of the MultiBoot spec is bogus. Module loading, IMO, is a task that > should be carried out by the OS, or an OS-specific boot stage if > necessary, _not_ an OS-independent bootloader. Module-loading isn't bogus if you want to do boot-time loadable disk drivers. There is a way to get around this on an x86, but it involves writing a V86-mode BIOS driver for the kernel (which I may do for Mach anyway, but not for bootloader functions). I don't know if there are such services for all the other machines that the kernel (or a bootloader) can take advantage of. > I'd like to suggest a four-stage sequence for booting, which deals with > both these problems: > > Stage 1: Architecture specific. (This corresponds to Erich's stage 1, > and the MBR code on a PC.) > Takes control of the computer from whatever's in ROM -- theoretically, > this could even *be* the ROM. Needs to fit in the MBR or equivalent > space; performs setup to make the other stages happy, finds stage 1 in > some fixed location and loads it in. This could perhaps also be the part > contained in a 'switch to other OS' executable. Essentially correct. > Stage 2: Filesystem specific. (Erich's stage 1.5.) > Interprets the filesystem, or accesses the network, to find stage 3 and > its config file in a standard location (/boot/grub.3, /boot/grub.conf or > something like that -- reserve 'grub' for the reconfiguration executable > in case we need one) and loads it. Also provides file-access services to > stage 3. This is optional, and depends on the filesystem. On some, you can't fit all of the filesystem and other system-specific code into it, as it is too big. > Stage 3: Generic. (Erich's stage 2.) > Displays the boot menu, allows the user to select their favorite OS and > pass parameters to it -- as before. Then calls on stage 2 to locate and > load the stage 4 module associated with that OS (see below), and passes > it the OS's parameters. The real purpose of separating out these parts is to allow a small, but still filesystem-aware, part which can load the generic portion. Quite a bit of machine/filesystem-specific stuff is still loaded here. There's also the point that making it "truly" generic might bloat it yet more. It's already gotten pretty big, and that's just writing this in C and allowing the state of the filesystem part to be manipulated easily. > Stage 4: OS-specific. > There's one of these for every operating system; it interprets the > parameters handed it, reads its associated kernel and modules into > memory, and hands off control. (Naming suggestion: /boot/grub.4..) > E.g. for Linux this can be a stupid program that loads and uncompresses > /vmlinuz and throws the parameters at it, but for the Hurd it's the module > loader roland and mib were discussing. First of all, the major number of OS's people are likely to boot between are on x86 boxes. Second, there's machine-specific info that needs to be passed to all of at least the x86 varieties (to my knowledge). I guess in your model, the OS-specific parts would know how to access the machine-specific parts. I suppose I could break this all apart and make a kind of "loadable OS- specific module" that uses the generic loader functions. What I want to know is what use is this in the short term? Heck, some of this stuff will change in the future to the point where it should be re-written anyway, so going too generic before more details are worked out seems a waste. > This mechanism can also be used to coerce 'unfriendly' OS's like the > Microsoft ones into loading w/o having to deal with chaining: we simply > write a stage 4 that puts the machine back into the state they expect and > then mimics their standard boot code. For example, DOS v5 can be loaded > by a stage 4 that reverts to 16-bit mode, reads the first three sectors > of IO.SYS, and hands off control. That's what a "chain"-module really is, is it not? I already have a facility in process for non-multiboot (nor free/net-bsd/etc.) OS's to do a chain-load process of sorts. > Questions: > Should stage 4 be allowed to use the stage 2 file access > facilities to read the disk/network? In this case, it wouldn't really work. > Should stage 3 communicate with stage 4 by the MultiBoot protocol, or > some other method? It can't, as the multiboot protocol doesn't pass enough information for some OS's. If you really want such genericism for x86 boxes, just do it all from DOS. (there is a free version of DOS being worked on) I don't know of any such simple low-level loader/OS running on other boxes. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Thu Feb 1 16:05:45 1996 id QAA26035; Thu, 1 Feb 1996 16:05:05 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Multiboot Proposal Date: Thu, 01 Feb 1996 15:04:52 -0800 Sender: erich@uruk.org Hi all. After thinking about several issues, I've made a bunch of revisions to the Multiboot proposal, almost all in the information passed to the OS via the "multiboot_info" structure. I've included the revised document at the end of this message. Unless I get any real objection, this is the model I'll write the interface to my bootloader with. It solves some of the sticky problems that were remaining, in particular: 1) Adds memory information passed to the OS (both the standard lower/upper memory ones plus the new format for mapping out multiple non-contiguous memory regions up to 2^64 bytes in size. The newer one doesn't work on all motherboards). I added a bit to the multiboot_header.flags word to require at least minimal information on memory configuration. 2) Allows passing of symbol information to an OS with an ELF kernel. 3) Disambiguated BIOS disk parameter (took out part saying that the OS cannot rely on the information in this parameter... if it cannot be provided fully, it really should not be set). I haven't revised anything at or past the "EXAMPLE CODE" heading, though I was planning to do so. Also, the organization of the fields in the multiboot_info structure became somewhat scrambled as I played around with them... if anyone wants me to, I can rearrange them to be compatible with the previous case, but I don't worry about it for now. Thanks! Erich Boleyn -------------------(start "boot-proposal-0.4" here)--------------- MultiBoot Standard PRELIMINARY PROPOSAL Version 0.4 6/26/95 (tweaked 10/22/95) (tweaked again until 12/23/95) (ESB revised until 1/30/96) MOTIVATION ~~~~~~~~~~ Every OS ever created tends to have its own boot loader. Installing a new OS on a machine generally involves installing a whole new set of boot mechanisms, each with completely different install-time and boot-time user interfaces. Getting multiple operating systems to coexist reliably on one machine through typical "chaining" mechanisms can be a nightmare. There is little or no choice of boot loaders for a particular operating system - if the one that comes with the OS doesn't do exactly what you want, or doesn't work on your machine, you're screwed. While we may not be able to fix this problem in existing commercial operating systems, it shouldn't be too difficult for a few people in the free OS communities to put their heads together and solve this problem for the popular free operating systems. That's what this standard aims for. Basically, it specifies an interface between a boot loader and a operating system, such that any complying boot loader should be able to load any complying operating system. This standard does NOT specify how boot loaders should work - only how they must interface with the OS being loaded. TERMINOLOGY ~~~~~~~~~~~ Throughout this document, the term "boot loader" means whatever program or set of programs loads the image of the final operating system to be run on the machine. The boot loader may itself consist of several stages, but that is an implementation detail not relevant to this standard. Only the "final" stage of the boot loader - the stage that eventually transfers control to the OS - needs to follow the rules specified in this document in order to be "MultiBoot compliant"; earlier boot loader stages can be designed in whatever way is most convenient. The term "OS image" is used to refer to the initial binary image that the boot loader loads into memory and transfers control to to start the OS. The OS image is typically an executable containing the OS kernel. The term "boot module" refers to other auxiliary files that the boot loader loads into memory along with the OS image, but does not interpret in any way other than passing their locations to the OS when it is invoked. SCOPE AND REQUIREMENTS ~~~~~~~~~~~~~~~~~~~~~~ Architectures ~~~~~~~~~~~~~ This standard is primarily targetted at PC's, since they are the most common and have the largest variety of OS's and boot loaders. However, to the extent that certain other architectures may need a boot standard and do not have one already, a variation of this standard, stripped of the x86-specific details, could be adopted for them as well. Operating systems ~~~~~~~~~~~~~~~~~ This standard is targetted toward free 32-bit operating systems that can be fairly easily modified to support the standard without going through lots of bureaucratic rigmarole. The particular free OS's that this standard is being primarily designed for are Linux, FreeBSD, NetBSD, Mach, and VSTa. It is hoped that other emerging free OS's will adopt it from the start, and thus immediately be able to take advantage of existing boot loaders. It would be nice if commercial operating system vendors eventually adopted this standard as well, but that's probably a pipe dream. Boot sources ~~~~~~~~~~~~ It should be possible to write compliant boot loaders that load the OS image from a variety of sources, including floppy disk, hard disk, and across a network. Disk-based boot loaders may use a variety of techniques to find the relevant OS image and boot module data on disk, such as by interpretation of specific file systems (e.g. the BSD/Mach boot loader), using precalculated "block lists" (e.g. LILO), loading from a special "boot partition" (e.g. OS/2), or even loading from within another operating system (e.g. the VSTa boot code, which loads from DOS). Similarly, network-based boot loaders could use a variety of network hardware and protocols. It is hoped that boot loaders will be created that support multiple loading mechanisms, increasing their portability, robustness, and user-friendliness. Boot-time configuration ~~~~~~~~~~~~~~~~~~~~~~~ It is often necessary for one reason or another for the user to be able to provide some configuration information to the OS dynamically at boot time. While this standard should not dictate how this configuration information is obtained by the boot loader, it should provide a standard means for the boot loader to pass such information to the OS. Convenience to the OS ~~~~~~~~~~~~~~~~~~~~~ OS images should be easy to generate. Ideally, an OS image should simply be an ordinary 32-bit executable file in whatever file format the OS normally uses. It should be possible to 'nm' or disassemble OS images just like normal executables. Specialized tools should not be needed to create OS images in a "special" file format. If this means shifting some work from the OS to the boot loader, that is probably appropriate, because all the memory consumed by the boot loader will typically be made available again after the boot process is created, whereas every bit of code in the OS image typically has to remain in memory forever. The OS should not have to worry about getting into 32-bit mode initially, because mode switching code generally needs to be in the boot loader anyway in order to load OS data above the 1MB boundary, and forcing the OS to do this makes creation of OS images much more difficult. Unfortunately, there is a horrendous variety of executable file formats even among free Unix-like PC-based OS's - generally a different format for each OS. Most of the relevant free OS's use some variant of a.out format, but some are moving to ELF. It is highly desirable for boot loaders not to have to be able to interpret all the different types of executable file formats in existence in order to load the OS image - otherwise the boot loader effectively becomes OS-specific again. This standard adopts a compromise solution to this problem. MultiBoot compliant boot images always either (a) are in ELF format, or (b) contain a "magic MultiBoot header", described below, which allows the boot loader to load the image without having to understand numerous a.out variants or other executable formats. This magic header does not need to be at the very beginning of the executable file, so kernel images can still conform to the local a.out format variant in addition to being MultiBoot compliant. Boot modules ~~~~~~~~~~~~ Many modern operating system kernels, such as those of VSTa and Mach, do not by themselves contain enough mechanism to get the system fully operational: they require the presence of additional software modules at boot time in order to access devices, mount file systems, etc. While these additional modules could be embedded in the main OS image along with the kernel itself, and the resulting image be split apart manually by the OS when it receives control, it is often more flexible, more space-efficient, and more convenient to the OS and user if the boot loader can load these additional modules independently in the first place. Thus, this standard should provide a standard method for a boot loader to indicate to the OS what auxiliary boot modules were loaded, and where they can be found. Boot loaders don't have to support multiple boot modules, but they are strongly encouraged to, because some OS's will be unable to boot without them. DETAILS ~~~~~~~ There are three main aspects of the boot-loader/OS image interface this standard must specify: * The format of the OS image as seen by the boot loader. * The state of the machine when the boot loader starts the OS. * The format of the information passed by the boot loader to the OS. OS Image Format ~~~~~~~~~~~~~~~ An OS image is generally just an ordinary 32-bit executable file in the standard format for that particular OS, except that it may be linked at a non-default load address to avoid loading on top of the PC's I/O region or other reserved areas, and of course it can't use shared libraries or other fancy features. Initially, only images in a.out format are supported; ELF support will probably later be specified in the standard. Unfortunately, the exact meaning of the text, data, bss, and entry fields of a.out headers tends to vary widely between different executable flavors, and it is sometimes very difficult to distinguish one flavor from another (e.g. Linux ZMAGIC executables and Mach ZMAGIC executables). Furthermore, there is no simple, reliable way of determining at what address in memory the text segment is supposed to start. Therefore, this standard requires that an additional header, known as a 'multiboot_header', appear somewhere near the beginning of the executable file. In general it should come "as early as possible", and is typically embedded in the beginning of the text segment after the "real" executable header. It _must_ be contained completely within the first 8192 bytes of the executable file, and must be longword (32-bit) aligned. These rules allow the boot loader to find and synchronize with the text segment in the a.out file without knowing beforehand the details of the a.out variant. The layout of the header is as follows: +-------------------+ 0 | magic: 0x1BADB002 | (required) 4 | flags | (required) +-------------------+ 8 | header_addr | (present if flags[16] is set) 12 | load_addr | (present if flags[16] is set) 16 | load_end_addr | (present if flags[16] is set) 20 | bss_end_addr | (present if flags[16] is set) 24 | entry_addr | (present if flags[16] is set) +-------------------+ All fields are in little-endian byte order, of course. The first field is the magic number identifying the header, which must be the hex value 0x1BADB002. The flags field specifies features that the OS image requests or requires of the boot loader. Bits 0-15 indicate requirements; if the boot loader sees any of these bits set but doesn't understand the flag or can't fulfill the requirements it indicates for some reason, it must notify the user and fail to load the OS image. Bits 16-31 indicate optional features; if any bits in this range are set but the boot loader doesn't understand them, it can simply ignore them and proceed as usual. Naturally, all as-yet-undefined bits in the flags word must be set to zero in OS images. This way, the flags fields serves for version control as well as simple feature selection. If bit 0 in the flags word is set, then all boot modules loaded along with the OS must be aligned on page (4KB) boundaries. Some OS's expect to be able to map the pages containing boot modules directly into a paged address space during startup, and thus need the boot modules to be page-aligned. If bit 1 in the flags word is set, then information on available memory via at least the method signified by bit 0 in the multiboot_info.flags word must be provided. Passing information via the memory map as well is not strictly required, but strongly recommended. If bit 16 in the flags word is set, then the fields at offsets 8-24 in the multiboot_header are valid, and the boot loader should use them instead of the fields in the actual executable header to calculate where to load the OS image. This information does not need to be provided if the kernel image is in ELF format, but it should be provided if the images is in a.out format or in some other format. Compliant boot loaders must be able to load images that either are in ELF format or contain the load address information embedded in the multiboot_header; they may also directly support other executable formats, such as particular a.out variants, but are not required to. All of the address fields enabled by flag bit 16 are physical addresses. The meaning of each is as follows: header_addr Contains the address corresponding to the beginning of the multiboot_header - the physical memory location at which the magic value is supposed to be loaded. This field serves to "synchronize" the mapping between OS image offsets and physical memory addresses. load_addr Contains the physical address of the beginning of the text segment. The offset in the OS image file at which to start loading is defined by the offset at which the header was found, minus (header_addr - load_addr). load_addr must be less than or equal to header_addr. load_end_addr Contains the physical address of the end of the data segment. (load_end_addr - load_addr) specifies how much data to load. This implies that the text and data segments must be consecutive in the OS image; this is true for existing a.out executable formats. bss_end_addr Contains the physical address of the end of the bss segment. The boot loader initializes this area to zero, and reserves the memory it occupies to avoid placing boot modules and other data relevant to the OS in that area. entry The physical address to which the boot loader should jump in order to start running the OS. Machine State ~~~~~~~~~~~~~ When the boot loader invokes the 32-bit operating system, the machine must have the following state: * CS must be a 32-bit read/execute code segment with an offset of 0 and a limit of 0xffffffff. * DS, ES, FS, GS, and SS must be a 32-bit read/write data segment with an offset of 0 and a limit of 0xffffffff. * Paging must be turned off. * The processor interrupt flag must be turned off. * EAX must contain the magic value 0x2BADB002; the presence of this value indicates to the OS that it was loaded by a MultiBoot-compliant boot loader (e.g. as opposed to another type of boot loader that the OS can also be loaded from). * EBX must contain the 32-bit physical address of the multiboot_info structure provided by the boot loader (see below). All other processor registers and flag bits are undefined. This includes, in particular: * ESP: the 32-bit OS must create its own stack as soon as it needs one. * GDTR: Even though the segment registers are set up as described above, the GDTR may be invalid, so the OS must not load any segment registers (even just reloading the same values!) until it sets up its own GDT. * IDTR: The OS must leave interrupts disabled until it sets up its own IDT. However, other machine state should be left by the boot loader in "normal working order", i.e. as initialized by the BIOS (or DOS, that that's what the boot loader runs from). In other words, the OS should be able to make BIOS calls and such after being loaded, as long as it does not overwrite the BIOS data structures before doing so. Also, the boot loader must leave the PIC programmed with the normal BIOS/DOS values, even if it changed them during the switch to 32-bit mode. Boot Information Format ~~~~~~~~~~~~~~~~~~~~~~~ Upon entry to the OS, the EBX register contains the physical address of a 'multiboot_info' data structure, through which the boot loader communicates vital information to the OS. The OS can use or ignore any parts of the structure as it chooses; all information passed by the boot loader is advisory only. The multiboot_info structure and its related substructures may be placed anywhere in memory by the boot loader (with the exception of the memory reserved for the kernel and boot modules, of course). It is the OS's responsibility to avoid overwriting this memory until it is done using it. The format of the multiboot_info structure (as defined so far) follows: +-------------------+ 0 | flags | (required) +-------------------+ 4 | mem_low | (present if flags[0] is set) 8 | mem_high | (present if flags[0] is set) +-------------------+ 12 | boot_device | (present if flags[1] is set) +-------------------+ 16 | cmdline | (present if flags[2] is set) +-------------------+ 20 | mods_count | (present if flags[3] is set) 24 | mods_addr | (present if flags[3] is set) +-------------------+ 28 - 40 | syms | (present if flags[4] or flags[5] is set) +-------------------+ 44 | mmap_length | (present if flags[6] is set) 48 | mmap_addr | (present if flags[6] is set) +-------------------+ The first longword indicates the presence and validity of other fields in the multiboot_info structure. All as-yet-undefined bits must be set to zero by the boot loader. Any set bits that the OS does not understand should be ignored. Thus, the flags field also functions as a version indicator, allowing the multiboot_info structure to be expanded in the future without breaking anything. If bit 0 in the multiboot_info.flags word is set, then the 'mem_*' fields are valid. 'mem_low' and 'mem_high' indicate the amount of lower and upper memory, respectively, in kilobytes. Lower memory starts at address 0, and upper memory starts at address 0x100000 (1 Megabyte). The maximum possible value for lower memory is 640 kilobytes. The values returned here are not guaranteed to represent all of the available memory, though generally will be correct up to 64MB. If bit 1 in the multiboot_info.flags word is set, then the 'boot_device' field is valid, and indicates which BIOS disk device the boot loader loaded the OS from. If the OS was not loaded from a BIOS disk, then this field must not be present (bit 3 must be clear). The OS may use this field as a hint for determining its own "root" device, but is not required to. The boot_device field is layed out in four one-byte subfields as follows: +-------+-------+-------+-------+ | drive | part1 | part2 | part3 | +-------+-------+-------+-------+ The first byte contains the BIOS drive number as understood by the BIOS INT 0x13 low-level disk interface: e.g. 0x00 for the first floppy disk or 0x80 for the first hard disk. The three remaining bytes specify the boot partition. 'part1' specifies the "top-level" partition number, 'part2' specifies a "sub-partition" in the top-level partition, etc. Partition numbers always start from zero. Unused partition bytes must be set to 0xFF. For example, if the disk is partitioned using a simple one-level DOS partitioning scheme, then 'part1' contains the DOS partition number, and 'part2' and 'part3' are both zero. As another example, if a disk is partitioned first into DOS partitions, and then one of those DOS partitions is subdivided into several BSD partitions using BSD's "disklabel" strategy, then 'part1' contains the DOS partition number, 'part2' contains the BSD sub-partition within that DOS partition, and 'part3' is 0xFF. DOS extended partitions are indicated as partition numbers starting from 4 and increasing, rather than as nested sub-partitions, even though the underlying disk layout of extended partitions is hierarchical in nature. For example, if the boot loader boots from the second extended partition on a disk partitioned in conventional DOS style, then 'part1' will be 5, and 'part2' and 'part3' will both be 0xFF. If bit 2 of the flags longword is set, the 'cmdline' field is valid, and contains the physical address of the the command line to be passed to the kernel. The command line is a normal C-style null-terminated string. If bit 3 of the flags is set, then the 'mods' fields indicate to the kernel what boot modules were loaded along with the kernel image, and where they can be found. 'mods_count' contains the number of modules loaded; 'mods_addr' contains the physical address of the first module structure. 'mods_count' may be zero, indicating no boot modules were loaded, even if bit 1 of 'flags' is set. Each module structure is formatted as follows: +-------------------+ 0 | mod_start | 4 | mod_end | +-------------------+ 8 | string | +-------------------+ 12 | reserved (0) | +-------------------+ The first two fields contain the start and end addresses of the boot module itself. The 'string' field provides an arbitrary string to be associated with that particular boot module; it is a null-terminated ASCII string, just like the kernel command line. The 'string' field may be 0 if there is no string associated with the module. Typically the string might be a command line (e.g. if the OS treats boot modules as executable programs), or a pathname (e.g. if the OS treats boot modules as files in a file system), but its exact use is specific to the OS. The 'reserved' field must be set to 0 by the boot loader and ignored by the OS. NOTE: Bits 4 & 5 are mutually exclusive. If bit 4 in the multiboot_info.flags word is set, then the following fields in the multiboot_info structure starting at byte 28 are valid: +-------------------+ 28 | tabsize | 32 | strsize | 36 | addr | 40 | reserved (0) | +-------------------+ These indicate where the symbol table from an a.out kernel image can be found. 'addr' is the physical address of the size (4-byte unsigned long) of an array of a.out-format 'nlist' structures, followed immediately by the array itself, then the size (4-byte unsigned long) of a set of null-terminated ASCII strings (plus sizeof(unsigned long) in this case), and finally the set of strings itself. 'tabsize' is equal to it's size parameter (found at the beginning of the symbol section), and 'strsize' is equal to it's size parameter (found at the beginning of the string section) of the following string table to which the symbol table refers. Note that 'tabsize' may be 0, indicating no symbols, even if bit 4 in the flags word is set. If bit 5 in the multiboot_info.flags word is set, then the following fields in the multiboot_info structure starting at byte 28 are valid: +-------------------+ 28 | num | 32 | size | 36 | addr | 40 | shndx | +-------------------+ These indicate where the section header table from an ELF kernel is, the size of each entry, number of entries, and the string table used as the index of names. They correspond to the 'shdr_*' entries ('shdr_num', etc.) in the Executable and Linkable Format (ELF) specification in the program header. All sections are loaded, and the physical address fields of the elf section header then refer to where the sections are in memory (refer to the i386 ELF documentation for details as to how to read the section header(s)). Note that 'shdr_num' may be 0, indicating no symbols, even if bit 5 in the flags word is set. If bit 6 in the multiboot_info.flags word is set, then the 'mmap_*' fields are valid, and indicate the address and length of a buffer containing a memory map of the machine provided by the BIOS. 'mmap_addr' is the address, and 'mmap_length' is the total size of the buffer. The buffer consists of one or more of the following size/structure pairs ('size' is really used for skipping to the next pair): +-------------------+ -4 | size | +-------------------+ 0 | BaseAddrLow | 4 | BaseAddrHigh | 8 | LengthLow | 12 | LengthHigh | 16 | Type | +-------------------+ where 'size' is the size of the associated structure in bytes, which can be greater than the minimum of 20 bytes. 'BaseAddrLow' is the lower 32 bits of the starting address, and 'BaseAddrHigh' is the upper 32 bits, for a total of a 64-bit starting address. 'LengthLow' is the lower 32 bits of the size of the memory region in bytes, and 'LengthHigh' is the upper 32 bits, for a total of a 64-bit length. 'Type' is the variety of address range represented, where a value of 1 indicates available RAM, and a value of 2 indicates a reserved memory region. Any other value for 'Type' should be interpreted as reserved. The map provided is guaranteed to list all standard RAM that should be available for normal use, and to list all reserved regions that are not part of the original PC/AT memory map (it will not list the VGA area as reserved, nor memory mapped add-in ISA or PCI devices, for example). NOTE: This data structure (starting with 'BaseAddrLow') is the same data returned by the BIOS INT 15h, AX=E820h call, and extra data past the first 20 bytes and/or new 'Type' values may be interpreted via newer definitions of that standard, in the section describing the 'Address Range Descriptor Structure'. EXAMPLE CODE ~~~~~~~~~~~~ The Mach 4 distribution, available by anonymous FTP from jaguar.cs.utah.edu:/flexmach, contains a C header file that defines the MultiBoot data structures described above; anyone is welcome to rip it out and use it for other boot loaders and OS's: mach4-i386/include/mach/machine/multiboot.h This distribution also contains code implementing a "Linux boot adaptor", which collects a MultiBoot-compliant OS image and an optional set of boot modules, compresses them, and packages them into a single traditional Linux boot image that can be loaded from LILO or other Linux boot loaders. This code can be used as-is or as a basis for other boot loaders. These are the directories of primary relevance: mach4-i386/boot mach4-i386/boot/bsd mach4-i386/boot/linux The Mach kernel itself in this distribution contains code that demonstrates how to create a compliant OS. The following files are of primary relevance: mach4-i386/kernel/i386at/boothdr.S (MultiBoot header and initialization code) Finally, I have created patches against the Linux 1.2.2 and FreeBSD 2.0 kernels, in order to make them compliant with this proposed standard. These patches are available in kahlua.cs.utah.edu:/private/boot. ISSUES ~~~~~~ Should command lines be passed as a list of null terminated strings, more like Unix command lines, instead of just one string? Feedback on these questions, and any other issues, are most welcome! AUTHOR ~~~~~~ Bryan Ford Computer Systems Laboratory University of Utah Salt Lake City, UT 84112 (801) 581-4280 baford@cs.utah.edu --------------------(end "boot-proposal-0.4" here)---------------- -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Feb 2 14:12:27 1996 id OAA05261; Fri, 2 Feb 1996 14:12:23 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: shap@aurora.cis.upenn.edu Subject: Web pages Date: Fri, 02 Feb 1996 13:12:11 -0800 Sender: erich@uruk.org I'm starting to get enough questions about the bootloader I'm working on that I'm making some web pages for it and relevant information like the multiboot proposal. The multiboot proposal page (and at least one link) are already there enough to read. I'm going to be revising and adding to the "other information" section extensively. It can be found by going to the root page of my web server (which I am finally in the process of updating), or go directly to: http://www.uruk.org/boot-proj/boot-proposal-0.4.html -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Wed Feb 7 01:13:15 1996 id BAA13909; Wed, 7 Feb 1996 01:13:13 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Bryan Ford cc: multiboot@schirf.cs.utah.edu Subject: Re: How goes life? In-reply-to: Your message of "Tue, 06 Feb 1996 14:38:33 MST." <199602062138.OAA08066@snake.cs.utah.edu> Date: Wed, 07 Feb 1996 00:12:29 -0800 Sender: erich@uruk.org Bryan Ford writes: > Actually, I was going to post a message about your latest multiboot stuff, > but didn't get around to it because it would only have been a > "looks good to me" message and didn't seem extremely high priority... :-) > Anyway, all the changes you've made recently look great, and I'll > integrate your changes to multiboot and mach4 into the source treese here > as soon as you're ready. I just got the code done and working (as far as I can tell at the moment) for the new multiboot interface. I have a set of newer files (with the memory stuff working, but not using the memory mapping BIOS interface on either side yet... you have to start somewhere :-) for mach4 plus a patched version of my bootloader to support the newer interface (the other stuff I've been working on isn't there yet, but will be soon). It's all in the usual place, "ftp://uruk.org/private/grub/". The names are pretty obvious. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Wed Feb 7 01:14:04 1996 id BAA13913; Wed, 7 Feb 1996 01:14:02 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback didn't use HELO protocol To: Bryan Ford cc: multiboot@schirf.cs.utah.edu Subject: Re: How goes life? In-reply-to: Your message of "Tue, 06 Feb 1996 14:38:33 MST." <199602062138.OAA08066@snake.cs.utah.edu> Date: Wed, 07 Feb 1996 00:13:30 -0800 Sender: erich@uruk.org Bryan Ford writes: > Actually, I was going to post a message about your latest multiboot stuff, > but didn't get around to it because it would only have been a > "looks good to me" message and didn't seem extremely high priority... :-) > Anyway, all the changes you've made recently look great, and I'll > integrate your changes to multiboot and mach4 into the source treese here > as soon as you're ready. I just got the code done and working (as far as I can tell at the moment) for the new multiboot interface. I have a set of newer files (with the memory stuff working, but not using the memory mapping BIOS interface on either side yet... you have to start somewhere :-) for mach4 plus a patched version of my bootloader to support the newer interface (the other stuff I've been working on isn't there yet, but will be soon). It's all in the usual place, "ftp://uruk.org/private/grub/". The names are pretty obvious. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ home #: +1 (503) 226-0741 WWW Site \_ USnail: 924 S.W. 16th Ave, #202 Motto: "I'll live forever or die trying" \ Portland, OR, USA 97205 From multiboot-request Fri Feb 23 20:39:53 1996 id UAA03432; Fri, 23 Feb 1996 20:39:53 -0700 id UAA27210; Fri, 23 Feb 1996 20:39:09 -0700 id UAA24093; Fri, 23 Feb 1996 20:37:54 -0700 To: erich@uruk.org cc: multiboot@schirf Subject: Latest MultiBoot proposal and Mach changes In-reply-to: Your message of "Wed, 07 Feb 96 00:12:29 PST." <199602070812.AAA00514@uruk.org> Date: Fri, 23 Feb 96 20:37:54 MST From: Bryan Ford OK, Erich, I finally got a chance to grab your latest MultiBoot-related Mach changes and integrate it into the main Mach 4 distribution. I had to fix a couple minor bugs (the biggest being that the Linux boot adaptor needed to be updated to supply the BIOS memory info), but other than that it worked fine. It should all show up in our next Mach release, which was supposed to happen this week, but should _definitely_ happen this next week at the very very latest (:-))... To everyone: I also updated all (well, most of) the files in the MultiBoot FTP directory (ftp://kahlua.cs.utah.edu/private/boot), and put a copy of Erich's latest version of the spec there, as version 0.5. With the latest changes I think the spec addresses all the significant problems people have brought up, so I suggest we call this version "stable" and try very hard to limit any further changes to clarifications and backwards-compatible changes. From what I'm told, several people are starting to work on adding MultiBoot support to different OS's, and for that to happen they need a stable specification to work from. Bryan From multiboot-request Fri Mar 22 20:17:46 1996 id UAA06866; Fri, 22 Mar 1996 20:17:43 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: GRUB public Beta release soon Date: Fri, 22 Mar 1996 19:18:54 -0800 Sender: erich@uruk.org Hi all. I've been working on GRUB again, and there are a lot of additions since my previous snapshots. I'm going to be making a public Beta release, with announcements in all the appropriate places, soon. The current speculative date is near the end of April (most of what will cause this to delay will be install issues... and if that looks like it will drag, I will make it a public alpha release... I've committed to getting it out feature-complete). For those interested, I put together a (still early) web-page describing the features that will be in the first release, which has links to the multiboot spec and some information I have gathered on PC partitioning and BIOS memory probing. The page is at: http://www.uruk.org/boot-proj/grub.html The features currently incomplete (but planned to be finished for the beta release) are: -- ELF symbol table loading -- menu interface -- FAT & ext2fs filesystems -- decompression support Everything else is moderately tested and working, including the finished memory probe (I took it to a test machine in our back lab at work, and it reported 2 Gigabytes of RAM correctly). No more having to hard-code the memory size for machines with greater than 64MB of RAM. When the above features are complete (maybe minus the ELF symbol table stuff), I'll make another snapshot that people can use. This will probably happen in the next week or two, and I'll send e-mail to the list about it. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Fri Mar 29 09:37:03 1996 id JAA09271; Fri, 29 Mar 1996 09:36:55 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Minor addition to Multiboot Proposal Date: Fri, 29 Mar 1996 08:38:58 -0800 Sender: erich@uruk.org A friend looked over the Multiboot proposal and noted that although using a magic number was a pretty good idea for finding the structure, we should really use a checksum to guarantee it's correct. It turns out that I've had a few problems (when trying to boot the Utah Mach4 monolithic image, it found the multiboot magic number that Bryan had in his boot-adapter instead of performing a BSD-ish boot, and died), but in my race to fix the technical issues like probing for arbitrary amounts of RAM, this completely slipped my mind. Anyway, I think it is fairly important, and is a relatively trivial change that is made to the boot "header" in the OS kernel image. This is the last change I think we should make before calling it version 1.0 of the standard. I'll update the copy on my web-site (plus some other minor wording changes in the memory information stuff) ASAP... but I'll leave the version number at 0.6 until no other serious objections to this being the first major version are made. BTW, I have a warning about the memory information (this is the gist of the "wording change"): the value for "upper memory" may not be guaranteed to represent all of available RAM, but on newer machines, it is highly recommended that this value include all RAM up to the first memory hole in upper memory. This is important to note, as several OS's consider it an error to report more than 64 MB of upper memory! This was really a bug in the bootloaders they were using, and they patched the OS to fix it (argh!). GRUB correctly reports as much as possible (which is how I found out about this problem :-( -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Sun Mar 31 19:46:11 1996 id TAA02523; Sun, 31 Mar 1996 19:46:07 -0700 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: New GRUB snapshot Date: Sun, 31 Mar 1996 18:48:00 -0800 Sender: erich@uruk.org I just put a snapshot of GRUB into my FTP area at: ftp://ftp.uruk.org/private/grub/ called "grub-960331.tar.gz". It doesn't contain any majorly new features (I've got more code, but I wanted to get out a last "stable" version with multiboot-0.6 support before possibly messing it up), but some things are new, plus several broken things are fixed. My new memory probe is in this code. I've tested in on a bunch of machines, but it'd be REALLY nice to know how it works on the machines people on this list have access to, both with less than or greater than 64 MB. Version 0.1 dated 3/31/96 (lots of changes): - Newer version of Multiboot Standard (version 0.6) supported. - Autodetects kernel types. Supports Multiboot, FreeBSD, NetBSD (Linux isn't finished). - Stage 1.5 works now. Default setup is now for working with a BSD FFS floppy loading "/grub/stage2" as the main bootloader. - Filesystem support improved. It didn't work on many floppies before (problem with the partition-detection code). - Memory probe now supports arbitrary amounts of RAM (some technical limitations exist, see Multiboot standard version 0.6 for details). - A mini-debugger is included by default, activated by hitting '~' on the command-line (it might interfere with things, but it seems OK for my alpha-testing). The commands are in the function "enter_sysdebug" defined in "common.c". If you have an Intel MPS- compatible machine, there are extra commands enabled for SMP cpu testing. 'q' exits and goes back to what you were doing before. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Fri Apr 12 11:44:24 1996 id LAA25835; Fri, 12 Apr 1996 11:44:16 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: miles@gnu.ai.mit.edu, roland@gnu.ai.mit.edu Subject: New GRUB snapshot... Date: Fri, 12 Apr 1996 10:45:13 -0700 Sender: erich@uruk.org I have the main UI mostly done now. I'm sort of in a hurry to get it out so I can get back to other things, but I feel confident enough to snapshot it. Config files with multiple entries, programmable timeout, etc. all appears to work. I want to add more commands, but they're pretty easy to add, now. It's all in the same place, at: ftp://ftp.uruk.org/private/grub/ and the new snapshot is "grub-960411.tar.gz". Let me know about problems, suggestions, etc. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Fri Apr 12 15:59:29 1996 id PAA27450; Fri, 12 Apr 1996 15:59:22 -0600 Date: Fri, 12 Apr 1996 17:57:51 -0400 From: Miles Bader To: erich@uruk.org Cc: multiboot@schirf.cs.utah.edu, roland@gnu.ai.mit.edu Subject: Re: New GRUB snapshot... In-Reply-To: <199604121745.KAA14402@uruk.org> References: <199604121745.KAA14402@uruk.org> Blat: Foop I've installed this as the default boot-loader on our hurd system, and everything works like a charm (including the menu interface). It's absurdly featureful... Thanks! -Miles -- Miles Bader / miles@gnu.ai.mit.edu / (617) 253-8568 Amadera e ike! From multiboot-request Fri Apr 12 18:36:17 1996 id SAA27735; Fri, 12 Apr 1996 18:36:15 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: multiboot@schirf.cs.utah.edu Subject: Re: New GRUB snapshot... Date: Fri, 12 Apr 1996 17:37:15 -0700 Sender: erich@uruk.org Miles Bader writes: > I've installed this as the default boot-loader on our hurd system, and > everything works like a charm (including the menu interface). Good. > It's absurdly featureful... Just following the tradition of GNU software... :-) Seriously, the primary purpose of the fancy UI and other features is a combination of usability and acessibility to both experts and naive users. I'm willing to take a small hit in expert convenience towards this end, but if it's much of a problem, I'd like to know. P.S.: My Internet link was on the fritz for the last week or two (and was abysmally slow), but appears to be running well again. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Tue May 21 00:29:10 1996 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: mach4-users@cs.utah.edu, erich@uruk.org Subject: New GRUB snapshot... Date: Mon, 20 May 1996 23:27:06 -0700 Sender: erich@uruk.org Hi all. Sorry that I've not put out another snapshot for a while. This is really an interim snapshot. Many more things work, but some of the new features are somewhat broken. Part of the delay was because I fundamentally re-wrote a few parts of the assembly code, in particular the "stage1" part. It should now be capable of autoprobing a floppy in the drive. (it's slowed down a bit, but optimizations will come after I'm sure it works right). Anyway, this version adds tons of features: ext2fs (thanks to who donated the basic implementation), some blocklist functionality, chain-loading, linux loading, the stage1 re-write, some preparations for LBA support for larger than 8 GB disks in some BIOSes, etc. etc. It's at: ftp://ftp.uruk.org/private/grub/grub-960520.tar.gz Here's the top of the news file: Version 0.3 dated 5/20/96: - Updated instructions (though still very sparse). - New floppy probe (works much like the Linux floppy boot probe) attempts to find the size of a floppy in a drive. Might still need work! Please try on various floppy drives with various media! - New floppy handler will claim a non-existent drive if the floppy disk isn't present in the drive. (for example, it won't be on the list of installed drives unless a floppy is present) - Stage1 now compatible with both a hard disk MBR and the DOS BIOS parameter block (see "install/README" for more details on how this can be used). - Block-list filesystem partially works, as described in the file "NOTES". Loading an a.out or elf kernel won't work with it, but all other filetypes pretty much should. (certainly chain-loading works OK) NOTE: you must use the full format "0+1,512" for just he first block... no parameters can be implicit in this version.. THis is being fixed too. - Linux ext2 filesystem works. (it's very slow for big files, but this is being fixed) - Linux boot type now supported. Use a standard piggybacked image as with LILO. Put in hack to support >64MB via GRUB placing the RAM size as the first item on the command-line automatically. Must pass root partition on command-line using normal Linux syntax... if not, it uses it's builtin root partition. - Supports chain-loading. For details, see "COMMANDS" and the examples directory. (was able to boot DOS and Windows NT on my test box). NOTE that the "root partition" must be set to work right. "makeactive" is currently a no-op. - Several weird bugs fixed. One important note: If you recompile, it will warn about a clash with builtin "strcmp". This is normal... do NOT remove the strcmp definition, as then GCC will possibly put inline code from it's own builtin function in some places. (my strcmp has slightly different functionality, hence the problem) - Mini-debugger is currently broken. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Sun Jun 2 14:27:11 1996 id OAA02370; Sun, 2 Jun 1996 14:27:06 -0600 From: erich@uruk.org X-Authentication-Warning: uruk.org: Host loopback [127.0.0.1] didn't use HELO protocol To: multiboot@schirf.cs.utah.edu cc: miles@gnu.ai.mit.edu Subject: GRUB public Alpha release Date: Sun, 02 Jun 1996 13:24:56 -0700 Sender: erich@uruk.org Hi all. Since I've been dithering around trying to cover all of the bases, the beta-release I've planned for GRUB has stretched out quite a bit. I DID manage to finish nearly all of the extra gunk I got involved in (a big part was re-writing the stage1 to be compatible with many more types of systems, including the FAT BIOS Parameter Block and floppy drive probes), but the decompression code and automated installation stuff isn't there. Anyway, that's pretty much all that's missing, I decided a public alpha- release was in order. Things such as installing GRUB on a DOS boot floppy, ext2fs partitions (both using blocklists for the stage2), FFS partitions and floppies, chainloading various other OS'es, FAT, ext2fs, and FFS filesystems etc. all just seem to work at this point. The installation stuff is still pretty manual, but I included an INSTALL file with much more options listed and in more detail than before. Other docs have been updated as well. It can be retrieved from my ftp site at: ftp://ftp.uruk.org/public/grub/ as the file: grub-960602.tar.gz Here's the top of the NEWS file for this release: Version 0.3 dated 6/2/96 -- first public "alpha" release. - Completed initial licenses. - Initial filesystem documentation written. - Block-list and FAT filesystems now work as documented (in particular, for the blocklist filesystem, shortcuts like "+1" for "0+1,512" now work correctly). - Fixed several problems (old and new) in the various filesystems (for example, the ext2fs filesystem code is now much faster, as it caches some mapping blocks where it didn't at all before). Filesystem semantics are much more uniform as well (symbolic links and reading a directory as a file now return errors where it would silently fail before). - "makeactive" now works for standard PC partitions on hard disks (not extended partitions... so any PC partition number above 3 will give a "no such partition" error). If a BSD sub-partition is is used, it will ignore it, and only use the primary PC partition number. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Fri Jun 7 13:29:39 1996 From: erich@uruk.org To: multiboot@schirf.cs.utah.edu, mach4-users@cs.utah.edu cc: miles@gnu.ai.mit.edu, roland@gnu.ai.mit.edu, gord@enci.ucalgary.ca, slx82@sys3.cs.usu.edu, Michael.Salay@ae.ge.com, patv@unx.dec.com, shap@aurora.cis.upenn.edu, rcollins@datastar.net, hohmuth@inf.tu-dresden.de, vax@linkdead.paranoia.com, egh@raisin.bagel.org, marcus@sysc.pdx.edu, rraffer1@osf1.gmu.edu, stephan@relay.xlink.net, jargon@ca2.so-net.or.jp, dfuqua@mercury.interpath.com, aknight@knight-apps.co.uk, mccallum@cs.rochester.edu, erich@microsoft.com Subject: GRUB public alpha-release Date: Fri, 07 Jun 1996 12:26:09 -0700 Sender: erich@uruk.org [Some of you may have gotten this announcement before; please bear with me] [I'll be putting together some kind of e-mail list for GRUB in the near future. If you want to be on it, send me a message] Since I've been dithering around trying to cover all of the bases, the beta-release I've planned for GRUB has stretched out quite a bit. I DID manage to finish nearly all of the extra gunk I got involved in (a big part was re-writing the stage1 to be compatible with many more types of systems, including the FAT BIOS Parameter Block and floppy drive probes), but the decompression code and automated installation stuff isn't there. Anyway, that's pretty much all that's missing, I decided a public alpha- release was in order. Things such as installing GRUB on a DOS boot floppy, ext2fs partitions (both using blocklists for the stage2), FFS partitions and floppies, chainloading various other OS'es, mostly just seems to work at this point. The two bugs I've discovered since the alpha-release (which are already fixed in my development tree) are: -- The 12-bit FAT filesystem code for reading floppies is broken such that any file over one block in size will return garbage data. Hard disks work OK. -- Changing floppies without performing some action to remount them (like accessing another device) will give garbage results. The installation stuff is still pretty manual, but I included an INSTALL file with much more options listed and in more detail than before. Other docs have been updated as well. Some of the docs are online (and the rest will be as of this weekend) at: http://www.uruk.org/grub/ GRUB can be retrieved from my ftp site at: ftp://ftp.uruk.org/public/grub/ as the file: grub-960602.tar.gz Here's the top of the NEWS file for this release: Version 0.3 dated 6/2/96 -- first public "alpha" release. - Completed initial licenses. - Initial filesystem documentation written. - Block-list and FAT filesystems now work as documented (in particular, for the blocklist filesystem, shortcuts like "+1" for "0+1,512" now work correctly). - Fixed several problems (old and new) in the various filesystems (for example, the ext2fs filesystem code is now much faster, as it caches some mapping blocks where it didn't at all before). Filesystem semantics are much more uniform as well (symbolic links and reading a directory as a file now return errors where it would silently fail before). - "makeactive" now works for standard PC partitions on hard disks (not extended partitions... so any PC partition number above 3 will give a "no such partition" error). If a BSD sub-partition is is used, it will ignore it, and only use the primary PC partition number. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel. From multiboot-request Tue Jun 18 05:36:15 1996 From: erich@uruk.org To: multiboot@schirf.cs.utah.edu, mach4-users@cs.utah.edu cc: miles@gnu.ai.mit.edu, roland@gnu.ai.mit.edu, gord@enci.ucalgary.ca, slx82@sys3.cs.usu.edu, Michael.Salay@ae.ge.com, patv@unx.dec.com, shap@aurora.cis.upenn.edu, rcollins@datastar.net, hohmuth@inf.tu-dresden.de, vax@linkdead.paranoia.com, egh@raisin.bagel.org, marcus@sysc.pdx.edu, rraffer1@osf1.gmu.edu, stephan@relay.xlink.net, jargon@ca2.so-net.or.jp, dfuqua@mercury.interpath.com, aknight@knight-apps.co.uk, mccallum@cs.rochester.edu, erich@microsoft.com, phk@freebsd.org Subject: GRUB Beta release (0.3 dated 960617) Date: Tue, 18 Jun 1996 04:36:52 -0700 Sender: erich@uruk.org This message is to announce the Beta release of GRUB (0.3 dated 960617). Please feel free to redistribute the announcement as desired. It is basically feature-complete with all the original goals except for ELF symbol tables and automatic decompression code. I still intend to do this in a later release. More importantly, it appears that the source of pretty much all of the showstopping bugs has been found. I changed enough other things for this release that a few more may appear in the shakeout, but it seems pretty solid. This version solves both the observed functionality bugs and adds some very important features (DEBUG code can do wonders with a generic framework!!). GRUB can be retrieved from: ftp://ftp.uruk.org/public/grub/ Please use the 960617 release! Earlier ones have serious bugs and lack the better usability features. >From the NEWS file: Version 0.3 dated 6/17/96 -- public "beta" release: - Yet more documentation improvements. - Known bugs in floppy operation fixed (12-bit FAT didn't work for most cases, and inserting other floppies didn't flush the filesystem cache). - NASTY uninitialized pointer bug causing "raw" floppy operation to crash on several PCs is now fixed. This seems to have been the root cause of all of the compatibility problems that have currently been observed. - debug-mode command added to automate most difficult step of installation for common cases (new install method #4). - Testing "mini-debugger" now merged with command-line when "DEBUG" defined in compile (no SYSDEBUG option anymore). See description of commands in the command-line for details. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" This is my home system, so I'm speaking only for myself, not for Intel.