Article: 8022 of comp.arch From: mash@mash.engr.sgi.com (John R. Mashey) Newsgroups: comp.arch Subject: Re: why is ia64 turning out so slow? (was: Re: 64bit Willy) Date: 31 Jul 2000 05:38:42 GMT Organization: Silicon Graphics, Inc. Lines: 73 Message-ID: <8m33d2$dgb$1@murrow.corp.sgi.com> References: <39819DB9.1B7C0D97@cisco.com> <8lschp07ae@news1.newsguy.com> <39833534.B24F7981@gmx.de> We've discussed the virtualization issue before. As is often the case, one needs to remember history: 1) The best success of virtualizable CPUs has been in the CP/67 -> VM/370 line of IBM hypervisors, combined with various evolutionary features in various successive system designs to support this. This started in the late 1960s [the 360/67 was the first one for which this was practical]. 2) Recall that IBM mainframes: (2a) Were expensive, and hence not likely to be something systems programmers could have bunches of sitting around to boot. (2b) Use a channel-oriented I/O architecture, and generally, various sorts of privileged opcodes to do privileged things. [I.e., you build a list of Channel Command Words in memory, then issue a Start I/O pointing at them, and then a channel crunches away CCWs without CPU intervention; fairly complex thing could be done, including the amazing "PCI-fetch" with dynamically-modified chains created as you read in disk records. In general, I/O devices were "arms-length", in some sense. 3) On the other hand, minis (especially DEC), and especially micros: (3a) Were less expensive, and it was much more likely that a systems programmer might be able to have one to play with. Also, since they arose as cost-driven entities, extra features were frowned on. (3b) Most often use memory-mapped I/O operations, and sometimes, priviliged operations done by touching specific memory locations. Many mini/micro systems have had things like a simple ASYNC I/O port for a console, or a frame buffer, and it takes some thought about what's necessary to efficceintly virtualize them. Even worse, there have been many such thigns with seriously timing-dependent interfaces of one sort or another, or real-time features, etc, etc, none of which respond well to hypervisoring. 4) Given (2a), there was strong motivation to use hypervisors by systems programmers [and features strongly desired by systems programmers do get a fighting chance to happen]. Also, the conversational subsystem (CMS) was fanatically preferred by some over TSS & then TSO. Finally, with an expensive resource, running mission-critical software, the evolving abilities of the hardware & software were also attractive to customers. Finally, given (2b), there was a fighting chance that one could efficiently run an OS in user mode, trap privileged operations, without knowing very much about what the OS was doing, and be able to provide a virtual CPU. For example, given a CCW chain, the hypervisor could go through and translate the addresses for I/O, and later, the hardware folks provided help in speeding, or avoiding the translations. 5) However, given (3a), there was far less motivation to add virtualization, given the relative costs. However, given (3b), it would have taken more work, and was much harder to make efficient, given the memory-mapped I/O. First, in micro-based systems, the I/O hardware has often been much less "at-arms-length". Second, the use of memory-mapped I/O is really ugly for virtualization, especially with devices whose states can change not only when you write their memory lcoations, but as sometimes happens, when you read them as well [i.e., input FIFOs or funny status registers.] Hence, the straightforward strategy would be to have to trap on all references to any aprt of the memory space that might be I/O devices. Given the way many drivers are written, this means a trap every few instructions. [Yes, I know, there are tricks, but it's a lot of work, and given that there wasn't a lot of motivation to do it, there wasn't much motivation to beg for the hardware help, and it didn't happen.] 6) Anyway, the bottom line is that it's not an issue of an instruction here or there, it's that the mainframe world has some different design tradeoffs that encourage virtualizaiton, and the mini->micro world has tradeoffs that discourage it. Micro systems are getting much of the same effect (in bigger systems) by providing partitioning support. -- -John Mashey EMAIL: mash@sgi.com DDD: 650-933-3090 FAX: 650-933-2663 USPS: SGI 1600 Amphitheatre Pkwy., ms. 562, Mountain View, CA 94043-1351