Graphic Frustrations

This last month I've run into a number of frustrations in my continuing quest to find, fast, portable, and cheap, ways to do graphics for the PC.

Standards that aren't standard.

At work I got a brand new 486DX2 with a VESA local bus video card based on the S3 graphics accelerator chip. Great machine. Love it. Of course, I immediately checked to see if it had VESA BIOS support. MSD (the diagnostic program that comes with DOS 6.0) reported that there was indeed VBE support present in the system. In fact it reported that the machine had support for VBE 2.0D.

Interesting. I know that VESA is working on a VESA 2.0 version of the VBE standard. When I called and asked for a copy I was told that since it is not yet an official VESA standard only member organizations could see the specification... Heavy sigh. I called the customer support number of the company that supplied the VBE for my new machine and asked them to give me a copy of the VESA 2.0D specification. I left a recorded message, couldn't get a human on the line, explaining my need for the spec. They didn't call back.

Much to my surprise, version 2.0D is different enough from version 1.2 that the code that was in my last column didn't work under 2.0D. Looks like the meaning of the video mode numbers has changed.

Oh well, after another call to VESA I got a couple of other numbers to call that might get me authorized to see the 2.0 draft specification. But, it really doesn't give me a warm fuzzy feeling to have the VBE standard change so much from version 1.2 to version 2.0 that code based on 1.2 won't run on a 2.0 system. Of course, since 2.0 isn't even a standard yet it's kind of scary to have a driver based on it.

The quest for speed and space.

In my last column I used Borland's Turbo C++ 3.0 compiler for all my example code. It's a pretty good compiler. It's very inexpensive. And it seems like everyone I know has a copy of some version of Turbo C++ on a shelf somewhere. So it seemed like a good choice to use for compiling example code that a number of people are going to see.

But, Turbo C++ produces 16 bit real mode code. 16 bit code is "bad" because I have a 32 bit machine. It seems silly that when I want to add two 32 bit numbers together Turbo C++ generates at least 2 instructions on a machine that can add two 32 bit numbers in one instruction. It also seems silly to be fetching and storing 16 bits at a time when I have a 32 bit machine with a 32 bit bus. PCs based on the 386, 486, and Pentium (hate that name) processors have 32 bit registers and can execute 32 bit instructions. All of those except the 386sx have at least a 32 bit bus. I want to use the CPU and bus at full power.

Real mode is "bad" because real mode code is restricted to the 640k memory limits of the original IBM PC. My home machine has 8 Megabytes of RAM. I want to be able to use all of it in the programs I write. And, I don't want to mess around with EMS or XMS paging schemes. I want to be able to say

unsigned char buffer [768][1024];

and have it compile and run. I can't do that in real mode. To do all that I want to do I need a compiler that generates 32 bit protected mode code programs and an operating system that lets me run them. (There is also a 16 bit protected mode, but I'm not really interested in it so when I say "protected mode" I mean 32 bit protected mode.)

At work and at home I use a 32 bit compiler. Watcom C/C++ 9.5a to be precise. Watcom is the third 32 bit DOS compiler I've used. I've also used DJGPP and Zortech C++ version 3.1. The Watcom compiler is by far the best of the 3 from my point of view. It generates excellent code. It comes with a good 32 bit debugger. It comes with a royalty free DOS extender. And, they finally lowered the price out of the stratosphere so that people can actually afford to buy it. It's a good package.

DJGPP is also an excellent compiler, and being freely available, it is the compiler of choice for many hobbyist and professional programmers.

So what does this have to do with graphics? Well, graphics sucks up memory and CPU speed like nothing else I've ever seen. Lets say you have a world with a hundred objects in it. And, say that each object is described using 100 vertices, each vertex is an x, y, z triple, and each x, y, z, is a long int. That comes out to 120,000 bytes. 120,000 bytes isn't a big chunk out of several megabytes, but it is a major part of the 640k of conventional memory that DOS gives you. And face it, 100 objects makes for a pretty trivial world.

Another place that graphics sucks up memory is in buffers. A simple 640 x 480 256 color buffer uses 300k bytes. A standard way to hidden surface and hidden line removal uses what is called a "Z buffer." A Z buffer stores the Z value of each pixel that has been drawn on the screen (or other buffer.) (In a Z buffered system you check the Z of each pixel that you want to draw against the Z the pixel currently has. This way only the pixels that are closest to the viewer are in the finally image.) You need at least 16 bits per Z value per pixel and 24 or 32 bits is better. A 16 bit Z buffer for a 640 by 680 display would use 600k bytes. Mem reports that the largest DOS program I can run is 609k bytes long. I'm not going to do any high resolution Z buffer code in real mode.

My project for this column was to convert my VESA BIOS Extension library (vbe.cpp on the disk) to work with the Rational Systems, Inc. DOS/4GW DOS extender that comes with the Watcom compilers. Using the Watcom compiler and a DOS extender I can use the 32 bit power of my machine and use all the physical memory I've paid for.

A DOS extender is a program that allows 32 bit protected mode programs to run under DOS. Considering that DOS is a 16 bit real mode operating system this is a real neat trick. The DOS extender switches the computer from real mode to protected mode, loads your program, and starts it running.

That's not even half the story of what a DOS extender has to do. You call DOS to do a lot of things for your program. Everything from reading and writing the disk to allocating memory. But, DOS only runs in real mode (and virtual 86 mode, but that's clouding the issue) and your program is running in protected mode. So, the DOS extender intercepts every call you make to any DOS or BIOS service and then decides whether to switch the machine back to real mode and actually call DOS, to just do the work it self, or to do a little of both.

For example, if you are calling DOS to ask for memory the DOS extender will intercept the call and handle it by itself because DOS doesn't know about the megabytes of RAM that you are asking for. If, on the other hand, you are trying to open a file the DOS extender will most likely pass that request through to DOS. If you are reading the disk then the extender is most likely to do the actual I/O using DOS, but since you may have asked to read a few megabytes of data the extender will have to break up the request into many calls to DOS each asking for a chunk small enough for DOS to handle.

I say "most likely" because a DOS extender could do it all by itself and never call DOS at all. But, I don't know of one that does that.

The first problem I had in porting to the extender was figuring out how to call the VESA BIOS Extensions from inside a protected mode program. Watcom C++ provides a pair of functions called int386() and int386x() that are the 386 equivalents of the familiar int86() and int86x(). I really didn't think they would work for calling VBE services but I tried them out anyway. I was right. They didn't work.

the int386() functions work for the normal INT 10h functions I've tried and all the other BIOS functions I've tried, but for the interrupt to work the DOS extender has to know how to handle it and this extender doesn't understand the VBE extensions to INT 10h. Fortunately, this is such a common problem that there is a standard way around it, I just had to find out what it was. This is such a common problem that it was covered in the "Commonly Asked Questions and Answers" manual that comes with Watcom C++. It was even in a section named "How Do I Simulate a Real-Mode Interrupt with DOS/4GW?" Did I mention how happy I am with Watcom's documentation?

There is a specification called the "DOS Protected Mode Interface" or DPMI for short. One of the services provided by DPMI is a way to do any real mode interrupt from inside a protected mode program. Usually DPMI services are provided by a DPMI server such as Windows or 386MAX neither of which I use very often. DOS/4GW provides a subset of the DPMI services, including the one I needed, even if you aren't running a DPMI server.

If you look in vbe.cpp you'll find a function named int386rm(). This function lets you use the DPMI function 0300h to call any real mode interrupt. Once I had written int386rm() I converted the VBE interface routines to use it and most of them worked perfectly on the first try.

There were a couple of problems left. Two of the most important functions in vbe.cpp are vbeGetInfo() and vbeGetModeInfo(). These functions pass pointers to buffers that are filled in by VBE with information about graphics configuration and with the details of a specific video mode.

There are two problems with passing a pointer from a protected mode program to real mode code. First off, real mode pointers are not the same as protected mode pointers. A real mode far pointer is made up of a 16 bit segment and a 16 bit offset. The physical address referred to by a segment/offset pair is given by:

address = (segment << 4) + offset.

Protected mode far pointers are made up of a 16 bit selector and a 32 bit offset. Not only are the offsets different sizes in a real mode pointer and a protected mode pointer but a selector is not a segment. A selector is an index into a table of descriptors that tell you what physical memory is actually referenced by the selector. A protected mode pointer doesn't mean anything in real mode and it can point to memory outside of the 1 meg that is all that real mode programs can see.

This means that you have to allocate memory that the real mode code can see, pass the real mode address of that memory to the real mode code, and then copy that memory up into your protected mode program. I guess you don't really have to copy it. But, if you don't copy it and free the real mode memory you might run out of real mode memory.

That sounds easy... I mentioned above that the DOS extender intercepts memory allocation requests and handles them itself. This means that calling the standard DOS memory allocation function doesn't get you memory that DOS or any BIOS can see. DPMI comes to the rescue again by providing functions to allocate and free real mode memory.

In vbe.cpp the function allocDosMem() and freeDosMem() let you get the buffers that are needed by vbeGetInfo() and vbeGetModeInfo() and DOS/4GW solves problem of seeing low memory from a protected mode program by mapping the low 1 megabyte of physical memory into the low 1 megabyte of your programs address space.

I was pleased with how clean the protected mode interface turned out. Unfortunately the overhead that has been added to the VBE calls has taken a measurable toll on graphics performance. It had very little effect on the pixel demo, but cost almost 20% on the rectangle demo. I'm told there is a way around this problem and I'm checking into it.

The Demo

All the code that is included with this article requires an SVGA display with VBE 1.2 support. Use the UNIVBE TSR even if your system already has VBE support.

demo.cpp -- VBE demo program

demo.exe

doit.bat -- batch file to build the programs

makefile -- Watcom makefile used by doit.bat

ptypes.h -- portable data types definitions

result -- where output from demo get written

vbe.cpp -- complete VBE interface library

vbe.h

vg.cpp -- start of a UNIVBE based graphics library

vg.h

dos4gw.exe -- the DOS extender that you need to run the demo

Before running the demo program you need to unzip univbe42.zip and install univbe.exe. Installation instructions and other interesting documentation can be found in the doc directory that will be created when you unzip UNIVBE.

Run the demo program like this:

demo query

Writes a description of all the VESA video mode support by your system to a file named "result" in the current directory.

demo 332

Draws 256 different colored rectangles on your screen.

demo pixel

Draws random colored pixels on the screen in 640x400 256 color mode. Writes the number of frames/second to the result file.

demo rect

Draws random colored rectangles on the screen in 640x400 256 color mode. Writes the number of frames/second to the result file.

References:

DOS Protected Mode Interface (DPMI) Specification 0.9, July 26, 1990.

"Commonly Asked Question and Answers", WATCOM International Corporation, Waterloo, Ontario, Canada.

"Intel486(TM) Microprocessor Family Programmer's Reference Manual", Intel Corporation.

"Extending DOS", Ray Duncan et al., 2nd ed. 1992, Addison-Wesley, ISBN 0-201-56798-9

VESA Super VGA Standard, Video Electronics Standards Association. http://www.vesa.org

"Programmer's Guide to the EGA and VGA Cards", Richard F. Ferraro, 2nd ed. 1990, Addison-Wesley, ISBN 0-201-57025-4

"The Universal VESA BIOS", Kendall Bennett, univbe.doc in the UNIVBE distribution. http://www.scitechsoft.com/