Custom Operating System Series
Part One : Configuring VMware Player
This article is the first in a series introducing system software development concepts for Intel x86 or compatible processors running in protected virtual address mode, or protected mode.
In this series we will develop a simple protected mode operating system in assembly language.
Concepts, techniques and tools introduced in this series of notes will include:
VMware Player Configuration
The NASM Assembler
The PC BIOS Boot Process
Protected Mode Operation
Throughout this series, we'll use VMware Player to provide a virtual computing environment for the operating system.
This approach allows us to use the same host operating environment to both develop and test our software and reduces the risk of causing unintended damage to our host during development.
In this first article, we will download and configure VMware Player to launch a virtual machine that boots from a virtual floppy disk image file (os.flp).
This will provide a starting point for introducing more concepts and techniques in later articles.
Step 1. Download VMware Player
In this series, we are using Windows 7 as the host operating system.
VMware Player is available for download from www.vmware.com.
Downloading VMware Player is free, but might require you to register prior to downloading.
As of this writing, the link to register and download VMware Player is
In our case, the installation file is named VMware-player-6.0.3-1895310.exe.
Launch the program once you have downloaded it.
During installation, you may be prompted to restart your computer.
Step 2. Install VMware Player
Follow the instructions presented by the VMware Player installation program.
If a computer restart is required, re-launch the installation program after the restart and continue to step through the installer screens.
After VMware Player is installed, launching the VMware Player opens the Home window:
Step 3. Create a New Virtual Machine
Select the "Create a New Virtual Machine" link on the Home window to open the New Virtual Machine Wizard.
We will select the third radio-buton option "I will install the operating system later" and click "Next".
Select "Other" as the Guest Operating System and "Other" as the Version. Click "Next".
You can name your virtual machine as you like.
We use "OS" as the name of the virtual machine and choose a location on our host operating system's file system to store the VMware files for the virtual machine.
Then click "Next".
Make sure you do not select a location where you already have a virtual machine's files stored.
We've chosen to leave the default value for the "Maximum disk size" and select the "Split virtual disk into multiple files" option.
Then click "Next".
Even though the selected maximum disk size is several Gigabytes, VMware will not initially use that much disk space.
The next wizard screen displays our chosen settings for the new virtual machine.
We will return later to customize our hardware.
For now, we click "Finish" so that our new directory will be created.
Step 4. Create a Floppy Disk Image File
The VMware Player now lists our new virtual machine, "OS", on the Home screen.
At this point, the virtual machine is "playable", but it will merely attempt to boot over the network because there are no bootable disks.
We will click "Edit virtual machine settings" to customize our floppy disk settings so that our operating system will load from a local disk image file.
Recent versions of VMware Player no longer include a Floppy Disk device by default.
We will add such a device to emulate booting the operating system from floppy disk image.
Click the "Add...." button to select additional hardware to emulate.
Highlight the "Floppy Drive" option and click "Next."
Our floppy drive image file does not yet exist, so select the "Create a blank floppy image" radio button and click "Next".
Navigate to the folder where our virtual machine files are stored and enter a name for the floppy disk image file.
Make sure the "Connect at power on" check-box is checked. Click "Finish".
Our virtual machine settings now show a Floppy Drive device.
Now if we look at the directory that we created for our virtual machine files, we will also see our "os.flp" file, which has been created as a 1.44MB file to replicate a 3 1/2" double-sided double-density diskette.
Even though we have added a floppy disk to our virtual machine, it will still try to boot from the network when we start it.
This is because our floppy disk image has not been prepared with a boot sector containing code to load an operating system.
In our next article, we will assemble a simple program into the very first sector on the floppy disk image file and demonstrate how VMWare Player will load and run this program.
Revised 10 October 2014