Notes on hobby os development
From Ggl's wiki
Contents |
Introduction
Reading Tanenbaum's books and others about actual implementations (linux, freebsd, opensolaris, mac os x and even windows) is great but it does not replace practice. These notes follow my journey in the writing of a hobby operating system.
Overview of an operating system
At the bottom layer, you have the hardware. Basically, you can consider:
- the CPU
- the memory
- I/O (input and output)
When you turn the power on, the CPU is put in a defined state. From this state you should setup the CPU with the features you want. Most processors follows a documented initialization routine. Then you need to setup the memory and map your code somewhere. After that, I'd guess anyone wants to interact with something. This is where I/O happens. For example, if the operating system only prints a "hello world!" message on the screen, it needs a way to communicate with the screen hardware. Most of the time, it's a complex piece of hardware and a controller is plug on a processor's I/O port.
The code of your operating system is compiled to a binary. You need to execute this binary on the target processor. The processor fetches the binary somewhere, either a ROM, EEPROM or a storage device. ROM/EEPROM is often directly addressable by the processor while a storage device implies I/O and a driver. However ROM/EEPROM is more expansive and provides few space. Let start simple and consider we write the binary on the ROM. We also need to know the entry point i.e. the offset where the operating system code begins. Usually, the code is splitted between the firmware which is more platform specific and handles the initialization of the process and the kernel of the operating system itself. When it ends the firmware transfers the execution to the kernel entry point.
Depending on the processor you may handle I/O by polling or interruption. Most processor provides interruptions. Consequently you should install interrupt service routines early in the initialization stage.
Coming back to our simple hello world, we need to map the screen memory and write the string at the expected offset. Then we send a sequence on the output port connected to the screen that tells to refresh the screen.
It's not very fun to have a operating system that only displays some characters on the screen. At least we want some input. We can display on the screen what we type on the keyboard. We need to know the I/O port connected to the keyboard and where its memory is mapped. We also need to install an interrupt handler on this port in order to call the handler each time the keyboard sends a keycode.
note: you might wonder why we still have not talk about memory management and scheduling? Because at the moment only one thread of execution is running and the memory we use is not larger than the physical memory.
Basic Bootloader for x86
The tutorials Hello World Boot Loader by David R. Faulkner and OSDev Babystep helped me a lot in this section.
Real-mode
A x86 processor starts in real-mode. This mode provides:
- 20-bit segmented memory address space i.e it uses one of the CS, DS, ES, FS, GS, and SS 16-bit segments and adds an 16-bit offset to it. Example: [DS:4B27] means multiply the value in DS by 16 (left shift of 4 bits = 0x0, 0xFFFF => 0xFFFF0) and add 0x4B27. If the value 0x7C00 is in DS:
0x7c00*16 + 0x4B27 = 0x80B27.
note: 0xFF = 256 = 2^8 is the maximum unsigned integer 8 bits can encode. It helps you to see the maximum according to the number of bits. For 16 bits, it is 0xFFFF. - 1 MB (usually less) of addressable memory.
0xFFFF*16 + 0xFFFF = 0xFFFF0 + 0xFFFF = 0x10FFeF 0xFFFF = 2^16 = 65535 = 64KiB 0xFFFF*16 = (2^16) * (2^4) = 2^(4 + 16) = 2^20 = 1MiB
- The segment [0xFFFF:offset] with 0x10 < offset < 0xFFFF defines the high-memory area in real-mode. It address memory from 0xFFFF0 + 0x10 = 0x100000 to 0xFFFFF + 0xFFFF = 0x10FFEF e.g. the last 64KiB segment.
- 16-bit operands and registers by default
- unlimited access to memory and I/Os
- access to the BIOS and BIOS functions
Boot Sequence
When the processor is switched on or reset it usually executes the code of the firmware interface located in an EEPROM (the first address the processor actually executes is defined by the reset vector). On a IBM PC or compatible hardware, the firmware is called BIOS (Basic Input/Output System). The BIOS runs series of diagnostics called POST (Power-On Self-Test). It includes the scanning of bootable devices in the order defined by the configuration. In this step, when it has found a suitable boot device, it loads the first 512-sector at the address 0x7C00 and jumps to this address.
So now we know the big picture of the boot sequence. We have to write 16-bit code with a base address at 0x7C00. This code is loaded from a block device. I use a floppy disk. It's simple and it works. At the beginning we can only address 1MiB of memory. This memory includes what the BIOS maps, then we have actually less than 1MiB.
Basic Boot Sector
We have to follow the structure the BIOS expects to have our code executed:
- 512 bytes long (in other words, one sector long on a block device)
- ends with the bootloader signature 0xAA55
- at address 0x7C00
note: remind the x86 has little-endian byte ordering.
Below is an example of a very basic bootloader that follow the structure we defined earlier:
[BITS 16] ; 16-bit code because it runs in real mode [ORG 0x7C00] ; Code origin set to 0x7C00 main: ; Main code label (Not really needed now but will be later) jmp $ ; Jump to the start of the instruction (never ending loop) ; An alternative would be 'jmp main' that would have the exact same ; effect. ; End matter times 510-($-$$) db 0 ; pad to have a 512 bytes binary file dw 0xAA55 ; bootloader signature
note: in Nasm 3.5 Expressions:
- $ is the current address i.e. it evaluates to the assembly position at the beginning of the line containing the expression
- $$ evaluates to the beginning of the current section; so you can tell how far into the section you are by using ($-$$). The code above has no section, so it evaluates to the beginning of the file. This way it calculates the size of the instructions before the current address.
Compiling and loading
I compile the assembly code with Nasm:
$ nasm boot4.asm -f bin -o boot.bin
- '-f bin' means flat binary file as the doc (nasm -hf) explains: "flat-form binary files (e.g. DOS .COM, .SYS)". This way the binary only contains the opcodes and values from the assembly code, without any section or address relocation (see also 7.1 bin: Flat-Form Binary Output from the manual.
$ hexdump boot.bin 0000000 feeb 0000 0000 0000 0000 0000 0000 0000 0000010 0000 0000 0000 0000 0000 0000 0000 0000 * 00001f0 0000 0000 0000 0000 0000 0000 0000 aa55 0000200
Two opcodes, zero padding and the bootloader signature at the end :). 0x0200 is 512:
$ printf "%d\n" 0x0200 512
I test the binary image as a floppy we qemu:
$ qemu -bios /usr/share/kvm/bios.bin -fda boot.bin
It displays nothing and frenetically loops:
Booting from floppy...
However we can see it running because the qemu process eats 100% of the CPU.
Printing with the BIOS
Now let write a message to the screen using the BIOS interrupt int 10 (and AH=0x0E) (inspired by OSDev Wiki Babystep2):
; print a message to the screen using BIOS interrupts
[ORG 0x7c00]
xor ax, ax ; set ax register to zero
mov ds, ax ; set DS (Data Segment) to zero
mov si, msg ; set index of data to msg's address
; string instructions use DS content as base address and
; SI as index (SI is a 16-bit register)
putstr:
lodsb ; load byte from source operand (DS:SI) into AL
or al, al ; end of string (0) ?
jz hang
mov ah, 0x0E ; print to screen the content of AL with int 0x10
int 0x10
jmp putstr
hang:
jmp $
msg db 'Hello world!', 13, 10, 0
times 510 - ($-$$) db 0
dw 0xAA55
Handling Keyboard Input
Next, we install a interruption service routine (ISR) to handle keyboard interrupt and display the key in hex. We directly write in video memory and do not call the BIOS (cf Babystep4 and Babystep5):
[ORG 0x7c00]
jmp start ; jump to entry point
start:
; Set registers
xor ax, ax
mov ds, ax ; set DS (Data Segment)
mov ss, ax ; set SS (stack segment)
mov fs, ax ; set FS (extra data segment)
mov sp, 0x9c00
mov ax, 0xb800 ; VGA text coor (32KiB) memory area
; http://wiki.osdev.org/Memory_Map_(x86)#ROM_Area
mov es, ax ; set ES (segment that holds destination of string instructions)
mov si, msg ; put msg address in Source Index register for string instructions
call sprint ; print message to screen
; Set interruptable table entry for keyboard interrupts
cli ; Clear Interrupt Flag => disable interrupts
; IRQ1 (keyboard) is mapped on hardware interrupt 9
; multiply by 4 to find offset in interrupt table
mov word [ds:9*4], keyhandler
; put the address of the procedure keyhandler at offset in
; interrupt table
mov word [ds:9*4+2], ds
; put the segment address in interrupt table
sti ; Set Interrupt Flag => enable interrupts again
jmp $
; print "%d\n", keycode
keyhandler: ; http://wiki.osdev.org/PS2_Keyboard
push ax ; save ax
in al, 0x60 ; read keyboard I/O port
mov bl, al ; save value in BL (used later to handle break key code)
mov byte [port60], al
; save al at [port60]. It is overwritten below to ack the interrupt by
; writting on the 0x20 port
; http://wiki.osdev.org/PS2_Keyboard#Receiving_Input continuously receive input from keyboard
; in al, 0x61
; mov ah, al
; or al, 0x80
; out 0x61, al
; xchg ah, al
; out 0x61, al
mov al, 0x20 ; tell the first PIC that the IRQ (Interrupt ReQuest) is handled
out 0x20, al ; i.e. acknowledge the interrupt
; http://wiki.osdev.org/Interrupts#From_the_OS.27s_perspective:
and bl, 0x80 ; handle break key code (key released)
jnz done
mov ax, [port60] ; restore ax from the backup value at [port60]
mov word [reg16], ax; put the lower 16 bits of ax at [reg16]
call printreg16
done:
pop ax ; restore ax
iret ; return from interrupt handler
dochar: call cprint
sprint:
lodsb ; load the byte in DS
cmp al, 0 ; does the byte in AL equal zero (end of string)?
jne dochar ; if zero flag is not set, jump to dochar
add byte [ypos], 1 ; move cursor to next line
mov byte [xpos], 0 ; move cursor to the beginning of the line
ret
cprint:
mov word dx, [attr] ; current character attribute
mov ah, dl ; define character attribute
next_color:
inc dx ; next color
and dx, 0x0f ; only foreground color
jz next_color ; not black no black
mov word [attr], dx
push ax
movzx ax, byte [ypos] ; move byte at ypos in AX and pad with zeros i.e. set ax with y position
mov dx, 160 ;
mul dx ; DX:AX <- DX * AX = 160 * ypos
movzx bx, byte [xpos] ; move byte at xpos in BX and pad with zeros i.e. set bx with x position
shl bx, 1 ; multiply BX by 2
mov di, 0 ; start of video memory
add di, ax ; add ax (y position)
add di, bx ; add bx (x position)
pop ax
stosw
add byte [xpos], 1
ret
printreg16:
mov di, outstr16 ; set outstr16 as the destination string
mov ax, [reg16] ; put the 16-bit value at [reg16] in ax
mov si, hexstr ; set hexstr as the source string
mov cx, 4 ; loop counter
hexloop: ; do { ... } while (--cx);
rol ax, 4 ; for example 'a' = 0x001e. rol ax, 4 => ax <- 01e0, 1e00, e001, 001e
mov bx, ax ; bx <- 01e0
and bx, 0x0f ; bx <- 0x0f & 0x01e0 = 0, 0, 1, e
mov bl, [si+bx] ; bl <- hexstr[1] = '0', '0', '1', 'e'
mov [di], bl ; outstr16[0] = '0', '0', '1', 'e'
inc di ; next wil set outstr16[1]
dec cx ; --cx
jnz hexloop ; loop if cx != 0
mov si, outstr16 ; string ready to print
call sprint
ret
msg db 'Hobby OS Welcome! >', 0
xpos db 0
ypos db 0
port60 dw 0
hexstr db '0123456789ABCDEF'
outstr16 db '0000', 0
reg16 dw 0
attr dw 0x0e
times 510-($-$$) db 0
dw 0xAA55
We directly talk to the Intel 8042 keyboard controller. It means we get scancodes. A scancode identify a keyboard event. When you press a key, the controller sends a make code. When you release a key it sends a break code. For example, you press if you press the 'a' key. You get the make code 0x1e. The corresponding break code is 0x1e + 0x80.
We may improve this basic bootloader by printing the actual character instead of the scancode. This way the bootloader would provide a simple command line. The main objective of the bootloader is to set the processor's state and load a kernel. We want to run the kernel in protected mode. Then we will set the A20 gate and enter the protected. Finally we transfer the execution to the kernel. The section Convenience to operating systems of the multiboot specification explains a way to provide a common interface to an operating system. It addresses the issue of handling various executable file formats. The bootloader plays the role of the loader. Your kernel is compiled to a binary file. This file follows a executable format as ELF or a.out. The loader has to find the entry point as well as the section it should relocates. When the compiler generates the object file it writes addresses that might change when the binary is loaded because several binaries may share the same address space.
Entering Protected Mode
cli
mov eax,cr0
or eax,1
mov cr0,eax
; set registers
mov ax,10h
mov ds,ax
mov es,ax
mov fs,ax
mov gs,ax
mov ss,ax
jmp 8:10000h
sti
Multiboot
In order to be multiboot-compliant a kernel must contain the multiboot header. The Grub Multiboot documentation provides an example OS code.
Loading the kernel
The bootloader reads the kernel file from the boot block device, loads it in memory and jump to the entry point. It can either use the BIOS or implement a basic ATA driver. If it uses the BIOS, it needs to run in real-mode. However the kernel may be larger than 1MB. In this case, the bootloader cannot access to the higher parts of its memory. Then it should run in unreal mode to switch between real-mode and protected mode while it loads the kernel to memory. The alternative is to switch to protected mode and interface the bootloader with the floppy disk controller or the ATA controller in PIO mode or DMA.
References:
- Loading sectors
- How to program the DMA
- Brans kernel development tutorial
- Detecting floppy drives
- Simple linker script
VGA programming
Resources
- VBE:in real mode or in protected mode (here VBE stands for VESA VBE but some answers are really interesting)
- VGA Hardware
- Tutorial on VGA Graphics
Basic Kernel for x86 (IA32)
Where to begin? The journey started from Bare Bone at OSDev.org. First step: boot a dummy kernel using grub on a floppy.
A common roadmap for a kernel is:
- Initialize processor
- Switch in protected mode
- Jump to the entry point
- Install ISR (Interrupt Vector)
The target architecture is Intel IA32. Then grub initializes the processor, switches to protected mode and maps the kernel elf32 binary. Once the kernel is executing we need to install some core structures: GDT (Global Descriptor Table) and IDT (Interrupt Descriptor Table).
What we want to do?
Let begin by a basic echo command line. Thus we want to get input from the keyboard (using a custom ps/2 driver) and write it on the screen.
To catch the keycode from the keyboard we need to install an interrupt handler (also called an Interrupt Service Routine). The PIC (Programmable Interrupt Controller) maps the keyboard on interrupt (IRQ) 1.
Resources:
- Protected mode programming and O/S development in Phrack #52-17
