2. Contents
Virtual Memory
Virtual Address Space: User/Kernel
Program Structure
Process' Virtual Address Space
– brk, sbrk
– mmap
– malloc, free, mallopt, mallinfo
– fork, vfork
– execv
The Content is specific to GNU/Linux System running on x86
3. Virtual Memory
Virtual memory is a technique that allows the
execution of processes that are not completely in Virtual Address Space of
physical memory a Process
The logical (or virtual) view
●
Programs can be larger than physical memory of how a process is stored
in memory
●
Separates the logical memory (large) as viewed by
the user from physical memory (small)
●
Makes the task of programming much easier, because
the programmer no longer needs to worry about the
amount of physical memory available
●
Libraries can be shared by several processes through
mapping of the shared object into a virtual address
space. Although each process considers the shared
libraries to be a part of its virtual address space, the
actual physical pages where the libraries reside in
physical memory are shared by all the processes
●
Allows one process to create a region of memory that
it can share with another process. Processes sharing
this region consider it part of their virtual address
space, yet the actual physical pages of memory are
shared
●
Allows for more efficient process creation
4. Virtual Memory
●
Virtual Address: Address generated by CPU
●
Physical Address: Actual address of the physical memory (RAM)
●
MMU does the virtual to physical address translation
●
The physical memory available in a system may be less than the virtual
memory
●
E.g. OMAP3430:
●
32 bit virtual addresses
●
Total virtual address space: 4 GB
●
The Virtual address space is split into two parts:
●
User space, which potentially changes with each full context
switch
●
Kernel space, which remains constant
●
The virtual memory is divided into pages (4 KB is typical). Backing
each page of virtual memory is a page of physical memory or some
secondary storage
●
In order for a process to access any part of a virtual page, the page
must at that moment be backed by (“connected to”) a page in the
physical memory. But because there is usually a lot more virtual memory
than real memory, the pages must move back and forth between main
memory and secondary storage regularly, coming into main memory
when a process needs to access them and then retreating to backing
store when not needed anymore. This movement is called paging. When
a program attempts to access a page which is not at that moment backed
by real memory, this is known as a page fault. When a page fault
occurs, the kernel suspends the process, places the page into the
physical memory (this is called “paging in”), then resumes the process
so that from the process’ point of view, the page was in physical memory
all along
5. Virtual Address Space Example
0x0000 0000
...
...
...
...
... User Space (3 GB )
...
...
...
...
...
...
TASK_SIZE PAGE_OFFSET 0xC000 0000
...
... Kernel Space (1 GB)
...
0xFFFF FFFF
6. Program Structure (ELF Format)
ELF Header
Program Header Table
Section Header Table
.symtab TEXT Segment (Loadable):
.strtab Contains read-only data and instructions
etc
.hash
.dynsym
.dynstr
.rel.dyn
.rel.plt
.init
.plt
.text DATA Segment (Loadable):
.fini Contains writable data and instructions
.rodata
etc
.ctors
.dtors
.dynamic
.got
.got.plt
.data
.bss
etc
7. Process' Virtual Address Space
0x0804 8000
TEXT SEGMENT 0x0804 8804 main()
0x0804 b000
0x0804 b1a0
gcArray [100000]
DATA SEGMENT 0x0806 383f
0x09d1 4000 (brk)
Available for:
HEAP growth 2.71 GB
and
mmap
0xb7e1 a000
Shared Libraries
0xb7f7 5000
0xb7f8 b000
Shared Libraries
0xb7fa 8000
Available for STACK growth
0xbfd9 2000
Stack 0xbfda 4a9c
lcArray [100]
0xbfda 4aff
0xbfd a 7000
8. brk(), sbrk(), malloc(), free()
int brk (void *addr)
void *sbrk (ptrdiff_t delta)
These functions are used to resize the Data Segment
System call: brk
●
brk() sets the high end of the calling process' Data Segment to addr
●
sbrk() is same as brk() except that the new end of the Data Segment is specified as an
offset delta. sbrk(0) gives you the current end of the Data Segment
void *malloc (size_t size)
This function is used to allocate a new size bytes long block
Uses:
●
sbrk()
OR
●
mmap() (for large sized blocks). This has great advantage that these chunks are returned to
the system immediately when they are freed. Therefore it cannot happen that a large chunk
becomes 'locked' in between smaller ones and, even after calling free(), wastes memory
void free (void *ptr)
This function deallocates the the block of memory pointed at by ptr
Occasionally, free() can actually return memory to the operating system and make the process
smaller. Usually, all it can do is allow a later call to malloc() to reuse the space. In the meantime,
the space remains in your program as part of a free-list used internally by malloc()
9. malloc (100000)
0x0804 8000 0x0804 8000
TEXT SEGMENT 0x0804 8804 main() TEXT SEGMENT 0x0804 8804 main()
0x0804 b000 0x0804 b000
0x0804 b1a0 0x0804 b1a0
gcArray [100000] gcArray [100000]
DATA SEGMENT 0x0806 383f 0x0806 383f
DATA SEGMENT
0x09d1 4000 (brk) 0x09d1 4008 Obtained by
0x09d2 c6a7 malloc(100000)
0x09d4 d000 (brk)
Available for: malloc (100000)
HEAP growth Available for:
and HEAP growth
mmap and
mmap
0xb7e1 a000 0xb7e1 a000
Shared Libraries Shared Libraries
0xb7f7 5000 0xb7f7 5000
0xb7f8 b000 0xb7f8 b000
Shared Libraries Shared Libraries
0xb7fa 8000 0xb7fa 8000
Available for STACK growth Available for STACK growth
0xbfd9 2000 0xbfd9 2000
Stack 0xbfda 4a9c Stack 0xbfda 4a9c
lcArray [100] lcArray [100]
0xbfda 4aff 0xbfda 4aff
0xbfd a 7000 0xbfd a 7000
10. mmap()
void *mmap (void *address, size_t length, int protect, int flags, int filedes,
off_t offset)
This function creates a new mapping, connected to bytes (offset) to (offset + length-1) in the
file open on filedes
System call: mmap
E.g.
char *buf = mmap (NULL, 1MB, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)
●
address: Preferred starting address for the mapping. NULL expresses no preference
●
protect: Access permissions
●
flags:
●
MAP_PRVATE: Specifies that the writes to region should never be written back to the
attached file. Instead, a copy is made for the process
●
MAP_SHARED: This means that the writes to the region will be written back to the file.
Changes will be shared with other processes mmaping the same file
●
MMAP_ANONYMOUS: Tells the system to create an anonymous mapping, not connected
to a file. The region is initialized with zeros
malloc() uses mmap() with MMAP_ANONYMOUS to allocate large sized blocks
11. mmap (NULL, 1MB, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)
0x0804 8000 0x0804 8000
TEXT SEGMENT 0x0804 8804 main() TEXT SEGMENT 0x0804 8804 main()
0x0804 b000 0x0804 b000
0x0804 b1a0 0x0804 b1a0
gcArray [100000] gcArray [100000]
0x0806 383f 0x0806 383f
DATA SEGMENT DATA SEGMENT
0x09d1 4008 Obtained by 0x09d1 4008 Obtained by
0x09d2 c6a7 malloc(100000) 0x09d2 c6a7 malloc(100000)
0x09d4 d000 (brk) 0x09d4 d000 (brk)
mmap (...,1MB,...)
Available for:
HEAP growth
and
mmap 0xb7d1 9000 Mapped initial
data.txt 1 MB of
0xb7e1 8fff data.txt using
mmap()
0xb7e1 a000 0xb7e1 a000
Shared Libraries Shared Libraries
0xb7f7 5000 0xb7f7 5000
0xb7f8 b000 0xb7f8 b000
Shared Libraries Shared Libraries
0xb7fa 8000 0xb7fa 8000
Available for STACK growth Available for STACK growth
0xbfd9 2000 0xbfd9 2000
Stack 0xbfda 4a9c Stack 0xbfda 4a9c
lcArray [100] lcArray [100]
0xbfda 4aff 0xbfda 4aff
0xbfd a 7000 0xbfd a 7000
12. malloc (1MB)
0x0804 8000 0x0804 8000
TEXT SEGMENT 0x0804 8804 main() TEXT SEGMENT 0x0804 8804 main()
0x0804 b000 0x0804 b000
0x0804 b1a0 0x0804 b1a0
gcArray [100000] gcArray [100000]
0x0806 383f 0x0806 383f
DATA SEGMENT DATA SEGMENT
0x09d1 4008 Obtained by 0x09d1 4008 Obtained by
0x09d2 c6a7 malloc(100000) 0x09d2 c6a7 malloc(100000)
0x09d4 d000 (brk) 0x09d4 d000 (brk)
malloc (1MB)
0xb7c1 8008 Obtained by
MMAP_ANONYMOUS malloc(1MB)
0xb7d1 8007
0xb7d1 9000 Mapped initial 0xb7d1 9000 Mapped initial
data.txt 1 MB of data.txt 1 MB of
0xb7e1 8fff data.txt using 0xb7e1 8fff data.txt using
mmap() mmap()
0xb7e1 a000 0xb7e1 a000
Shared Libraries Shared Libraries
0xb7f7 5000 0xb7f7 5000
0xb7f8 b000 0xb7f8 b000
Shared Libraries Shared Libraries
0xb7fa 8000 0xb7fa 8000
Available for STACK growth Available for STACK growth
0xbfd9 2000 0xbfd9 2000
Stack 0xbfda 4a9c Stack 0xbfda 4a9c
lcArray [100] lcArray [100]
0xbfda 4aff 0xbfda 4aff
0xbfd a 7000 0xbfd a 7000
13. mallopt(), mallinfo()
int mallopt (int param, int value)
This function is used to adjust some parameters for dynamic memory allocation
Parameters:
●
M_MMAP_THRESHOLD: All chunks larger than this value are allocated outside the normal
heap, using mmap()
●
M_MMAP_MAX: The maximum number of chunks to allocate with mmap()
●
M_TOP_PAD: The amount of extra memory to obtain from the system when a call to sbrk()
is required. It also specifies the number of bytes to retain when shrinking the heap by
calling sbrk() with a negative argument. This provides the necessary hysteresis in in heap
size such that excessive amounts of system calls can be avoided
●
M_TRIM_THRESHOLD: The minimum size (in bytes) of the top-most, releasable chunk that
will call sbrk() to be called with a negative argument in order to return memory to the
system
struct mallinfo mallinfo (void)
This function is used to get information about dynamic memory allocator
struct mallinfo
●
arena: Total size of memory allocated with sbrk() by malloc(), in bytes
●
hblkhd: Total size of memory allocated with mmap() by malloc(), in bytes
●
fordblks: Total size of memory occupied by free (not in use) chunks, in bytes
●
etc
14. fork()
pid_t fork(void)
This function is used to create a new process
System call: fork
●
Called by the parent
●
Called once, returns twice
●
Return value in child: 0
●
Return value in parent: pid of the child
●
Creates a complete copy of the Virtual Address Space of the parent for the child
●
A technique known as Copy-On-Write is used
●
Initially the parent and the child share the same physical pages in their address spaces
●
These pages are marked as copy-on-write, meaning that if either process writes to the
shared page, a copy of the shared page is created
15. P1 calls fork()
P1 (parent) P2 (child)
0x0804 8000 0x0804 8000
TEXT SEGMENT 0x0804 8804 main() TEXT SEGMENT 0x0804 8804 main()
0x0804 b000 0x0804 b000
0x0804 b1a0 0x0804 b1a0
gcArray [100000] gcArray [100000]
0x0806 383f 0x0806 383f
DATA SEGMENT DATA SEGMENT
0x09d1 4008 Obtained by 0x09d1 4008 Obtained by
0x09d2 c6a7 malloc(100000) 0x09d2 c6a7 malloc(100000)
0x09d4 d000 (brk) 0x09d4 d000 (brk)
0xb7c1 8008 Obtained by 0xb7c1 8008 Obtained by
MMAP_ANONYMOUS malloc(1MB) MMAP_ANONYMOUS malloc(1MB)
0xb7d1 8007 0xb7d1 8007
0xb7d1 9000 Mapped initial 0xb7d1 9000 Mapped initial
data.txt 1 MB of data.txt 1 MB of
0xb7e1 8fff data.txt using 0xb7e1 8fff data.txt using
mmap() mmap()
0xb7e1 a000 0xb7e1 a000
Shared Libraries Shared Libraries
0xb7f7 5000 0xb7f7 5000
0xb7f8 b000 0xb7f8 b000
Shared Libraries Shared Libraries
0xb7fa 8000 0xb7fa 8000
Available for STACK growth Available for STACK growth
0xbfd9 2000 0xbfd9 2000
Stack 0xbfda 4a9c Stack 0xbfda 4a9c
lcArray [100] lcArray [100]
0xbfda 4aff 0xbfda 4aff
0xbfd a 7000 0xbfd a 7000
16. Copy-On-Write
Before Process 1
modifies page C
After Process1
modifies page C
Copy of
page C
17. execv()
int execv (const char *pathname, char *const argv[])
This function is used to execute a program
System call: execve
●
Terminates the currently running program
●
Replaces the current process (Text, Data, Heap, Stack) with a new program from disk
●
The pages of the binary file are mapped into regions of Virtual Address Space of the
process. Only when the program tries to access a given page will a page fault result in the
loading of that page into physical memory using Demand Paging
●
Executes the new program in the context of the existing process
18. P2 calls execv()
P2 (child) P2 (child)
0x0804 8000 0x0804 8000
TEXT SEGMENT 0x0804 8804 main() TEXT SEGMENT 0x0804 8554 main()
0x0804 b000 0x0804 a000
0x0804 b1a0
gcArray [100000]
0x0806 383f DATA SEGMENT
DATA SEGMENT
0x09d1 4008 Obtained by 0x0937 4000 (brk)
0x09d2 c6a7 malloc(100000)
0x09d4 d000 (brk)
execv()
Available for:
HEAP growth
0xb7c1 8008 Obtained by and
MMAP_ANONYMOUS malloc(1MB) mmap
0xb7d1 8007
0xb7d1 9000 Mapped initial
data.txt 1 MB of
0xb7e1 8fff data.txt using
mmap()
0xb7e1 a000 0xb7e3 c000
Shared Libraries Shared Libraries
0xb7f7 5000 0xb7f9 7000
0xb7f8 b000 0xb7fa d000
Shared Libraries Shared Libraries
0xb7fa 8000 0xb7fc a000
Available for STACK growth Available for STACK growth
0xbfd9 2000 0xbfcb 5000
Stack 0xbfda 4a9c Stack
lcArray [100]
0xbfda 4aff
0xbfd a 7000 0xbfcc a000
19. vfork()
pid_t vfork(void)
This function is used to create a new process when the purpose of the new process is to
exec a new program
System call: vfork
●
The child process shares the Virtual Address Space of the parent
●
The parent process is suspended till the child calls exec or exit
●
If the child process (before calling exec or exit) changes any pages of the parent's Address
Space, the altered pages will be visible to the parent once it resumes
20. References
●
Operating System Principles, 7th ed, Silberschatz, Galvin, Gagne
●
Understanding the Linux Virtual Memory Manager, Mel Gorman
●
The GNU C Library Reference Manual 0.12 ed
●
Advanced Programming in the UNIX Environment, W. Richard Stevens
●
Tool Interface Standard (TIS) Executable and Linking Format (ELF) Specification
Version 1.2
●
OMAP3430 TRM
●
http://www.tenouk.com/Bufferoverflowc/Bufferoverflow1c.html
●
http://www.linuxjournal.com/article/6059
●
http://www.cis.gvsu.edu/~wolffe/courses/cs656/projects/tutorial_UNIX.html