1. 1
22CS201
Computer Organization and Architecture
Module – III
Dr.A.Kathirvel, Dean, Computing Cluster
Sri Krishna College of Technology, Coibatore

2. 2
22CS201 Computer Organization and Architecture
Module III - Parallel Processors and Memory Organization
Parallel Processors: Introduction to parallel processors, Concurrent
access to memory and cache coherency. Introduction to multicore
architecture. Memory system design: semiconductor memory
technologies, memory organization. Memory interleaving, concept
of hierarchical memory organization, cache memory, cache size vs.
block size, mapping functions, replacement algorithms, write
policies.
Case Study: Instruction sets of some common CPUs - Design of a
simple hypothetical CPU- A sequential Y86-64 design-Sun Ultra
SPARC II pipeline structure.
Dr.A.Kathirvel, Professor & DEAN,
DCSE, SKCT
Kathirvel.a@skct.edu.in

3. 3
Module III
Text Books:
1. David A. Patterson and John L. Hennessy, “Computer Organization and
Design: The Hardware/Software Interface”, 6th Edition, Morgan
Kaufmann/Elsevier, 2020.
2. 2. Carl Hamacher, Zvonko Vranesic, Safwat Zaky, Naraig Manjikian,
“Computer Organization and Embedded Systems”, McGraw- Hill, 6th
Edition 2017.
Reference Books:
1. John P. Hayes, “Computer Architecture and Organization”, McGraw-Hill,
3rd Edition, 2017
2. William Stallings, “Computer Organization and Architecture Designing for
Performance”, 11th Edition, Pearson Education 2018.
3. Vincent P. Heuring and Harry F. Jordan, “Computer System Design and
Architecture”, 2nd Edition, Pearson Education 2004.

4. Parallel Processing
• Introduction
• Concurrent access to memory
• Cache coherency

5. PARALLEL PROCESSING
Parallel Processing
• Instead of processing each instruction sequentially, a parallel
processing system provides concurrent data processing to improve
the execution time.
• In this the system may have two or more ALU's and should be able
to execute two or more instructions at the same time.
• The purpose of parallel processing is to speed up the computer
processing capability and increase its throughput.
• parallel processing can be achieved by using multiple functional
units that perform many operations simultaneously.

6. PARALLEL PROCESSING

7. PARALLEL PROCESSING
Data Transfer Modes of a Computer System
• According to the data transfer mode, computer can be divided into 4
major groups:
– SISD
– SIMD
– MISD
– MIMD

8. PARALLEL PROCESSING
• SISD (Single Instruction Stream, Single Data Stream)
– It represents the organization of a single computer containing a
control unit, processor unit and a memory unit. Instructions are
executed sequentially. It can be achieved by pipelining or multiple
functional units.

9. PARALLEL PROCESSING
• SIMD (Single Instruction Stream, Multiple Data Stream)
– It represents an organization that includes multiple processing units under
the control of a common control unit. All processors receive the same
instruction from control unit but operate on different parts of the data.
– They are highly specialized computers. They are basically used for
numerical problems that are expressed in the form of vector or matrix.
– SIMD is typically used to analyze large data sets that are based on the
same specified benchmarks.
– But they are not suitable for other types of computations

10. PARALLEL PROCESSING
• MISD (Multiple Instruction Stream, Single Data Stream)
– It consists of a single computer containing multiple processors
connected with multiple control units and a common memory unit.
– It is capable of processing several instructions over single data
stream simultaneously.
– MISD structure is only of theoretical interest since no practical
system has been constructed using this organization.

11. PARALLEL PROCESSING
• MIMD (Multiple Instruction Stream, Multiple Data Stream
– It represents the organization which is capable of processing
several programs at same time.
– It is the organization of a single computer containing multiple
processors connected with multiple control units and a shared
memory unit.
– The shared memory unit contains multiple modules to
communicate with all processors simultaneously.
– Multiprocessors and multicomputer are the examples of MIMD. It
fulfills the demand of large scale computations.

12. PARALLEL PROCESSING
Shared-Memory Multicomputers

13. PARALLEL PROCESSING
Shared-Memory Multicomputers
• In a shared-memory multiprocessor, all processors have access to the
same memory.
• Tasks running in different processors can access shared variables in
the memory using the same addresses.
• The memory are distributed into multiple modules, so that simultaneous
requests from different processors are more likely to access different
memory modules.
• Three most common shared memory multiprocessors models are
– Uniform Memory Access (UMA)
– Non-uniform Memory Access (NUMA)
– Cache Only Memory Architecture (COMA)

14. PARALLEL PROCESSING
Uniform Memory Access (UMA)

15. PARALLEL PROCESSING
Uniform Memory Access (UMA)
• An interconnection network enables any processor to access any
module that is a part of he shared memory.
• When memory modules are kept physically separate from the
processors, all requests to access memory must pass through the
network, which introduces latency.
• A system which has the same network latency for all accesses from
the processors to the memory modules is called a Uniform Memory
Access (UMA) multiprocessor.
• Although the latency is uniform, it may be large for a network that
connects many processors and memory modules.

16. PARALLEL PROCESSING
Uniform Memory Access (UMA)
• Each processor may have a private cache memory.
• Same rule is followed for peripheral devices
• When all the processors have equal access to all the peripheral
devices, the system is called a symmetric multiprocessor.
• When only one or a few processors can access the peripheral
devices, the system is called an asymmetric multiprocessor.

17. PARALLEL PROCESSING
Non-uniform Memory Access (NUMA)

18. PARALLEL PROCESSING
Non-uniform Memory Access (NUMA)
• For better performance(to reduce the latency), place a memory module
close to each processor.
• The result is a collection of nodes, each consisting of a processor and a
memory module. The nodes are then connected to the network.
• The network latency is avoided when a processor makes a request to
access its local memory.
• However, a request to access a remote memory module must pass
through the network.
• Because of the difference in latencies for accessing local and remote
portions of the shared memory, systems of this type are called Non-
Uniform Memory Access (NUMA) multiprocessors.

19. PARALLEL PROCESSING
Cache Only Memory Architecture (COMA)

20. PARALLEL PROCESSING
Cache Only Memory Architecture (COMA)
• The COMA model is a special case of the NUMA model.
• Here, all the distributed main memories are converted to cache
memories.
• The local memories (typically DRAM) at each node are used as cache.
• In NUMA, When processors access some data, a copy is made in their
local cache, but space remains allocated in the home node.
• Instead, with COMA, there is no home. An access from a remote node
may cause that data to migrate.
• Compared to NUMA, this reduces the number of redundant copies and
may allow more efficient use of the memory resources.

21. PARALLEL PROCESSING
Cache Coherence

22. PARALLEL PROCESSING
Cache Coherence
• Each variable in a program has a unique address location in the memory,
which can be accessed by any processor.
• Each processor has its own cache. When any processor writes to a
shared variable in its own cache, all other caches that contain a copy of
that variable will then have the old, incorrect value.
• So, They must be informed of the change so that they can either update
their copy to the new value or invalidate it.
• This is the issue of maintaining cache coherence, which requires having
a consistent view of shared data in multiple caches.
• The approach, Similar to write-through approach or write-back approach
can be used to address cache coherence in a multiprocessor system.

23. PARALLEL PROCESSING
1.Write-Through Protocol
• A write-through protocol can be implemented in one of two ways.
– updating the values in other caches,
– invalidating the copies in other caches.

24. PARALLEL PROCESSING
1.Write-Through Protocol
a) Update protocol
• When a processor writes a new value to a block of data in its cache, the
new value is also written into the memory module containing the block
being modified.
• Since copies of this block may exist in other caches, these copies must
be updated to reflect the change caused by the Write operation.
• The simplest way of doing this is to broadcast the written data to the
caches of all processors in the system.
• As each processor receives the broadcast data, it updates the contents
of the affected cache block if this block is present in its cache.

25. PARALLEL PROCESSING
1.Write-Through Protocol
b) Invalidation of copies.
• When a processor writes a new value into its cache, this value is
also sent to the appropriate location in memory, and all copies in
other caches are invalidated.
• Again, broadcasting can be used to send the invalidation requests
throughout the system.

26. PARALLEL PROCESSING
2.Write-Back protocol
• This method is based on the concept of ownership of a block of data.
• Initially, the memory is the owner of all blocks, and the memory
retains ownership of any block that is read by a processor to place a
copy in its cache.
Getting ownership
• If any processor wants to write to a block in its cache, it must first
become the exclusive owner of this block.
• To do so, all copies in other caches must first be invalidated with a
broadcast request.
• The new owner of the block may then modify the contents.

27. PARALLEL PROCESSING
2.Write-Back protocol
Changing ownership
• When another processor wishes to read a modified block , the
request must be forwarded to the current owner.
• The data are then sent to the requesting processor. and the
appropriate memory module by the current owner.
• Memory module reacquires ownership and updates the contents of
the block in the memory.
• The block is now shared with two caches and the memory.
• Subsequent requests from other processors to read the same block
are serviced by the memory module containing the block.

28. PARALLEL PROCESSING
2.Write-Back protocol
• When another processor wishes to write to a block that has been
modified, the current owner (memory module) sends the data to the
requesting processor. It also transfers ownership of the block to the
requesting processor and invalidates its cached copy.
Advantages
• creating less traffic than the write-through protocol. (may perform
several writes to a cache block before this block is needed by
another processor)

29. PARALLEL PROCESSING
3.Snoopy Caches
• In a single-bus system, all transactions between processors and memory
modules occur via requests and responses on the bus. So,
it broadcast to all units connected to the bus.
• Each processor cache has a controller circuit that observes, or snoops, all
transactions on the bus.
Write into block
• Consider a processor has a copy of a block and before writing to this block for
the first time, the processor get the ownership and broadcast an invalidation
request to all other caches.
• The controllers of other processor cache ,accept the request and invalidate any
copies of the same block.
• The processor may then write to the block and mark it as being modified

30. PARALLEL PROCESSING
3.Snoopy Caches
Read request
• Now, if another processor broadcasts a read request on the bus for
the same block, the memory must not respond because it is not the
current owner of the block.
• The processor owning the requested block snoops the read request
on the bus, it asserts a special signal on the bus to prevent the
memory from responding.
• The owner then broadcasts a copy of the block on the bus, and
marks its copy as clean (unmodified).
• The data response on the bus is accepted by the cache of the
processor that issued the read request .

31. PARALLEL PROCESSING
3.Snoopy Caches
Read request
• The data response is also accepted by the memory to update its
copy of the block & reacquires ownership of the block, and the
block is marked as shared state because copies of it are in the
caches of two processors.
• Coherence is maintained because the two cached copies and the
copy of the block in the memory contain the same data.
• This scheme is called snoopy-cache techniques.

32. PARALLEL PROCESSING
4.Directory-Based Cache Coherence
• The concept of snoopy caches is easy to implement in single-bus
systems.
• Large shared memory multiprocessors use interconnection networks
such as rings and meshes.
• In such systems, broadcasting every single request to the caches of
all processors is inefficient.
• So uses directories in each memory module to indicate which nodes
may have copies of a given block in the shared state.
• Each request from a processor must be sent first to the memory
module & the directory information for that block is used to
determine the action that is taken.

33. PARALLEL PROCESSING
4.Directory-Based Cache Coherence
• A read request is forwarded to the current owner if the block is
modified.
• In write request ,for a block that is shared, individual invalidations
are sent only to nodes that may have copies of the block .
• The cost and complexity of the directory-based approach for
enforcing cache coherence limits its use to large systems.
• Small multiprocessors, including current multicore chips, typically
use snooping.

34. 34
MODULE 4 - PIPELINING
Session Topic
4.1 Pipelining: Basic concepts of pipelining
4.2 Pipelining - throughput and speedup
4.3 Pipeline Hazards – Data Hazards
4.4 Pipeline Hazards – Instructional- Structural Hazards
4.5 Parallel Processors: Introduction to parallel processors
4.6 Concurrent access to memory
4.7 Cache coherency
4.8 Dynamic Scheduling: Tomasulo’s Algorithm
4.9 Introduction to multicore architecture

35. PARALLEL PROCESSING
MULTI CORE
PROCESSOR
4.9 MULTICORE ARCHITECTURE

36. PARALLEL PROCESSING
Introduction: What is Processor?
• A processor is the logic circuitry that responds to and processes the
basic instructions that drive a computer. The term processor has
generally replaced the term central processing unit (CPU).
• The processor in a personal computer or embedded in small
devices is often called a microprocessor.

37. PARALLEL PROCESSING
What Is Core?
Actually, a CORE is the part of something that is central to its
existence or character.
Similarly, in computer system the CPU-ALU is referred as CORE.
Basically, there are two types of core processor:
1. Single Core Processor
2. Multi Core Processor

38. PARALLEL PROCESSING
Single Core
• It is a processor that has only one core, so it
can only start one operation at a time.
• It can however in some situations start a new
operation before the previous one is complete.
• Originally all processors were single core.
Examples are Intel Pentium 4 670, AMD
Athlon 64 FX-55.

39. PARALLEL PROCESSING
Multicore
• Also called multicore technology, it is a
type of architecture where a single
physical processor contains the core
logic of two or more cores (functional
units).
• These processors are packaged into a
single integrated circuit (IC). Examples
are Intel core i7, intel core 2 duo.

40. PARALLEL PROCESSING
Processor consists:
 A single chip package that fits in a socket
 Cores can have functional units, cache, etc.
associated with them,
 Cores can be fast or slow,
 Shared resources
 More cache
 Other integration: Memory controllers, high-speed serial links, etc.
 One system interface for System Bus and System Memory

41. PARALLEL PROCESSING
Problems with Single Core
• Toexecute the tasks faster you must increase the clock time.
• Increasing clock times too high drastically increases power consumption and
heat dissipation to extremely high levels, making the processor inefficient.
• E.g. Single core with 3GHz performance is similar to two core with 1.5GHz
• Power dissipation increases with increasing clock rate(frequency).

42. PARALLEL PROCESSING
Need for Multi-Core
For over 30 years the performance of processors has doubled every 2 years
Driven mainly by shrinkage of circuits
Smaller circuits
more transistors per chip
shorter connections
lower capacitance
Smaller circuits go faster
In 2000s the rate started to decrease

43. PARALLEL PROCESSING

44. PARALLEL PROCESSING
Multi Core solution
• Creating two cores or more on the same die increases processing power
while keeping clock speeds at an efficient level.
• A processor with 2 cores running at efficient clock speeds can process
instructions with similar speed to a single core processor running at twice
the clock speed, yet the dual core processor would still consume less
energy.

45. PARALLEL PROCESSING
• In the early 1970’s the first Microprocessor
was developed by Intel.
• It was a 4 bit machine that was named
the 4004.
• The 4004 was followed by Intel’s 8008 and 8080,
as well as motorola’s 6800 and 68000.
Origination of Multicore

46. PARALLEL PROCESSING
Single-core computer
4
6

47. Single-core CPU chip
the single core
4
7

48. PARALLEL PROCESSING
Multi-core architectures
• Replicate multiple processor cores on a single die.
 The cores fit on a single processor socket.

49. PARALLEL PROCESSING
Multi-core CPU chip
49
• The 4 cores fit on a single processor socket
• Also called CMP (Chip Multi-Processor)
C
o
r
e
1
c
O
r
e
2
c
o
r
e
3
c
o
r
e
4

50. PARALLEL PROCESSING
The cores run in parallel
50
c
o
r
e
1
c
o
r
e
2
c
o
r
e
3
c
o
r
e
4
thread 1 thread 2 thread 3 thread 4

51. PARALLEL PROCESSING
Within each core, threads are time-sliced (just like
on a uniprocessor)
51
c
o
r
e
1
c
o
r
e
2
c
o
r
e
3
c
o
r
e
4
several
threads
several
threads
several
threads
several
threads
The cores run in parallel (like on a uniprocessor)

52. PARALLEL PROCESSING
Multi-Core Advantages with thread
• While working with many threads, a Multi Core processor with n cores can
execute n threads simultaneously by assigning a core to each thread.
• If it must process more than n threads , say x, it can apply multithreading
procedures with each core working with an average of x/n threads.
• A Single core processor must multithread with every single thread.

53. PARALLEL PROCESSING
Multi-Core Computer
A multi-core processor is a processing system composed of two or more
independent cores (or CPUs). The cores are typically integrated onto a
single integrated circuit die (known as a chip multiprocessor or CMP), or they
may be integrated onto multiple dies in a single chip package.
A many-core processor is one in which the number of cores is large
enough that traditional multi-processor techniques are no longer efficient -
this threshold is somewhere in the range of several tens of cores - and likely
requires a network on chip.

54. PARALLEL PROCESSING
Multi-Core Computer- Dual core
• dual-core processor contains two independent
microprocessors.
• A dual core set-up is somewhat comparable to having
multiple, separate processors installed in the same computer,
but because the two processors are actually plugged into the
same socket, the connection between them is faster.
• Ideally, a dual core processor is nearly twice as powerful as
a single core processor.
• In practice, performance gains are said to be about fifty
percent: a dual core processor is likely to be about one-and-
a-half times as powerful as a single core processor.

55. PARALLEL PROCESSING
Multi-Core Computer
A multi-core processor implements multiprocessing in a single physical package.
Cores in a multi-core device may be coupled together tightly or loosely.
• For example, cores may or may not share caches, and they may implement
message passing or shared memory inter-core communication methods.
Common network topologies to interconnect cores include: bus, ring, 2-
dimentional mesh, and crossbar.
All cores are identical in symmetric multi-core systems and they are not identical
in asymmetric multi-core systems. Just as with single-processor systems, cores in
multi-core systems may implement architectures such as superscalar, vector
processing, or multithreading.

56. PARALLEL PROCESSING
Growth
With each new generation of processors there were several developments such as:
 Smaller size
 Faster
 Increased heat dissipation
 Greater Consumption of power

57. PARALLEL PROCESSING
Motivation

58. PARALLEL PROCESSING
What applications benefit from multi-core?
• Database servers
• Web servers
• Telecommunication markets
• Multimedia applications
• Scientific applications
• In general, applications with Thread-level parallelism (as
opposed to instruction-level parallelism)
Each can
run on its
own core

59. PARALLEL PROCESSING
More examples
59
• Editing a photo while recording a TV show through a digital video recorder
• Downloading software while running an anti-virus program
• “Anything that can be threaded today will map efficiently to multi-core”
• BUT: some applications difficult to parallelize

60. PARALLEL PROCESSING
Programming for multi-core
• Programmers must use threads or processes.
• Spread the workload across multiple cores.
• Write parallel algorithms.
• OS will map threads/processes to cores

61. PARALLEL PROCESSING
Interaction with OS
• Most major OS support multi-core today.
• OS perceives each core as a separate processor.
• OS scheduler maps threads/processes to different
cores.

62. PARALLEL PROCESSING
Why Multi-core?
• Better Performance
• For the Multi tasking
• e.g. Burning CD with graphic works at the same time
• Power consumption and Heat generation Caused from the
advance of CPU clock speed
• Save the room of motherboard
Two single cores → In one die
We can use this room more efficiently

63. PARALLEL PROCESSING
Why Multi-core?
• Simplicity
• We need additional systems to control the
several single cores.
• Economical efficiency
A dual-core is much cheaper than two single cores

64. PARALLEL PROCESSING
Shared-Memory Multicomputers

65. PARALLEL PROCESSING
Shared-Memory Multicomputers
• In a shared-memory multiprocessor, all processors have access to the
same memory.
• Tasks running in different processors can access shared variables in
the memory using the same addresses.
• The memory are distributed into multiple modules, so that simultaneous
requests from different processors are more likely to access different
memory modules.
• Three most common shared memory multiprocessors models are
– Uniform Memory Access (UMA)
– Non-uniform Memory Access (NUMA)
– Cache Only Memory Architecture (COMA)

66. PARALLEL PROCESSING
Uniform Memory Access (UMA)

67. PARALLEL PROCESSING
Uniform Memory Access (UMA)
• An interconnection network enables any processor to access any
module that is a part of he shared memory.
• When memory modules are kept physically separate from the
processors, all requests to access memory must pass through the
network, which introduces latency.
• A system which has the same network latency for all accesses from
the processors to the memory modules is called a Uniform Memory
Access (UMA) multiprocessor.
• Although the latency is uniform, it may be large for a network that
connects many processors and memory modules.

68. PARALLEL PROCESSING
Uniform Memory Access (UMA)
• Each processor may have a private cache memory.
• Same rule is followed for peripheral devices
• When all the processors have equal access to all the peripheral
devices, the system is called a symmetric multiprocessor.
• When only one or a few processors can access the peripheral
devices, the system is called an asymmetric multiprocessor.

69. PARALLEL PROCESSING
Non-uniform Memory Access (NUMA)

70. PARALLEL PROCESSING
Non-uniform Memory Access (NUMA)
• For better performance(to reduce the latency), place a memory module
close to each processor.
• The result is a collection of nodes, each consisting of a processor and a
memory module. The nodes are then connected to the network.
• The network latency is avoided when a processor makes a request to
access its local memory.
• However, a request to access a remote memory module must pass
through the network.
• Because of the difference in latencies for accessing local and remote
portions of the shared memory, systems of this type are called Non-
Uniform Memory Access (NUMA) multiprocessors.

71. PARALLEL PROCESSING
Cache Only Memory Architecture (COMA)

72. PARALLEL PROCESSING
Cache Only Memory Architecture (COMA)
• The COMA model is a special case of the NUMA model.
• Here, all the distributed main memories are converted to cache
memories.
• The local memories (typically DRAM) at each node are used as cache.
• In NUMA, When processors access some data, a copy is made in their
local cache, but space remains allocated in the home node.
• Instead, with COMA, there is no home. An access from a remote node
may cause that data to migrate.
• Compared to NUMA, this reduces the number of redundant copies and
may allow more efficient use of the memory resources.

73. PARALLEL PROCESSING
Cache Coherence

74. PARALLEL PROCESSING
Cache Coherence
• Each variable in a program has a unique address location in the memory,
which can be accessed by any processor.
• Each processor has its own cache. When any processor writes to a
shared variable in its own cache, all other caches that contain a copy of
that variable will then have the old, incorrect value.
• So, They must be informed of the change so that they can either update
their copy to the new value or invalidate it.
• This is the issue of maintaining cache coherence, which requires having
a consistent view of shared data in multiple caches.
• The approach, Similar to write-through approach or write-back approach
can be used to address cache coherence in a multiprocessor system.

75. PARALLEL PROCESSING
1.Write-Through Protocol
• A write-through protocol can be implemented in one of two ways.
– updating the values in other caches,
– invalidating the copies in other caches.

76. PARALLEL PROCESSING
1.Write-Through Protocol
a) Update protocol
• When a processor writes a new value to a block of data in its cache, the
new value is also written into the memory module containing the block
being modified.
• Since copies of this block may exist in other caches, these copies must
be updated to reflect the change caused by the Write operation.
• The simplest way of doing this is to broadcast the written data to the
caches of all processors in the system.
• As each processor receives the broadcast data, it updates the contents
of the affected cache block if this block is present in its cache.

77. PARALLEL PROCESSING
1.Write-Through Protocol
b) Invalidation of copies.
• When a processor writes a new value into its cache, this value is
also sent to the appropriate location in memory, and all copies in
other caches are invalidated.
• Again, broadcasting can be used to send the invalidation requests
throughout the system.

78. PARALLEL PROCESSING
2.Write-Back protocol
• This method is based on the concept of ownership of a block of data.
• Initially, the memory is the owner of all blocks, and the memory
retains ownership of any block that is read by a processor to place a
copy in its cache.
Getting ownership
• If any processor wants to write to a block in its cache, it must first
become the exclusive owner of this block.
• To do so, all copies in other caches must first be invalidated with a
broadcast request.
• The new owner of the block may then modify the contents.

79. PARALLEL PROCESSING
2.Write-Back protocol
Changing ownership
• When another processor wishes to read a modified block , the
request must be forwarded to the current owner.
• The data are then sent to the requesting processor. and the
appropriate memory module by the current owner.
• Memory module reacquires ownership and updates the contents of
the block in the memory.
• The block is now shared with two caches and the memory.
• Subsequent requests from other processors to read the same block
are serviced by the memory module containing the block.

80. PARALLEL PROCESSING
2.Write-Back protocol
• When another processor wishes to write to a block that has been
modified, the current owner (memory module) sends the data to the
requesting processor. It also transfers ownership of the block to the
requesting processor and invalidates its cached copy.
Advantages
• creating less traffic than the write-through protocol. (may perform
several writes to a cache block before this block is needed by
another processor)

81. PARALLEL PROCESSING
3.Snoopy Caches
• In a single-bus system, all transactions between processors and memory
modules occur via requests and responses on the bus. So,
it broadcast to all units connected to the bus.
• Each processor cache has a controller circuit that observes, or snoops, all
transactions on the bus.
Write into block
• Consider a processor has a copy of a block and before writing to this block for
the first time, the processor get the ownership and broadcast an invalidation
request to all other caches.
• The controllers of other processor cache ,accept the request and invalidate any
copies of the same block.
• The processor may then write to the block and mark it as being modified

82. PARALLEL PROCESSING
3.Snoopy Caches
Read request
• Now, if another processor broadcasts a read request on the bus for
the same block, the memory must not respond because it is not the
current owner of the block.
• The processor owning the requested block snoops the read request
on the bus, it asserts a special signal on the bus to prevent the
memory from responding.
• The owner then broadcasts a copy of the block on the bus, and
marks its copy as clean (unmodified).
• The data response on the bus is accepted by the cache of the
processor that issued the read request .

83. PARALLEL PROCESSING
3.Snoopy Caches
Read request
• The data response is also accepted by the memory to update its
copy of the block & reacquires ownership of the block, and the
block is marked as shared state because copies of it are in the
caches of two processors.
• Coherence is maintained because the two cached copies and the
copy of the block in the memory contain the same data.
• This scheme is called snoopy-cache techniques.

84. PARALLEL PROCESSING
4.Directory-Based Cache Coherence
• The concept of snoopy caches is easy to implement in single-bus
systems.
• Large shared memory multiprocessors use interconnection networks
such as rings and meshes.
• In such systems, broadcasting every single request to the caches of
all processors is inefficient.
• So uses directories in each memory module to indicate which nodes
may have copies of a given block in the shared state.
• Each request from a processor must be sent first to the memory
module & the directory information for that block is used to
determine the action that is taken.

85. PARALLEL PROCESSING
4.Directory-Based Cache Coherence
• A read request is forwarded to the current owner if the block is
modified.
• In write request ,for a block that is shared, individual invalidations
are sent only to nodes that may have copies of the block .
• The cost and complexity of the directory-based approach for
enforcing cache coherence limits its use to large systems.
• Small multiprocessors, including current multicore chips, typically
use snooping.

86. MODULE 5 – MEMORY ORGANIZATION
Session Topic
5.1 Semiconductor Memory Technologies, Memory Organization
5.2 Concept of Hierarchical Memory Organization
5.3 Cache Memory, cache size vs. block size
5.4 Mapping Functions
5.5 Replacement Algorithms
5.6 Memory Interleaving
5.7 Write Policies

87. PARALLEL PROCESSING
THE MEMORY SYSTEM
5.1.1 Fundamental Concepts
5.1.2 Semiconductor RAM Memories
5.1.3 Read Only Memories (ROM)
17CS308 - COMPUTER ARCHITECTURE 87

88. PARALLEL PROCESSING
88
• Memory is a collection of cells, in which each cell is capable of
storing a bit of information.
• The good memory - fast, large, and inexpensive.
• Increased speed and size are achieved at increased cost
– Speed depends on data transfer rate between process and
memory
– Maximum size of the memory in any computer is determined by
the addressing scheme.
5.1.1 Fundamental Concepts

89. PARALLEL PROCESSING
SOME BASIC CONCEPTS
89
17CS308 - COMPUTER ARCHITECTURE
• Maximum size of the Main Memory = 2k locations
• K is number of addressing bits
• 216 = 64K (kilo), 64K memory location is used
• 232 = 4G (giga) , 4G memory location is used
• 264 = 16E (exa) ≈ 16 × 1018 memory location is used.
• Two types of byte address are
 big endian : used in Motorola
 little endian: used in Intel
• In 32 bit addressing,
High order 30 bits are used to determines which word will be
accessed.
Low order 2 bits are used to determines which byte in the is
1 word
1B 1B 1B 1B

90. PARALLEL PROCESSING
SOME BASIC CONCEPTS
90
17CS308 - COMPUTER ARCHITECTURE
CPU-Main Memory Connection

91. PARALLEL PROCESSING
 Measures for the speed of a memory:
 memory access time – time b/w initiation, completion of memory
operation
 memory cycle time – time delay b/w the initiation two successive
memory access.
 RAM - primary memory- access time is same.
 An important design issue is to provide a computer system with as
large and fast a memory as possible, within a given cost target.
 Several techniques to increase the effective size and speed of the
memory:
 Cache memory (to increase the effective speed).
 Virtual memory (to increase the effective size).
SOME BASIC CONCEPTS(CONTD.,)
91
17CS308 - COMPUTER ARCHITECTURE

92. PARALLEL PROCESSING
92
17CS308 - COMPUTER ARCHITECTURE
 Memory access time : Time that elapses between the initiation of an
operation to transfer a word of data and the completion of that operation
 is how long it takes for a character in memory to be transferred to or
from the CPU.
 Memory Cycle time : minimum time delay required between the
initiation of two successive memory operations
 usually measured in nanosecond s, between the start of one random
access memory ( RAM ) access to the time when the next access
can be started.

93. PARALLEL PROCESSING
POINTS TO PONDER
 Memory size
 Memory access time
 Memory cycle time
 RAM
 Cache memory
 Virtual memory
93

94. PARALLEL PROCESSING
94
17CS308 - COMPUTER ARCHITECTURE
THE MEMORY SYSTEM
5.1.2 Semiconductor RAM memories

95. PARALLEL PROCESSING
 RAM : Semiconductor memory is an electronic data storage
device, often used as computer memory, implemented on a
semiconductor-based integrated circuit (IC).
 Most modern semiconductor memory devices are implemented
allowing random access, which means that it takes the same
amount of time to access any memory location, so data can be
efficiently accessed in any random order.
95
17CS308 - COMPUTER ARCHITECTURE

96. PARALLEL PROCESSING
• RAM differs from other data storage media such as hard disks and
CDs
• Semiconductor memory also has much faster access times than
other types of data storage;
– a byte of data can be written to or read from semiconductor
memory within a few nanoseconds, while access time for
rotating storage such as hard disks is in the range of
milliseconds.
• For these reasons it is used for main computer memory, to hold data
the computer is currently working on, among other uses.
96

97. PARALLEL PROCESSING
97
17CS308 - COMPUTER ARCHITECTURE
INTERNAL ORGANIZATION OF MEMORY CHIPS
 Each memory cell can hold one bit of information.
 Memory cells are organized in the form of an array.
 One row is one memory word.
 All cells of a row are connected to a common line, known as the
“word line”.
 Word line is connected to the address decoder.
 The cells in each column are connected to a Sense/Write circuit
by two bit lines,
 Sense/write circuits are connected to the data input/output lines of
the memory chip.

98. PARALLEL PROCESSING
98
17CS308 - COMPUTER ARCHITECTURE
During read operation, these circuit sense (read) the information
stored in the cells selected by a word line and transmit to the o/p
date lines.
During a write operation, the sense/write input information and
store it is the cells of the selected word .

99. PARALLEL PROCESSING
17CS308 - COMPUTER ARCHITECTURE 99
Internal organization of memory chips (Contd.,)

100. PARALLEL PROCESSING
External connection
• 128 bits and requires 14 external connections for address-4, data-8,
and control lines-2. It also needs two lines for power supply and ground
connections.
• For 1K (1024) memory cells (organized as a 128 × 8 memory, 12827 ,
7 address lines), it requires a total of 19 external connections.
• Alternatively, the same number of cells can be organized into a 1K×1
format, which needs 10-bit address is needed, but there is only one data
line, resulting in 15 external connection.
• The 10-bit address is divided into row and column addresses of each
having 5 bits.
100

101. PARALLEL PROCESSING
101
17CS308 - COMPUTER ARCHITECTURE
A MEMORY CHIP

102. PARALLEL PROCESSING
102
Two types of RAMs
• Static RAM (SRAM)
• Dynamic RAM (DRAM)
– Asynchronous DRAM (ADRAM)
– Synchronous DRAM(SDRAM)
• Double Data Rate SDRAM

103. PARALLEL PROCESSING
103
17CS308 - COMPUTER ARCHITECTURE
Static RAM Cell (SRAM)
 Memories that retaining their state as long as power is applied are
known as static memories
 Two inverters are cross connected to form a latch (flip-flop).
 The latch is connected to one word line and two bits lines by
transistors T1 and T2
 When word line is at ground level, the transistors are turned off and
the latch retains its state
 Read operation: In order to read state of SRAM cell, the word line is
activated to close switches T1 and T2. Sense/Write circuits at the
bottom monitor the state of b and b’

104. PARALLEL PROCESSING
104
17CS308 - COMPUTER ARCHITECTURE
Static RAM Cell
 Write operation : Sense/Write circuit places the appropriate value
on bit line b and its complement on b’ and activates the word line.
This forces the cell into the corresponding state, which the cell retains
when the word line is deactivated

105. PARALLEL PROCESSING
105
17CS308 - COMPUTER ARCHITECTURE
SRAM – recap
 Consist of circuits that are capable of retaining their state as long
as the power is applied.
 Volatile memories, because their contents are lost when power is
interrupted.
 Access times of static RAMs are in the range of few nanoseconds.
 However, the cost is usually high.

106. PARALLEL PROCESSING
106
17CS308 - COMPUTER ARCHITECTURE
Dynamic RAMs (DRAMs):
 Static RAMs are fast, but their cells require several transistors.
 Less expensive and higher density RAMs can be implemented with
simpler cells.
 But, these simpler cells do not retain their state for a long period,
unless they are accessed frequently for Read or Write operations.
 Memories that use such cells are called dynamic RAMs (DRAMs).

107. PARALLEL PROCESSING
• Information is stored in a dynamic memory cell in the form of a
charge on a capacitor.
• This charge can be maintained for only tens of milliseconds.
• Since the cell is required to store information for a much longer time,
its contents must be periodically refreshed by restoring the capacitor
charge to its full value.
107

108. PARALLEL PROCESSING
108
17CS308 - COMPUTER ARCHITECTURE
ASYNCHRONOUS DRAMS
 256 Mb DRAM configured as 32M X 8 .
 The cells are organized in the form of a 16K × 16K array.
 The 16,384 cells in each row are divided into 2,048 groups of 8, forming
2,048 bytes of data.
 In total, a 25-bit address is needed to access a byte in this memory.
 The high-order 14 bits and the low-order 11 bits of the address constitute
the row and column addresses of a byte, respectively.
Read or Write operation :
 The row address is applied first. It is loaded into the row address latch in
response to a Row Address Strobe (RAS)signal .
 All cells in the selected row are read and refreshed.

109. PARALLEL PROCESSING
109
256 Megabit DRAM

110. PARALLEL PROCESSING
• The column address is loaded into the column address latch under
control of ColumnAddress Strobe (CAS).
• The information in this latch is decoded and the appropriate group of
8 Sense/Write circuits is selected.
• For Read operation, the output values of the selected circuits are
transferred to the data lines, D7−0.
• For a Write operation, the information on theD7−0 lines is
transferred to the selected circuits, then used to overwrite the
contents of the selected cells in the corresponding 8 columns
110

111. PARALLEL PROCESSING
111
17CS308 - COMPUTER ARCHITECTURE
FAST PAGE MODE
 If we want to access the consecutive bytes in the selected row, following
can be done without having to reselect the row.
 Add a latch at the output of the sense circuits in each row.
 All the latches are loaded when the row is selected.
 Different column addresses can be applied to select and place
different bytes on the data lines.
 Consecutive sequence of column addresses can be applied under the
control signal CAS, without reselecting the row.
 Allows a block of data to be transferred at a much faster rate than
random accesses.
 A small collection/group of bytes is usually referred to as a block.
 This transfer capability is referred to as the fast page mode feature.

112. PARALLEL PROCESSING
112
17CS308 - COMPUTER ARCHITECTURE
SYNCHRONOUS DRAMS (SDRAMs)
• SDRAMs Operation is directly synchronized with processor clock signal.

113. PARALLEL PROCESSING
113
17CS308 - COMPUTER ARCHITECTURE
• The outputs of the sense circuits are connected to a latch.
• During a Read operation, the contents of the cells in a row are
loaded onto the latches.
• During a refresh operation, the contents of the cells are refreshed
without changing the contents of the latches.
• Data held in the latches correspond to the selected columns are
transferred to the output.
• For a burst mode of operation, successive columns are selected
using column address counter and clock.
• CAS signal need not be generated externally. A new data is placed
during raising edge of the clock

114. PARALLEL PROCESSING
114
• Timing diagram for a typical burst read of length 4.
• First, the row address is latched under control of the RAS signal. The memory
typically takes 5 or 6 clock cycles to activate the selected row.
• Then, the column address is latched under control of the CAS signal.
• After a delay of one clock cycle, the first set of data bits is placed on the data lines
& next 3 set of bits are placed onthe data lines in the next 3 clock cycles.

115. PARALLEL PROCESSING
LATENCY, BANDWIDTH & DDRSDRAM
115
17CS308 - COMPUTER ARCHITECTURE
 Memory latency is the time it takes to transfer a first word of block to
or from memory
 Memory bandwidth is the number of bits or bytes that can be
transferred in one second.
 DDRSDRAMs (Double-Data-Rate SDRAM)
 Faster version of SDRAMs
 To make the best use of the available clock speed, data are
transferred externally on both the rising and falling edges of the
clock.

116. PARALLEL PROCESSING
Structure of Larger Memories
1.Static Memory Systems
116

117. PARALLEL PROCESSING
STATIC MEMORIES
117
17CS308 - COMPUTER ARCHITECTURE
• Implement a memory unit of 2M words of 32 bits each.
• Use 512x8 static memory chips.
• Each column consists of 4 chips. Each chip implements one byte
position.
• A chip is selected by setting its chip select control line to 1.
• Selected chip places its data on the data output line, outputs of other
chips are in high impedance state.
• 21 bits are used to address a 32-bit word.
• High order 2 bits are needed to select the row, by activating
the four Chip Select signals.
• Lower order 19 bits are used to access specific byte locations
inside the selected chip.

118. PARALLEL PROCESSING
DYNAMIC MEMORIES
 Large dynamic memory systems can be implemented using DRAM chips in a
similar way to static memory systems.
 Placing large memory systems directly on the motherboard will occupy a large
amount of space.
 Also, this arrangement is inflexible since the memory system cannot be
expanded easily.
 Packaging considerations have led to the development of larger memory units
known as SIMMs (Single In-line Memory Modules) and DIMMs (Dual In-line
Memory Modules).
 Memory modules are an assembly of memory chips on a small board that plugs
vertically onto a single socket on the motherboard.
 Occupy less space on the motherboard.
 Allows for easy expansion by replacement. 118
17CS308 - COMPUTER ARCHITECTURE

119. PARALLEL PROCESSING
 Recall that in a dynamic memory chip, to reduce the number of pins,
multiplexed addresses are used.
 Address is divided into two parts:
 High-order address bits select a row in the array.
 They are provided first, and latched using RAS signal.
 Low-order address bits select a column in the row.
 They are provided later, and latched using CAS signal.
 However, a processor issues all address bits at the same time.
 In order to achieve the multiplexing in memory
 controller circuit is inserted between the processor and memory.
MEMORY CONTROLLER
119
17CS308 - COMPUTER ARCHITECTURE

120. PARALLEL PROCESSING
MEMORY CONTROLLER (CONTD..)
120
17CS308 - COMPUTER ARCHITECTURE

121. PARALLEL PROCESSING
121
17CS308 - COMPUTER ARCHITECTURE
THE MEMORY SYSTEM
5.1.3 Read Only Memories (ROM)

122. PARALLEL PROCESSING
122
 SRAM and SDRAM chips are volatile:
 Lose the contents when the power is turned off.
 Many applications need memory devices to retain contents after the power is
turned off.
 For example, computer is turned on, the operating system must be loaded
from the disk into the memory.
 Store instructions which would load the OS from the disk.
 Need to store these instructions so that they will not be lost after the power is
turned off.
 We need to store the instructions into a non-volatile memory.
 Non-volatile memory is read in the same manner as volatile memory.
 Separate writing process is needed to place information in this memory.
 Normal operation involves only reading of data, this type
of memory is called Read-Only memory (ROM).

123. PARALLEL PROCESSING
123
Types of ROM
 ROM
 PROM
 EPROM
 EEPROM
 Flash Memory

124. PARALLEL PROCESSING
124
 Read-Only Memory:
 Data are written into a ROM when it is manufactured.
 Programmable Read-Only Memory (PROM):
 Allow the data to be loaded by a user.
 Process of inserting the data is irreversible.
 Storing information specific to a user in a ROM is expensive.
 Providing programming capability to a user may be better.

125. PARALLEL PROCESSING
125
 Erasable Programmable Read-Only Memory (EPROM):
 Stored data to be erased and new data to be loaded.
 Flexibility, useful during the development phase of digital systems.
 Erasable, reprogrammable ROM.
 Erasure requires exposing the ROM to UV light.
 Electrically Erasable Programmable Read-Only
Memory (EEPROM):
 To erase the contents of EPROMs, they have to be exposed to
ultraviolet light.
 Physically removed from the circuit.
 EEPROMs the contents can be stored and erased electrically.
 Need different voltages for reading, writing, erasing etc.,

126. PARALLEL PROCESSING
126
 Flash memory:
 Has similar approach to EEPROM.
 Read the contents of a single cell, but write the contents of an
entire block of cells.
 Flash devices have greater density.
▪ Higher capacity and low storage cost per bit.
 Power consumption of flash memory is very low, making it
attractive for use in equipment that is battery-driven.
 Single flash chips are not sufficiently large, so larger memory
modules are implemented using flash cards and flash drives.

127. PARALLEL PROCESSING
 Elaborate on the various memory technologies
 Draw different memory address layouts & brief the technique
used to increase the average rate of fetching words from the
main memory.
 The amount of ROM needed to implement a 4 bit multiplier is
(A) 64 bits
(B) 128 bits
(C) 1 Kbits
(D) 2 Kbits
 How many RAM chips of size (256K x 1 bit) are required to
build 1M Byte memory ?
A) 8
B) 12
C) 24
(D) 32 Kbits
127
17CS308 - COMPUTER ARCHITECTURE
TRY YOURSELF

128. MODULE 5 – MEMORY ORGANIZATION
Session Topic
5.1 Semiconductor Memory Technologies, Memory Organization
5.2 Concept of Hierarchical Memory Organization
5.3 Cache Memory, cache size vs. block size
5.4 Mapping Functions
5.5 Replacement Algorithms
5.6 Memory Interleaving
5.7 Write Policies

129. Performance Considerations

130. PARALLEL PROCESSING
Performance considerations
• Performance of a processor depends on:
– How fast machine instructions can be brought into the
processor for execution.
– How fast the instructions can be executed.

131. PARALLEL PROCESSING
Interleaving
• The memory is structured as a collection of physically separate
modules
– Each module has its own address buffer register (ABR) and data
buffer register (DBR).
• There are 2 cases in structuring these modules;

132. PARALLEL PROCESSING
Case 1 :Consecutive words are placed in a
module.
• Consecutive words are placed in a module.
• High-order k bits of a memory address determine the module.
• Low-order m bits of a memory address determine the word within a
module.
• When a block of words is transferred from main memory to cache,
only one module is busy at a time.

133. PARALLEL PROCESSING
Case 1 :Consecutive words are placed in a
module.

134. PARALLEL PROCESSING
Case 2 :Consecutive words are located in
consecutive modules.
 Consecutive words are located in consecutive modules.
 High-order m bits of a memory address determine the word within a
module.
 Low-order k bits of a memory address determine the module.
 Consecutive addresses can be located in consecutive modules.
 While transferring a block of data, several memory modules can be
kept busy at the same time.
 This leads to faster access. This technique is called memory
interleaving.

135. PARALLEL PROCESSING
Case 2 :Consecutive words are located in
consecutive modules.

136. PARALLEL PROCESSING
Hit Rate and Miss Penalty
• Hit rate :Number of hits stated as a fraction of all attempted
accesses.
• Miss rate : Number of misses stated as a fraction of attempted
accesses
• Miss penalty :The extra time needed to bring the desired
information into the cache.
• Hit rate can be improved by increasing block size, while keeping
cache size constant
• Block sizes that are neither very small nor very large give best
results.
• Miss penalty can be reduced if load-through approach is used when
loading new blocks into cache.

137. PARALLEL PROCESSING
Example:
Let
• h be the hit rate
• m be the Penalty
• c be the time to access the information in the cache.
• The average access time is given by,
tavg=hc+(1-h)m.

138. PARALLEL PROCESSING
Caches on the processor chip
• In high performance processors, 2 levels are normally used.
• L1 cache is fabricated on the processor chip and L2 cache is
fabricated externally.
• L1 fast and small. L2 is slow and big .
• Avg access time in a system with 2 levels of caches is
– Tave = h1c1+(1-h1)h2c2+(1-h1)(1-h2)M

139. PARALLEL PROCESSING
Other Performance Enhancements
1.Write buffer
• Write-through:
– Each write operation involves writing to the main memory.
– If the processor has to wait for the write operation to be
complete, it slows down the processor.
– Processor does not depend on the results of the write operation.
– Write buffer can be included for temporary storage of write
requests.
– Processor places each write request into the buffer and
continues execution.

140. PARALLEL PROCESSING
Other Performance Enhancements
• Write-back:
– Block is written back to the main memory when it is replaced.
– If the processor waits for this write to complete, before reading
the new block, it is slowed down.
– Fast write buffer can hold the block to be written, and the new
– block can be read first.

141. PARALLEL PROCESSING
Other Performance Enhancements (Contd.,)
2.Prefetching
• New data are brought into the processor when they are first needed.
• Processor has to wait before the data transfer is complete.
• Prefetch the data into the cache before they are actually needed, or
a before a Read miss occurs.
• Prefetching can be accomplished through software by including a
special instruction in the machine language of the processor.
– Inclusion of prefetch instructions increases the length of the
programs.

142. PARALLEL PROCESSING
Other Performance Enhancements (Contd.,)
2.Prefetching
• Prefetching can also be accomplished using hardware:
– Circuitry that attempts to discover patterns in memory references
and then prefetches according to this pattern.

143. PARALLEL PROCESSING
Other Performance Enhancements (Contd.,)
3.Lockup-Free Cache
• Cache structure which supports multiple outstanding misses is
called a lockup free cache.
• Since only one miss can be serviced at a time, a lockup free cache
must include circuits that keep track of all the outstanding misses.
• Special registers may hold the necessary information about these
misses.

144. MODULE 5 – MEMORY ORGANIZATION
Session Topic
5.1 Semiconductor Memory Technologies, Memory Organization
5.2 Concept of Hierarchical Memory Organization
5.3 Cache Memory, cache size vs. block size
5.4 Mapping Functions
5.5 Replacement Algorithms
5.6 Memory Interleaving
5.7 Write Policies

145. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Speed, Size, and Cost
145
 A big challenge in the design of a computer system is to provide a
sufficiently large memory, with a reasonable speed at an affordable
cost.
 Static RAM:
 Very fast, but expensive, because a basic SRAM cell has a complex
circuit making it impossible to pack a large number of cells onto a
single chip.
 Dynamic RAM:
 Simpler basic cell circuit, hence are much less expensive, but
significantly slower than SRAMs.

146. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Speed, Size, and Cost
146
 Magnetic disks:
 Storage provided by DRAMs is higher than SRAMs, but is still less
than what is necessary.
 Secondary storage such as magnetic disks provide a large amount
of storage, but is much slower than DRAMs.

147. 5.2 HIERARCHICAL MEMORY ORGANIZATION
147

148. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Memory Hierarchy Design
148

149. 5.2 HIERARCHICAL MEMORY ORGANIZATION
149

150. 5.2 HIERARCHICAL MEMORY ORGANIZATION
150
• Fastest access is to the data held in processor registers. Registers are
at the top of the memory hierarchy.
• Relatively small amount of memory that can be implemented on the
processor chip. This is processor cache.
• Two levels of cache. Level 1 (L1) cache is on the processor chip. Level
2 (L2) cache is in between main memory and processor.
• Next level is main memory, implemented as SIMMs. Much larger, but
much slower than cache memory.
• Next level is magnetic disks. Huge amount of inexepensive storage.
• Speed of memory access is critical, the idea is to bring instructions and
data that will be used in the near future as close to the processor as
possible.

151. 5.2 HIERARCHICAL MEMORY ORGANIZATION
MODULE 5 – MEMORY ORGANIZATION
Session Topic
5.1 Semiconductor Memory Technologies, Memory Organization
5.2 Concept of Hierarchical Memory Organization
5.3 Cache Memory, cache size vs. block size
5.4 Mapping Functions
5.5 Replacement Algorithms
5.6 Memory Interleaving
5.7 Write Policies

152. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache Memories
• Processor is much faster than the main memory.
– As a result, the processor has to spend much of its time waiting
while instructions and data are being fetched from the main
memory.
– Major obstacle towards achieving good performance.
• Speed of the main memory cannot be increased beyond a certain
point.
• Cache memory is an architectural arrangement which makes the main
memory appear faster to the processor than it really is.
• Effectiveness of Cache memory is based on the property of computer
programs known as “locality of reference”.

153. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Locality of Reference
• Analysis of programs indicates that many instructions in a program are
executed repeatedly during some period of time, while the others are
accessed relatively less frequently.
– These instructions may be the ones in a loop, nested loop or few
procedures calling each other repeatedly.
– This is called “locality of reference”. – two types Temporal and
Spatial
• Temporal locality of reference:
– Recently executed instruction is likely to be executed again very
soon.
• Spatial locality of reference:

154. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache memories
Cache
Main
memory
Processor

155. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache memories
• Processor issues a Read request, a block of words is transferred from
the main memory to the cache, one word at a time.
• Subsequent references to the data in this block of words are found in
the cache.
• At any given time, only some blocks in the main memory are held in
the cache. Which blocks in the main memory are in the cache is
determined by a “mapping function”.
• When the cache is full, and a block of words needs to be transferred
from the main memory, some block of words in the cache must be
replaced. This is determined by a “replacement algorithm”.

156. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache hit
• Existence of a cache is transparent to the processor. The processor
issues Read and Write requests in the same manner.
• If the data is in the cache it is called a Read or Write hit.
• Read hit:
– The data is obtained from the cache.

157. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache hit
• Write hit:
– Cache has a replica of the contents of the main memory.
– Contents of the cache and the main memory may be updated
simultaneously. This is the write-through protocol.
– Update the contents of the cache, and mark it as updated by
setting a bit known as the dirty bit or modified bit. The contents of
the main memory are updated when this block is replaced. This is
write-back or copy-back protocol.

158. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache miss
• If the data is not present in the cache, then a Read miss or Write
miss occurs.
• Read miss:
– Block of words containing this requested word is transferred from
the memory. After the block is transferred, the desired word is
forwarded to the processor.
– The desired word may also be forwarded to the processor as soon
as it is transferred without waiting for the entire block to be
transferred. This is called load-through or early-restart.

159. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache miss
• Write-miss:
– If Write-through protocol is used, then the contents of the main
memory are updated directly.
– If write-back protocol is used, the block containing the addressed
word is first brought into the cache.
– Then the desired word is overwritten with new information.

160. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache Coherence Problem
• A bit called as “valid bit” is provided for each block.
• If the block contains valid data, then the bit is set to 1, else it is 0.
• Valid bits are set to 0, when the power is just turned on.
• When a block is loaded into the cache for the first time, the valid bit is set
to 1.
• Data transfers between main memory and disk occur directly bypassing
the cache.
• When the data on a disk changes, the main memory block is also
updated.
• However, if the data is also resident in the cache, then the valid bit is set
to 0.

161. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Cache Coherence Problem
• What happens if the data in the disk and main memory changes and the
write-back protocol is being used?
• In this case, the data in the cache may also have changed and is
indicated by the dirty bit.
• The copies of the data in the cache, and the main memory are different.
This is called the cache coherence problem.
• One option is to force a write-back before the main memory is updated
from the disk.

162. 5.2 HIERARCHICAL MEMORY ORGANIZATION
MODULE 5 – MEMORY ORGANIZATION
Session Topic
5.1 Semiconductor Memory Technologies, Memory Organization
5.2 Concept of Hierarchical Memory Organization
5.3 Cache Memory, cache size vs. block size
5.4 Mapping Functions
5.5 Replacement Algorithms
5.6 Memory Interleaving
5.7 Write Policies

163. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Mapping functions
• Mapping functions determine how memory blocks are placed in the
cache.
• A simple processor example:
– Cache consisting of 128 blocks of 16 words each.
– Total size of cache is 2048 (2K) words.
– Main memory is addressable by a 16-bit address.
– Main memory has 64K words.
– Main memory has 4K blocks of 16 words each.

164. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Mapping functions
• Three mapping functions:
– Direct mapping
– Associative mapping
– Set-associative mapping.

165. 5.2 HIERARCHICAL MEMORY ORGANIZATION
1. DIRECT MAPPING
165
17CS308 - COMPUTER ARCHITECTURE
• Simplest mapping technique - each block of main memory maps to only
one cache line
– i.e. if a block is in cache, it must be in one specific place
• Formula to map a memory block to a cache line:
– i = j mod c
• i=Cache Line Number
• j=Main Memory Block Number
• c=Number of Lines in Cache
– i.e. we divide the memory block by the number of cache lines and
the remainder is the cache line address

166. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Direct mapping
Main
memory Block 0
Block 1
Block 127
Block 128
Block 129
Block 255
Block 256
Block 257
Block 4095
7 4
Main memory address
T ag Block W ord
5
tag
tag
tag
Cache
Block 0
Block 1
Block 127

167. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Direct mapping
• Block j of the main memory maps to j modulo 128 of the cache.
• 0,128,256… maps to 0, and 1,129,257,… maps to 1 and
255255
• 32 block of MM is mapped to one block of cache.
• More than one memory block is mapped onto the same position in the
cache.
• May lead to contention for cache blocks even if the cache is not full.
• Resolve the contention by allowing new block to replace the old block,
leading to a trivial replacement algorithm.

168. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Direct mapping
• Main memory address is divided
into three fields:
- Low order 4 bits determine one of the 16 words in a block.
- When a new block is brought into the cache, the next 7 bits determine
which cache block this new block is placed in. (128 line=27)
- High order 5 bits determine which of the possible 32 blocks is
currently present in the cache. These are tag bits.
• Simple to implement but not very flexible.

169. 5.2 HIERARCHICAL MEMORY ORGANIZATION
169
17CS308 - COMPUTER ARCHITECTURE
DIRECT MAPPING WITH C=4
• Shrinking our example to a cache line size of 4 slots (each
slot/line/block still contains 4 words):
– Cache Line Memory Block Held
• 0 0, 4, 8, …
• 1 1, 5, 9, …
• 2 2, 6, 10, …
• 3 3, 7, 11, …
– In general:
• 0 0, C, 2C, 3C, …
• 1 1, C+1, 2C+1, 3C+1, …
• 2 2, C+2, 2C+2, 3C+2, …
• 3 3, C+3, 2C+3, 3C+3, …

170. 5.2 HIERARCHICAL MEMORY ORGANIZATION
170
17CS308 - COMPUTER ARCHITECTURE
DIRECT MAPPING WITH C=4
Block 0
Block 1
Block 2
Block 3
Block 4
Block 5
Block 6
Block 7
Main
Memory
Slot 0
Slot 1
Slot 2
Slot 3
Cache Memory
Each slot contains K words
Tag: Identifies which memory block is in the slot
Valid: Set after block copied from memory to
indicate the cache line has valid data
TAG

171. 5.2 HIERARCHICAL MEMORY ORGANIZATION
A computer system uses 16-bit memory addresses. It has a 2K-byte cache
organized in a direct-mapped manner with 64 bytes per cache block. Assume that
the size of each memory word is 1 byte.
(a) Calculate the number of bits in each of the Tag, Block, and Word fields of the
memory address.
(a) Block size = 64 bytes = 26 bytes
Therefore, Number of bits in the Word field = 6
Cache size = 2K-byte = 211 bytes
Number of cache blocks = Cache size / Block size = 211 / 26 = 25
Therefore, Number of bits in the Block field = 5
Total number of address bits = 16
Therefore, Number of bits in the Tag field = 16 - 6 - 5 = 5
For a given 16-bit address,
the 5 most significant bits, represent the Tag,
the next 5 bits represent the Block,
and the 6 least significant bits represent the Word.

172. 5.2 HIERARCHICAL MEMORY ORGANIZATION
DIRECT MAPPING PROS & CONS
• Simple
• Inexpensive
• Fixed location for given block
– If a program accesses 2 blocks that map to the same line
repeatedly, cache misses are very high – condition called
thrashing

173. 5.2 HIERARCHICAL MEMORY ORGANIZATION
173
17CS308 - COMPUTER ARCHITECTURE
2. ASSOCIATIVE MAPPING (OR) FULLY ASSOCIATIVE
• A fully associative mapping scheme can overcome the problems of the
direct mapping scheme
– A main memory block can load into any line of cache
– Memory address is interpreted as tag and word
– Tag uniquely identifies block of memory
– Processor gives Tag address which is compared to tag bit of each
cache to find match.
– Every cache line’s tag is examined for a match
– Also need a Dirty and Valid bit
• But Cache searching gets expensive!
– Ideally need circuitry that can simultaneously examine all tags for a
match
– Lots of circuitry needed, high cost
• Need replacement policies now that anything can get thrown out of the
cache

174. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Associative mapping
Main
memory Block 0
Block 1
Block 127
Block 128
Block 129
Block 255
Block 256
Block 257
Block 4095
4
Main memory address
Tag Word
12
tag
tag
tag
Cache
Block 0
Block 1
Block 127

175. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Associative mapping
• Main memory block can be placed into any cache position.
• Memory address is divided into two fields:
- Low order 4 bits identify the word within a block.
- High order 12 bits or tag bits identify a memory block when it is
resident in the cache.
• Flexible, and uses cache space efficiently.
• Replacement algorithms can be used to replace an existing block in
the cache when the cache is full.
• Cost is higher than direct-mapped cache because of the need to
search all 128 patterns to determine whether a given block is in the
cache.

176. 5.2 HIERARCHICAL MEMORY ORGANIZATION
176
17CS308 - COMPUTER ARCHITECTURE
ASSOCIATIVE MAPPING EXAMPLE
Block 0
Block 1
Block 2
Block 3
Block 4
Block 5
Block 6
Block 7
Main
Memory
Slot 0
Slot 1
Slot 2
Slot 3
Cache Memory
Valid Dirty Tag
Block can map to any slot
Tag used to identify which block is in which slot
All slots searched in parallel for target

177. 5.2 HIERARCHICAL MEMORY ORGANIZATION
177
17CS308 - COMPUTER ARCHITECTURE
3.SET ASSOCIATIVE MAPPING
• Compromise between fully-associative and direct-mapped
cache
– Cache is divided into a number of sets
– Each set contains a number of lines
– A given block maps to any line in a specific set
• Use direct-mapping to determine which set in the cache corresponds
to a set in memory
• Memory block could then be in any line of that set
– e.g. 2 lines per set
• 2 way associative mapping
• A given block can be in either of 2 lines in a specific set
– e.g. K lines per set
• K way associative mapping
• A given block can be in one of K lines in a specific set
• Much easier to simultaneously search one set than all lines

178. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Set-Associative mapping

179. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Set-Associative mapping
• Blocks of cache are grouped into sets.
• Mapping function allows a block of the main memory to reside in any block
of a specific set.
• Divide the cache into 64 sets, with two blocks per set.
• Memory block 0, 64, 128 etc. map to block 0, and they can occupy either of
the two positions.
• Memory address is divided into three fields:
- 6 bit field determines the set number.
- High order 6 bit fields are compared to the tag fields of the two blocks
in a set.
• Set-associative mapping combination of direct and associative mapping.
• Number of blocks per set is a design parameter.
- One extreme is to have all the blocks in one set, requiring no set bits
(fully associative mapping).
- Other extreme is to have one block per set, is the same as direct
mapping.

180. 5.2 HIERARCHICAL MEMORY ORGANIZATION
180
17CS308 - COMPUTER ARCHITECTURE
SET ASSOCIATIVE MAPPING
• To compute cache set number:
– SetNum = j mod v
• j = main memory block number
• v = number of sets in cache
Block 0
Block 1
Block 2
Block 3
Main Memory
Slot 0
Slot 1
Slot 2
Slot 3
Set 0
Set 1 Block 4
Block 5

181. 5.2 HIERARCHICAL MEMORY ORGANIZATION
A computer system uses 16-bit memory addresses. It has a 2K-byte cache organized
in a 2-way set-associative cache mapping manner with 64 bytes per cache block.
Assume that the size of each memory word is 1 byte.
(a) Calculate the number of bits in each of the Tag, Block, and Word fields of the
memory address.
(a) Block size = 64 bytes = 26 bytes = 26 words
Therefore, Number of bits in the Word field = 6
Cache size = 2K-byte = 211 bytes
Number of cache blocks per set = 2
Number of sets = Cache size / (Block size * Number of set) = 211 /(26 * 2) = 24
Therefore, Number of bits in the Set field = 4
Total number of address bits = 16
Therefore, Number of bits in the Tag field = 16 - 6 - 4 = 6
For a given 16-bit address,
the 6 most significant bits, represent the Tag,
the next 4 bits represent the set,
and the 6 least significant bits represent the Word.

182. 5.2 HIERARCHICAL MEMORY ORGANIZATION
182
17CS308 - COMPUTER ARCHITECTURE
SET ASSOCIATIVE MAPPING
64K CACHE EXAMPLE
• E.g. Given our 64Kb cache, with a line size of 4 bytes, we have 16384
lines. Say that we decide to create 8192 sets, where each set contains 2
lines. Then we need 13 bits to identify a set (213=8192)
• Use set field to determine cache set to look in
• Compare tag field of all slots in the set to see if we have a hit, e.g.:
– Address = 16339C = 0001 0110 0011 0011 1001 1100
• Tag = 0 0010 1100 = 02C
• Set = 0 1100 1110 0111 = 0CE7
• Word = 00 = 0
– Address = 008004 = 0000 0000 1000 0000 0000 0100
• Tag = 0 0000 0001 = 001
• Set = 0 0000 0000 0001 = 0001
• Word = 00 = 0
Tag 9 bit Set 13 bit
Word
2 bit

183. 5.2 HIERARCHICAL MEMORY ORGANIZATION
183
17CS308 - COMPUTER ARCHITECTURE
REPLACEMENT POLICY (1)
• The replacement policy is the technique we use to determine
which line in the cache should be thrown out when we want
to put a new block in from memory
• Direct mapping
– No choice
– Each block only maps to one line
– Replace that line

184. 5.2 HIERARCHICAL MEMORY ORGANIZATION
184
17CS308 - COMPUTER ARCHITECTURE
REPLACEMENT ALGORITHMS (2)
ASSOCIATIVE & SET ASSOCIATIVE
• Algorithm must be implemented in hardware (speed)
• Least Recently used (LRU)
– e.g. in 2 way set associative, which of the 2 block is LRU?
• For each slot, have an extra bit, USE. Set to 1 when accessed, set all
others to 0.
– For more than 2-way set associative, need a time stamp for each
slot - expensive
• First in first out (FIFO)
– Replace block that has been in cache longest
– Easy to implement as a circular buffer
• Least frequently used (LFU)
– Replace block which has had fewest hits
– Need a counter to sum number of hits
• Random
– Almost as good as LFU and simple to implement

185. 5.2 HIERARCHICAL MEMORY ORGANIZATION
185
17CS308 - COMPUTER ARCHITECTURE
WRITE POLICY
• So far we’ve only discussed reading memory, we may
also write back to memory
• If a memory write only updates the cache, then the cache
is now inconsistent with main memory
– Could cause problems
– Reading in another memory block that maps to the
same cache line
– I/O device or another processor might directly try to
read/write to memory

186. 5.2 HIERARCHICAL MEMORY ORGANIZATION
186
17CS308 - COMPUTER ARCHITECTURE
WRITE THROUGH
• Simplest technique to handle the cache inconsistency
problem - All writes go to main memory as well as cache.
• Multiple CPUs must monitor main memory traffic (snooping)
to keep local cache local to its CPU up to date in case
another CPU also has a copy of a shared memory location in
its cache
• Simple but Lots of traffic
• Slows down writes

187. 5.2 HIERARCHICAL MEMORY ORGANIZATION
187
17CS308 - COMPUTER ARCHITECTURE
WRITE BACK
• Updates initially made in cache only
– Dirty bit is set when we write to the cache, this indicates
the cache is now inconsistent with main memory
• Dirty bit for cache slot is cleared when update occurs
• If cache line is to be replaced, write the existing cache line to
main memory if dirty bit is set before loading the new
memory block

188. 5.2 HIERARCHICAL MEMORY ORGANIZATION
EXAMPLE OF MAPPING FUNCTIONS
• A cache has only eight blocks of data.
• Each block consists of only one word of data. Memory is word
addressable with 16 bit address.
• LRU replacement algorithm is used for block replacement.
Problem:
• A 2 dimensional array (4x10) array of numbers, each occupying one
word is stored in the main memory locations 7A00 through 7A27. The
elements of the are stored in the column order

189. 5.2 HIERARCHICAL MEMORY ORGANIZATION
• A(0,0), A(0,1), A(0,2)…….A(0,9)
• A(1,0), A(1,1), A(1,2)…….A(1,9)
• A(2,0), A(2,1), A(2,2)…….A(2,9)
• A(3,0), A(3,1), A(3,2)…….A(3,9)

190. 5.2 HIERARCHICAL MEMORY ORGANIZATION

191. 5.2 HIERARCHICAL MEMORY ORGANIZATION
The following code is Operated on these data
Sum=0
for j=0 to 9 do
sum=sum+A(0,j) ______________LOOP(1)
end
AVE:= sum/10
for i=9 down to 0 do _________________LOOP(2)
A(0,i):=A(0,i)/AVE;
end

192. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Direct Mapping

193. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Block
position
Content of data cache after pass
j=0 ,
j=1
J=2,j=3 j=4,5 j=6,7 j=9,i=9,
i=8
i=7,6 i=5,4 i=3,2 i=1,0
0 A(0,0) A(0,2) A(0,4) A(0,6) A(0,8) A(0,6) A(0,4) A(0,2) A(0,0)
1
2
3
4 A(0,1) A(0,3) A(0,5) A(0,7) A(0,9) A(0,7) A(0,5) A(0,3) A(0,1)
5
6
7

194. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Assosiative Mapped Cache

195. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Block position Content of data cache after pass
j=0 to j=7 j=8 j=9
(i=9 to i=2)
i=1 i=0
0 A(0,0) A(0,8) A(0,8) A(0,8) A(0,0)
1 A(0,1) A(0,1) A(0,9) A(0,1) A(0,1)
2 A(0,2) A(0,2) A(0,2) A(0,2) A(0,2)
3 A(0,3) A(0,3) A(0,3) A(0,3) A(0,3)
4 A(0,4) A(0,4) A(0,4) A(0,4) A(0,4)
5 A(0,5) A(0,5) A(0,5) A(0,5) A(0,5)
6 A(0,6) A(0,6) A(0,6) A(0,6) A(0,6)
7 A(0,7) A(0,7) A(0,7) A(0,7) A(0,7)

196. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Set –Associative Mapped Cache

197. 5.2 HIERARCHICAL MEMORY ORGANIZATION
• Here the data cache is divided into 2 sets , each capable of holding
4 blocks .The blocks having even addresses are mapped into set 0
of cache. In set 0,they can occupy any one of the four blocks.
• Hence in general ,associative mapping Performs best, set-
associative is next best and direct mapping is the worst.
• However, associative mapping is expensive to implement, so set
associative mapping is performed.

198. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Block
position
Content of data cache after pass
j=0 t0
j=3
j=4 to
j= 7
j=8,9 &
i=9 to6
i=5 i=4 i=3 i=2 i=1 i=0
Set 0 A(0,0) A(0,4) A(0,8) A(0,8) A(0,4) A(0,4) A(0,4) A(0,4) A(0,0)
A(0,1) A(0,5) A(0,9) A(0,5) A(0,5) A(0,5) A(0,5) A(0,1) A(0,1)
A(0,2) A(0,6) A(0,6) A(0,6) A(0,6) A(0,6) A(0,2) A(0,2) A(0,2)
A(0,3) A(0,7) A(0,7) A(0,7) A(0,7) A(0,3) A(0,3) A(0,3) A(0,3)
Set 1

199. 5.2 HIERARCHICAL MEMORY ORGANIZATION
Session Topic
5.1 Semiconductor Memory Technologies, Memory Organization
5.2 Concept of Hierarchical Memory Organization
5.3 Cache Memory, cache size vs. block size
5.4 Mapping Functions
5.5 Replacement Algorithms
5.6 Memory Interleaving
5.7 Write Policies

200. 5.2 HIERARCHICAL MEMORY ORGANIZATION
200
17CS308 - COMPUTER ARCHITECTURE
REPLACEMENT POLICY 1
• The replacement policy is the technique we use to determine which line
in the cache should be thrown out when we want to put a new block in
from memory
• Direct mapping
– No choice
– Each block only maps to one line
– Replace that line

201. 5.2 HIERARCHICAL MEMORY ORGANIZATION
201
17CS308 - COMPUTER ARCHITECTURE
REPLACEMENT ALGORITHMS (2)
ASSOCIATIVE & SET ASSOCIATIVE
• Algorithm must be implemented in hardware (speed)
• Least Recently used (LRU)
– e.g. in 2 way set associative, which of the 2 block is LRU?
• For each slot, have an extra bit, USE. Set to 1 when accessed,
set all others to 0.
– For more than 2-way set associative, need a time stamp for each
slot - expensive

202. 5.2 HIERARCHICAL MEMORY ORGANIZATION
202
17CS308 - COMPUTER ARCHITECTURE
REPLACEMENT ALGORITHMS (2)
ASSOCIATIVE & SET ASSOCIATIVE
• First in first out (FIFO)
– Replace block that has been in cache longest
– Easy to implement as a circular buffer
• Least frequently used
– Replace block which has had fewest hits
– Need a counter to sum number of hits
• Random
– Almost as good as LFU and simple to implement

203. 5.2 HIERARCHICAL MEMORY ORGANIZATION
203
17CS308 - COMPUTER ARCHITECTURE
WRITE POLICY
• So far we’ve only discussed reading memory, we may also write
back to memory
• If a memory write only updates the cache, then the cache is now
inconsistent with main memory
– Could cause problems
– Reading in another memory block that maps to the same cache
line
– I/O device or another processor might directly try to read/write to
memory

204. 5.2 HIERARCHICAL MEMORY ORGANIZATION
204
17CS308 - COMPUTER ARCHITECTURE
WRITE THROUGH
• Simplest technique to handle the cache inconsistency problem - All
writes go to main memory as well as cache.
• Multiple CPUs must monitor main memory traffic (snooping) to keep
local cache local to its CPU up to date in case another CPU also has a
copy of a shared memory location in its cache
• Simple but Lots of traffic
• Slows down writes

205. 5.2 HIERARCHICAL MEMORY ORGANIZATION
205
17CS308 - COMPUTER ARCHITECTURE
WRITE BACK
• Updates initially made in cache only
– Dirty bit is set when we write to the cache, this indicates the cache is
now inconsistent with main memory
• Dirty bit for cache slot is cleared when update occurs
• If cache line is to be replaced, write the existing cache line to main
memory if dirty bit is set before loading the new memory block

206. 5.2 HIERARCHICAL MEMORY ORGANIZATION
READ OPERATION
• LOAD THROUGH
• EARLY RESTART

207. 5.2 HIERARCHICAL MEMORY ORGANIZATION
A key design objective of a computer system is to achieve the best
possible performance at the lowest possible cost.
– Price/performance ratio is a common measure of success.
Performance of a processor depends on:
– How fast machine instructions can be brought into the processor for
execution.
– How fast the instructions can be executed.
PERFORMANCE CONSIDERATIONS

208. 5.2 HIERARCHICAL MEMORY ORGANIZATION
208
17CS308 - COMPUTER ARCHITECTURE
• Algorithm must be implemented in hardware (speed)
• Least Recently used (LRU)
– e.g. in 2 way set associative, which of the 2 block is LRU?
• For each slot, have an extra bit, USE. Set to 1 when accessed, set all
others to 0.
– For more than 2-way set associative, need a time stamp for each
slot - expensive
• First in first out (FIFO)
– Replace block that has been in cache longest
– Easy to implement as a circular buffer
• Least frequently used (LFU)
– Replace block which has had fewest hits
– Need a counter to sum number of hits
• Random
– Almost as good as LFU and simple to implement

209. 5.2 HIERARCHICAL MEMORY ORGANIZATION
209
17CS308 - COMPUTER ARCHITECTURE
• Least Frequently Used (LFU) Page Replacement Algorithm

210. 5.2 HIERARCHICAL MEMORY ORGANIZATION
210
17CS308 - COMPUTER ARCHITECTURE
• First In First Out (FIFO) Page Replacement Algorithm

211. 5.2 HIERARCHICAL MEMORY ORGANIZATION
211
17CS308 - COMPUTER ARCHITECTURE
• Optimal Page Replacement Algorithm

212. 5.2 HIERARCHICAL MEMORY ORGANIZATION
212
17CS308 - COMPUTER ARCHITECTURE
• Least Recently Used (LRU) Replacement Algorithm

213. 5.2 HIERARCHICAL MEMORY ORGANIZATION
213
17CS308 - COMPUTER ARCHITECTURE

214. MODULE 5 – MEMORY ORGANIZATION
Session Topic
5.1 Semiconductor Memory Technologies, Memory
Organization
5.2 Concept of Hierarchical Memory Organization
5.3 Cache Memory, cache size vs. block size
5.4 Mapping Functions
5.5 Replacement Algorithms
5.6 Memory Interleaving
5.7 Write Policies

215. Performance Considerations

216. 5.7 WRITE POLICIES
Other Performance Enhancements
1.Write buffer
•Write-through:
– Each write operation involves writing to the main memory.
– If the processor has to wait for the write operation to be
complete, it slows down the processor.
– Processor does not depend on the results of the write operation.
– Write buffer can be included for temporary storage of write
requests.
– Processor places each write request into the buffer and
continues execution.

217. 5.7 WRITE POLICIES
Other Performance Enhancements
• Write-back:
– Block is written back to the main memory when it is replaced.
– If the processor waits for this write to complete, before reading
the new block, it is slowed down.
– Fast write buffer can hold the block to be written, and the new
– block can be read first.

218. 5.7 WRITE POLICIES
Other Performance Enhancements (Contd.,)
2.Prefetching
•New data are brought into the processor when they are first needed.
•Processor has to wait before the data transfer is complete.
•Prefetch the data into the cache before they are actually needed, or a
before a Read miss occurs.
•Prefetching can be accomplished through software by including a
special instruction in the machine language of the processor.
– Inclusion of prefetch instructions increases the length of the
programs.

219. 5.7 WRITE POLICIES
Other Performance Enhancements (Contd.,)
2.Prefetching
•Prefetching can also be accomplished using hardware:
– Circuitry that attempts to discover patterns in
– memory references and then prefetches according
– to this pattern.

220. 5.7 WRITE POLICIES
Other Performance Enhancements (Contd.,)
3.Lockup-Free Cache
•Cache structure which supports multiple outstanding misses is called a
lockup free cache.
•Since only one miss can be serviced at a time, a lockup free cache
must include circuits that keep track of all the outstanding misses.
•Special registers may hold the necessary information about these
misses.

221. TOPIC: INSTRUCTION SET ON CPU

222. INSTRUCTION SETS ON CPU
 The complete collection of commands and instructions that are
understood by a CPU
 Each instruction typically consists of an opcode that specifies the type
of operation to perform
 In this case study, we will compare the instruction sets of four
common CPUs: Intel x86, ARM, IBM Power, and MIPS.
What is an instruction set?

223. This case study covers the Intel x86 ISA, which is used in many
desktop and laptop CPUs. It provides an overview of the different
instruction categories, including data movement, arithmetic,
logical, and control transfer instructions, and discusses the role
of registers, flags, and memory addressing modes. It also
explores the evolution of the x86 ISA over time and how it has
been extended to support new features and performance
improvements.
INTEL x86 ARCHITECTURE

224. MOV: This instruction moves data between registers or between a register and
memory.
ADD: This instruction adds two operands and stores the result in a register or
memory location.
CMP: This instruction compares two operands and sets flags based on the
result.
JMP: This instruction jumps to a specified memory location.
CALL: This instruction calls a subroutine at a specified memory location
Some of the common instructions in the x86-64 ISA include:

225.  This case study covers the ARM ISA, which is used in many mobile
and embedded CPUs. It provides an overview of the different
instruction categories, including data processing, load/store, branch
and control transfer, and system instructions. It also discusses the
role of registers, condition codes, and memory addressing modes.
The case study explores how the ARM ISA has been optimized for
power efficiency and low-cost implementations.
ARM INSTRUCTION SET

226. .
MOV: This instruction moves data from one location to another.
ADD/SUB: These instructions perform arithmetic addition and subtraction operations.
CMP: This instruction compares two values and sets flags based on the result.
B: This instruction branches to a specified location in memory.
BL: This instruction calls a subroutine and saves the return address in a link register.
PUSH/POP: These instructions push and pop values onto and from the stack.
BX: This instruction branches to a new location and switches execution mode.
Some examples of ARM instructions are:

227. This case study covers the IBM POWER ISA, which is used in many high-end
servers and supercomputers. It provides an overview of the different
instruction categories, including arithmetic, logical, load/store, branch and
control transfer instructions, and discusses the role of registers, flags, and
memory addressing modes. It also explores the unique features of the
POWER ISA, such as its support for out-of-order execution and its vector
processing capabilities.
IBM POWER INSTRUCTION SET

228. This case study covers the MIPS ISA, which is used in many
embedded and educational CPUs. It provides an overview of
the different instruction categories, including arithmetic, logical,
load/store, branch and jump instructions, and discusses the
role of registers, flags, and memory addressing modes. It also
explores the unique features of the MIPS ISA, such as its 32-
bit fixed-width instruction format and its delay slot for branch
instructions.
MIPS INSTRUCTION SET

229. BEQ: This instruction branches to a specified memory location if two operands are
equal.
BNE: This instruction branches to a specified memory location if two operands are
not equal.
AND/OR/XOR: These instructions perform bitwise logical operations.
SLT: This instruction sets a register to 1 if a specified value is less than another
value.
J: This instruction jumps to a new location in the code.
LW/SW: These instructions load and store data from and to memory.
Some of the common instructions in the MIPS ISA include:

230. In conclusion, instruction sets play a crucial role
in the functioning of modern computers, and their
design requires a careful balancing of simplicity
and flexibility.
CONCLUSION

231. DESIGN OF A SIMPLE HYPOTHETICAL
CPU

232. Architecture Overview The architecture
of a CPU is critical to its performance.
We will delve into the various
components that make up a CPU,
including the control unit, arithmetic
logic unit, and registers. We will also
look at different architectures and their
tradeoffs in terms of performance and
power consumption.

233. The main purpose of central processing
units will be
It controls the system.
It performs data movements.
It have the data processing capability.
It have the data storage functionality.

235.  The design of a hypothetical processor can consist of
an ALU to operate on the operands, an accumulator
used to store one of the operands to be processed by
ALU and also to store the partial result and a
temporary register, register file containing four
general-purpose registers.
The processor can perform processing,calculation and
decisions

236. CPU INTERNAL ORGANISATION

237. Background: The CPU should be capable of
performing basic arithmetic and logical
operations, as well as accessing and
manipulating data in memory. The system
should have a minimal instruction set
architecture (ISA) and be optimized for power
efficiency.

238. Step 1: Define the ISA.The first step in
designing the CPU is to define the ISA. The
ISA will determine the instructions that the
CPU can execute, as well as the format of
those instructions. For this hypothetical
system, we'll use a simple RISC (reduced
instruction set computer) ISA with the
following instructions:

239.  Load word (lw): Load a 32-bit word from memory into a
register
 Store word (sw): Store a 32-bit word from a register to
memory
 Add (add):Add two registers and store the result in a third
register
 Subtract (sub): Subtract one register from another and store
the result in a third register
 And (and): Perform a bitwiseAND operation on two
registers and store the result in a third register
 Or (or): Perform a bitwise OR operation on two registers
and store the result in a third register.
 Jump (j): Jump to a target address

240. Step 2: Design the datapathThe
datapath is the part of the CPU
that performs arithmetic and
logical operations. For this
simple CPU, we'll use a 3-stage
pipeline with the following
stages:

241. Instruction fetch (IF): Fetch the next instruction from
memory
Instruction decode and register fetch (ID): Decode the
instruction and fetch the necessary operands from the
register file
Execute (EX): Perform the operation specified by the
instructionThe datapath will include the following
components:
 Program counter (PC): A register that holds the address of
the next instruction to be fetched
 Instruction memory (IM):Amemory unit that holds the
instructions to be executed
 Register file (RF): Aset of registers that hold data values
 ALU:Aunit that performs arithmetic and logical
operations
 Data memory (DM):Amemory unit that holds data value

242. Y 86-64

267. SUN ULTRASPARC
II PIPELINE
STRUCTURE

268. UltraSPARC II

269. INTRODUCTION
❑ The UltraSPARC II processor is a high-performance processor designed
by Sun Microsystems (now Oracle) in the late 1990s.
❑ It was designed to be used in high-end servers and workstations, where
performance and reliability are critical.
❑ One of the key design features of the UltraSPARC II processor is its use
of pipelining to improve its efficiency.
❑ Pipelining is a technique used in processor design to increase the
throughput of instructions
❑ The UltraSPARC II processor has a five-stage pipeline, which is relatively
small compared to some other modern processors.

271. INSTRUCTION FETCH
❑ In the instruction fetch stage, the processor fetches the instruction from
memory. This involves sending the calculated address to the memory controller
and waiting for the instruction to be returned.
❑ In the cache access stage, the processor checks whether the fetched instruction
is in the instruction cache. If the instruction is in the cache, it is retrieved and
forwarded to the decode stage of the pipeline.
❑ By using pipelining in the fetch stage, the UltraSPARC II processor can overlap
the processing of multiple instructions, which can improve the processor's
performance. However, pipelining can also introduce hazards

272. INSTRUCTION DECODE
❑ During the decode stage, the processor examines the opcode of the
instruction that was fetched in the previous stage, and decodes it into a set of
micro-operations that can be executed by the processor's execution units.
❑ This stage includes the decoding of instruction types, register addressing
modes, and immediate values.
❑ The decode stage is important for the efficient execution of instructions in a
pipeline.
❑ By decoding instructions in advance, the processor can prepare the data and
control signals needed for the execution stage, enabling the instruction to be
executed more quickly and efficiently.

273. EXECUTION
❑ During the execute stage, the processor performs the actual computation or
operation specified by the instruction.
❑ The type of operation performed during this stage depends on the instruction
being executed, such as arithmetic operations, logical operations, memory
operations, or branch operations.
❑ The UltraSPARC II processor is a superscalar processor with multiple execution
units, which allows it to execute multiple instructions in parallel, even if they
are dependent on each other.
❑ The execution stage is split into multiple sub-stages, each of which is
responsible for a specific operation. These sub-stages include:

274. ❑ Integer Execution Unit (IEU) - Performs integer operations such as add,
subtract, compare, and shift.
❑ Floating-Point Execution Unit (FPU) - Performs floating-point operations such
as add, subtract, multiply, divide, and square root.
❑ Load/Store Unit (LSU) - Performs memory operations such as loading data
from memory into registers and storing data from registers into memory.
❑ The execute stage also includes forwarding and bypassing logic that allows
data to be passed directly from one execution unit to another without being
written back to the register file, which helps to reduce stalls and improve
performance.

275. MEMORY ACCESS
❑the memory stage is one of the stages in the pipeline, and it is responsible for
accessing data from memory.
❑the memory stage is pipelined to improve performance . The memory stage pipeline
is divided into two sub-stages:
❑ the address generation sub-stage
❑ the data access sub-stage.
❑The memory stage pipeline allows the processor to begin accessing memory while
previous instructions are still being executed, improving overall performance.
❑ By overlapping the memory access with other operations the processor can make
better use of its resources and reduce the amount of time it spends waiting for data
to be fetched from memory