vdmcache(7) DG/UX R4.11MU05 vdmcache(7)
NAME
vdmcache - Virtual Disk Manager Cache Subdriver
DESCRIPTION
The Cache Subdriver provides higher speed access to slow disk devices
by caching read and write I/Os to a fast device. The large, slow
disk where the data is stored is called the Back-End Device, or BED.
The small, fast device that the caching is done to is called the
Front-End Device, or FED.
A cache works by intercepting the read and write I/Os to the slow BED
and redirecting them to the fast FED. For reads, if the data needed
is already in the FED, then the I/O can be satisfied by reading the
data from the fast FED instead of the slow BED. For writes, the data
is written to the fast FED and the I/O returns to the caller. Later,
the cache will flush the I/O from the FED to the BED for permanent
storage. This caching can provide significant performance
improvements to the applications using the data. If caches are used
in inappropriate environments, or are tuned poorly, they can have a
significant negative effect on performance as well, so they must be
used carefully.
Pictorially, a cache containing one FED and the BED looks like this:
+-------------------+
| cache "X11" |
| is 120000 blocks |
+-------------------+
| |
+-------+ +-------+
| |
+-------------------+ +-------------------+
| FED partition | | BED partition |
| starts at 1 | | starts at 1000 |
| is 2000 blocks | | is 120000 blocks |
+-------------------+ +-------------------+
| |
| |
| |
+-------------------+ +-------------------+
| physical | | physical |
| nvrd(vme(0),0) | | sd(ncsc(0,7),1,0) |
+-------------------+ +-------------------+
| |
| |
+==========+ ----------
| | ( )
| NVRAM | ( )
| Board | ( disk )
| | ( drive )
+==========+ ( 1 )
( )
----------
Caches refer to their children as FEDs and BEDs. So in this example,
the cache has two children, one of which is the FED on a fast NVRAM
board, and the other of which is the BED on a slow disk drive.
The BED is where all the user data is permanently stored. It is a
large, relatively slow disk device. The FED is a small, relatively
fast device that is used as the temporary caching store. The Cache
Subdriver supports two types of FEDs.
The most common type of FED is the NVRAM, or Non-Volatile Random
Access Memory board. It is a small, 2- or 4-megabyte memory board
that has a long-life battery on-board. To the DG/UX system, it
appears as a very small, very fast disk device. The data written to
the board is kept safe during power failures by the battery that is
permanently attached to the board. When the system comes back on-
line, the cache will flush the data from the FED to the BED as a
normal part of cache recovery.
Fast disk drives can also be used as FEDs for caches. For this to be
practical, the speed of the FED disk must be many times the speed of
the BED disk. Otherwise, the overhead cost of moving the data
between the FED and the BED exceeds the cost of reading and writing
to the BED directly without the cache present. NVRAM boards satisfy
the speed difference requirement, so they are ideal FED devices.
Most traditional disk devices make poor FED devices.
Caches can help performance the most when the application is issuing
small fixed-size random reads and writes, especially if the access
pattern creates "hot spots," where a group of disk blocks are read or
written frequently.
Applications that issue large sequential I/Os, very heavy I/O
volumes, or significantly varying sizes of I/Os will tend to overrun
the cache's ability to store the I/Os efficiently in the FEDs. This
will force the cache into "pass-through" mode, and the performance
benefits of the caching will be lost. In such cases, the performance
with the cache enabled is frequently worse than without it.
In order to use the Cache Subdriver, you must configure vdmcache()
into your kernel. It appears in all new system files by default, so
it is usually already configured.
For more details about how the Cache Subdriver fits into the Virtual
Disk Manager, see the vdm(7) man page.
SETTING UP A NVRAM BOARD
A NVRAM board appears to the DG/UX system as a small, fast disk
drive. It is treated as a disk drive from the VDM's perspective.
This means that in order for it to be used, it must be soft-formatted
with admpdisk(1M) and registered to the VDM. Once it is registered,
build virtual disks on it for the FED or FEDs you wish to use.
There is no need to install bootstraps on a NVRAM device, since they
cannot be booted from. The bootstraps will just take up valuable
space that could be used for caching.
There is no need to install bad-block remapping on a NVRAM device,
either. NVRAMs use ECC memory, so they can correct single-bit faults
in their memory chips. Bad-block remapping provides similar
protection, which would be redundant for NVRAM boards. Further, the
remap areas will just take up valuable space that could be used for
caching.
If you are concerned about the potential failure of a NVRAM board,
you can set up NVRAM board hardware pairs that will mirror the data
on two boards. If one board fails, the other board will take over
and continue.
For more information on NVRAM devices, see the nvrd(7) man page.
MULTIPLE FRONT-END DEVICES
A cache always has one BED device where the user data is permanently
stored. The cache may have zero, one, or several FEDs, depending on
your needs. A cache with zero FEDs is said to be in "pass-through"
mode, where each I/O that is issued is passed through the cache
directly to the BED for processing. No caching is being done in this
case.
A cache normally has one FED dedicated to it. This allows the cache
exclusive use of the FED and provides the best performance to the
applications.
A cache may have multiple FEDs dedicated to it. This gives the cache
a larger caching store to process I/Os, and can improve the
performance of the cache if a single FED is constantly being overrun
because it is too small to handle the load on the cache. For best
performance, specify each FED as a separate child to the cache rather
than aggregating them together.
Multiple caches can also share a single FED or a group of FEDs. If
you have multiple BEDs that you wish to cache, but the I/O loads to
them differ widely, sharing FEDs between them can be a cost-effective
solution. For example, if one virtual disk gets a heavy load during
the day, but a light load at night, and a second virtual disk gets
the opposite loading, then sharing FEDs between them would work well.
If both virtual disks are under similar loads, sharing FEDs will not
work well because the BEDs will have to compete with each other for
space in the FEDs to process I/Os.
You can adjust the tuning parameters of the caches to bias the FED
accesses towards a particular BED or towards particular types of I/O.
FORMATTING A FRONT-END DEVICE
A FED is implicitly formatted when it is linked to a cache as part of
the link operation of admvdisk(1M). There is no way for a custom
written program to format a cache FED.
CACHE TUNING
There are a large number of tuning parameters for caches that can be
used to control the caching behavior. The details of the parameters
are explained in the structure definitions below.
In general, you want to tune the cache to have high hit rates,
meaning that a large percentage of the I/Os are satisfied by the FED
directly without accessing the BED. Further, you want low stall
counts, which means that the FED is being kept clean enough so that
new I/Os can be processed efficiently, without first being required
to flush an old I/O buffer to the BED. The combination of these two
goals will mean that most I/O goes to the FED and returns quickly,
and the cache flushes it efficiently behind the scenes to the BED
later.
The table below summarizes the parameters and their meaning.
Cache reads, writes, and only metadata
These control the types of I/O that the cache will store in
the FEDs. Other I/O will be handled pass-through to the BED.
Async flush on first or all writes
These control whether the cache will flush written data from
the FED to the BED on just the first write or on all writes.
Various settings can keep hot data in the FED for best
performance, or force flush it to the BED to help keep the
FEDs clean so other data can be cached.
Disable DMA-to-VME-transfers
This controls whether direct DMA transfers are allowed between
the FED and the BED. Generally, direct DMA transfers are the
fastest.
Read and write retention values
These are values that are added to the retention value for an
I/O buffer in the FED each time it is touched for a read or
write. When searching for buffers to reuse, ones with lower
retention values are reused first. Various settings enable
you to bias the data kept in the FED to improve performance
for the particular types of I/Os that you care about.
Search percentage
This controls the percentage of the FED buffers that are
searched when a new I/O has to be cached in the FED and an
existing I/O buffer must be reused. A small percentage means
the search will happen quickly, but will do a poorer job of
finding the best buffer to reuse. The setting is a trade-off
between speed and efficiency.
Flusher type
This controls the type of flusher thread that will be used for
the cache. Various settings allow you to control how the
flushing is done.
The best way to tune a cache is to make small incremental changes in
the setup and test each one to measure the effects. We recommend
that you first measure the performance of your application without a
cache to determine a baseline to compare against. Then create a
cache with the default settings and measure the performance again.
You can measure the performance with nsar(1) and by getting the
cache's attributes with the list operation of admvdisk(1M). The
cache listing includes items such as the cache hit rates, the FED
buffer counts, the I/O stall counts, etc.
Make sure that each measurement of performance is significant. This
means that you must run the measurement long enough to ensure that
you're testing what you really want to test instead of intermittent
spikes in I/O load. Generally, a measurement of 15 to 30 minutes
provides very useful results.
As you identify each cache parameter that you want to change, change
them individually, then measure the performance effect. If the
effect is positive, keep the setting. If it's negative, go back to
the previous setting.
Tuning a cache is a complex process. It depends upon the I/O load
that your application is generating, and therefore the best
configuration will be application-dependent.
USABILITY
For you to be able to use a cache (i.e., open it), all of its
children must be available and usable. This means that the BED must
be usable, and each of the FEDs must also be usable. If any children
are missing, the cache cannot be opened, and the data in it is
unavailable.
If the BED disk is moved to another system without also moving the
FEDs, then the cache will not be usable on the new system because the
FEDs will be missing. To cleanly move a BED disk, unlink the FEDs
from the cache on the original system first. This will cause the
cache to flush all the data from the FEDs to the BED. Then the BED
can be moved independently of the FEDs.
EXPANDING AND SHRINKING CACHES
Caches cannot be expanded or shrunk. In order to change the size of
the BED device, dismantle the cache first by unlinking the FEDs from
the cache and extracting the cache itself. Then change the size of
the BED. Finally, rebuild the cache by inserting it and linking back
the FEDs.
This is required because the cache is not prepared to handle the BED
changing size while it is in use.
CLUSTER DISKS
Caches are not supported on cluster disks. This is because NVRAM
FEDs cannot be placed on the shared SCSI bus, which means that other
cluster nodes could not access them. This prevents any practical
sharing of caching resources among cluster nodes.
Note that CLARiiON® hardware disk caching works for cluster disks and
provides similar performance benefits.
PROGRAMMING INTERFACE
The ioctl(2) system call is used for all communication with the Cache
Subdriver. Use the open(2) system call to open a channel to the
/dev/vdm node, and issue all ioctl(2) commands through that channel.
See vdm(7) for more details on the programming interface.
The <sys/ioctl.h> include file is required for communicating with the
Cache Subdriver. The <errno.h>, <sys/dg_sysctl.h>, <sys/types.h>,
<stdio.h>, and <unistd.h> include files will be helpful.
SUBDRIVER-SPECIFIC COMMANDS
DG_VDMCACHE_GET_FED_STATS
Return the statistics for a cache Front-End Device.
COMMON DISK COMMANDS
The following commands are common to all device drivers that manage
physical disks. The VDM and the subdrivers support the commands on
virtual disks as well, because they present the same interfaces as
physical disks.
DSKIOCGET
This command returns the size of the virtual disk in sectors. For
virtual disks, one sector is one disk block, or 512 bytes. The
command and associated structure is described in more detail in
dsk(7).
DSKIOCUSAGE
This command returns I/O count and performance statistics on the I/O
issued to a virtual disk. The command and associated packet is
described in more detail in dsk(7).
SUBDRIVER-SPECIFIC DEFINES
DG_VDMCACHE_SUBDRIVER_ID
The unique ID that identifies the Cache Subdriver.
DG_VDMCACHE_IOCTL_PACKET_VERSION_0
The version zero stamp used in all Cache Subdriver ioctl(2) packets.
The stamp indicates which version of the Cache Subdriver interface is
being used and allows the Cache Subdriver to support previous
versions without requiring that applications be recompiled.
DG_VDMCACHE_BED_CHILD
The value indicating that this child is the Back-End Device. The BED
is the large, slow device that stores all the user data.
DG_VDMCACHE_FED_CHILD
The value indicating that this child is the Front-End Device. The
FED is the small, fast device that stores the cached user data from
the BED.
DG_VDMCACHE_RETURN_STATS
The flag value indicating that the current statistics are to be
returned.
DG_VDMCACHE_RESET_STATS
The flag value indicating that the current statistics are to be
returned, then all the statistics counters are to be reset to zero.
DG_VDMCACHE_POLICY_NO_FLUSHER
The flusher policy value indicating that no flusher is to be used
with the cache. Buffers are flushed to the BED only when space is
required for new buffers.
DG_VDMCACHE_POLICY_CYCLE_FLUSHER
The flusher policy value indicating that a cycling flusher is to be
used with the cache. The flusher periodically cycles through the
cache, flushing out any dirty buffers to the BED.
DG_VDMCACHE_POLICY_MIN_SEARCH_PERCENTAGE
The policy value indicating the minimum legal buffer search
percentage. If this value is used, the first available buffer in the
cache is used.
DG_VDMCACHE_POLICY_MAX_SEARCH_PERCENTAGE
The policy value indicating the maximum legal buffer search
percentage. If this value is used, the entire cache is searched for
the best buffer to use.
DG_VDMCACHE_STATE_UP
The cache state indicating that the cache is up and ready to be used.
DG_VDMCACHE_STATE_DOWN
The cache state indicating that the cache is normally shut down.
DG_VDMCACHE_STATE_CHANGING_POLICY
The cache state indicating that the cache policy is being changed.
DG_VDMCACHE_STATE_RECOVERING
The cache state indicating that recovery is being run on the FED
buffers.
DG_VDMCACHE_STATE_SEALED_BED_FAILED
The cache state indicating that the Back-End Device has failed.
DG_VDMCACHE_STATE_SEALED_FED_FAILED
The cache state indicating that the Front-End Device has failed.
DG_VDMCACHE_STATE_SEALED_CACHE_FAILED
The cache state indicating that the internal structures used to
manage the cache are corrupted.
STRUCTURES
dg_vdmcache_create_packet
The Cache Subdriver ioctl(2) packet for creating a new cache
instance. Put a pointer to this packet into the packet for the
DG_VDM_CREATE_INSTANCE command.
struct dg_vdmcache_create_packet
{
int version;
dev_t child_device_number;
};
version
The version of the packet.
child_device_number
The device number of the Back-End Device. Format the Front-
End Devices and link them into the cache after the cache is
created.
dg_vdmcache_get_attributes_packet
The Cache Subdriver ioctl(2) packet for getting the attributes of a
cache instance. Put a pointer to this packet into the packet for the
DG_VDM_GET_ATTRIBUTES command.
If both the cache_reads and cache_writes flags are set to zero, or
the cache has no Front-End Devices associated with it, the cache will
act in pass-through mode.
struct dg_vdmcache_get_attributes_packet
{
int version;
unsigned int flags;
unsigned int state;
unsigned int read_count;
unsigned int write_count;
unsigned int read_miss_count;
unsigned int write_miss_count;
unsigned int read_hit_count;
unsigned int write_hit_count;
unsigned int read_purge_count;
unsigned int write_purge_count;
unsigned int allocate_purge_count;
unsigned int bed_write_count;
unsigned int bed_read_count;
unsigned int total_buffers_count;
unsigned int total_data_blocks_count;
unsigned int dirty_buffers_count;
unsigned int allocated_buffers_count;
unsigned int allocated_data_blocks_count;
unsigned int state_stall_count;
unsigned int flush_stall_count;
unsigned int cache_reads : 1;
unsigned int cache_writes : 1;
unsigned int cache_only_meta_data : 1;
unsigned int async_flush_on_first_write : 1;
unsigned int async_flush_on_all_writes : 1;
unsigned int disable_dma_to_vme : 1;
int read_retention;
int write_retention;
int search_percentage;
int flusher_type;
};
version
The version of the packet.
flags An input flag indicating whether the statistics are to be
returned (DG_VDMCACHE_RETURN_STATS), or whether the statistics
are to be returned and then reset to zero
(DG_VDMCACHE_RESET_STATS).
state The returned state of the cache, which will be
DG_VDMCACHE_STATE_UP, DG_VDMCACHE_STATE_DOWN,
DG_VDMCACHE_STATE_CHANGING_POLICY,
DG_VDMCACHE_STATE_RECOVERING,
DG_VDMCACHE_STATE_SEALED_BED_FAILED,
DG_VDMCACHE_STATE_SEALED_FED_FAILED, or
DG_VDMCACHE_STATE_SEALED_CACHE_FAILED.
read_count
The returned total number of reads done to the cache.
write_count
The returned total number of writes done to the cache.
read_miss_count
The returned total number of times a read did not find the
target buffer in the cache.
write_miss_count
The returned total number of times a write did not find the
target buffer in the cache.
read_hit_count
The returned total number of times a read did find the target
buffer in the cache.
write_hit_count
The returned total number of times a write did find the target
buffer in the cache.
read_purge_count
The returned total number of times a read caused existing
buffers to be purged, because the read request overlapped but
did not exactly match the existing buffers.
write_purge_count
The returned total number of times a write caused existing
buffers to be purged, because the write request overlapped but
did not exactly match the existing buffers.
allocate_purge_count
The returned total number of times a buffer allocate request
caused existing buffers to be purged, because there was not
sufficient contiguous space available in the existing buffers
to satisfy the request.
bed_write_count
The returned total number of times a buffer was written to the
Back-End Device.
bed_read_count
The returned total number of times a buffer was read from the
Back-End Device.
total_buffers_count
The returned total number of buffers on all of the cache's
Front-End Devices. Buffers awaiting recovery are not usable.
total_data_blocks_count
The returned total number of data blocks on all of the cache's
Front-End Devices. Data blocks awaiting recovery are not
usable.
dirty_buffers_count
The returned number of dirty buffers.
allocated_buffers_count
The returned number of allocated buffers.
allocated_data_blocks_count
The returned number of data blocks allocated to buffers.
state_stall_count
The returned total number of times a thread waited for an
outstanding I/O on a buffer to complete before it could
perform its own I/O request on the buffer.
flush_stall_count
The returned total number of times a thread had to flush a
buffer before it could use it.
cache_reads
The returned flag indicating whether read requests are being
cached. If set to one, reads are cached.
cache_writes
The returned flag indicating whether write requests are being
cached. If set to one, writes are cached.
cache_only_meta_data
The returned flag indicating whether only DG/UX file system
metadata is being cached. If set to one, only metadata is
being cached (user data is read or written in pass-through
mode).
async_flush_on_first_write
The returned flag indicating whether the first time a new
buffer is written to the cache, an asynchronous I/O should be
issued to flush it to the Back-End Device. If set to one, a
buffer will be flushed on its first write. This setting helps
to keep the cache clean when there are a lot of buffers that
are written only once (e.g., large sequential writes).
async_flush_on_all_writes
The returned flag indicating whether any time a buffer is
written to the cache, an asynchronous I/O should be issued to
flush it to the Back-End Device. If set to one, a buffer will
be flushed on any write. This setting may improve performance
in caches doing more reads than writes. If this setting is
used, async_flush_on_first_write is also assumed.
disable_dma_to_vme
The returned flag indicating whether DMA-to-VME-transfer is
disabled, even if the system is capable of performing it. If
set to one, the direct transfer is disabled, and I/O will be
done using the standard I/O mechanisms. Enabling the direct
transfer may improve performance on NVRAM caches when used
with disk controllers that are capable of DMA-to-VME-
transfers.
read_retention
The returned read retention value. Each time a buffer is
read, this value is added to the buffer's retention count.
When the cache is being searched for available space, buffers
with lower retention counts are reused first. Thus, buffers
with high retention counts are retained in the cache longer.
This value is used only if cache_reads is set to one.
write_retention
The returned write retention value. Each time a buffer is
written, this value is added to the buffer's retention count.
When the cache is being searched for available space, buffers
with lower retention counts are reused first. Thus, buffers
with high retention counts are retained in the cache longer.
This value is used only if cache_writes is set to one.
search_percentage
The returned percentage of the cache that is searched each
time a buffer or data block is required. This percentage of
the cache is searched looking for a clean buffer with the
lowest retention count. If no clean buffer is found, a dirty
buffer with the lowest retention value is flushed and reused.
On the next search, the search will begin where the last
search left off, so eventually the entire cache will be
searched. If set to DG_VDMCACHE_POLICY_MIN_SEARCH_PERCENTAGE,
the first available buffer is used. If set to
DG_VDMCACHE_POLICY_MAX_SEARCH_PERCENTAGE, the entire cache is
searched.
flusher_type
The returned type of flusher being used with the cache. Will
be either DG_VDMCACHE_POLICY_NO_FLUSHER or
DG_VDMCACHE_POLICY_CYCLE_FLUSHER.
dg_vdmcache_update_attributes_packet
The Cache Subdriver ioctl(2) packet for updating the attributes of a
cache instance. Put a pointer to this packet into the packet for the
DG_VDM_UPDATE_ATTRIBUTES command.
If both the cache_reads and cache_writes flags are set to zero, or
the cache has no Front-End Devices associated with it, the cache will
act in pass-through mode.
All fields are input fields and must be set to the value desired. If
no change is desired, set the field to its current value from the
cache's attributes.
struct dg_vdmcache_update_attributes_packet
{
int version;
unsigned int cache_reads : 1;
unsigned int cache_writes : 1;
unsigned int cache_only_meta_data : 1;
unsigned int async_flush_on_first_write : 1;
unsigned int async_flush_on_all_writes : 1;
unsigned int disable_dma_to_vme : 1;
int read_retention;
int write_retention;
int search_percentage;
int flusher_type;
};
version
The version of the packet.
cache_reads
The flag indicating whether read requests are to be cached.
If set to one, reads will be cached.
cache_writes
The flag indicating whether write requests are to be cached.
If set to one, writes will be cached.
cache_only_meta_data
The flag indicating whether only DG/UX file system metadata is
to be cached. If set to one, only metadata will be cached
(user data is read or written in pass-through mode).
async_flush_on_first_write
The flag indicating whether the first time a new buffer is
written to the cache, an asynchronous I/O should be issued to
flush it to the Back-End Device. If set to one, a buffer will
be flushed on its first write. This setting helps to keep the
cache clean when there are a lot of buffers that are written
only once (e.g., large sequential writes).
async_flush_on_all_writes
The flag indicating whether any time a buffer is written to
the cache, an asynchronous I/O should be issued to flush it to
the Back-End Device. If set to one, a buffer will be flushed
on any write. This setting may improve performance in caches
doing more reads than writes. If this setting is used,
async_flush_on_first_write is also assumed.
disable_dma_to_vme
The flag indicating whether DMA-to-VME-transfer is to be
disabled, even if the system is capable of performing it. If
set to one, the direct transfer is disabled, and I/O will be
done using the standard I/O mechanisms. Enabling the direct
transfer may improve performance on NVRAM caches when used
with disk controllers that are capable of DMA-to-VME-
transfers.
read_retention
The read retention value. Each time a buffer is read, this
value is added to the buffer's retention count. When the
cache is being searched for available space, buffers with
lower retention counts are reused first. Thus, buffers with
high retention counts are retained in the cache longer. This
value is used only if cache_reads is set to one.
write_retention
The write retention value. Each time a buffer is written,
this value is added to the buffer's retention count. When the
cache is being searched for available space, buffers with
lower retention counts are reused first. Thus, buffers with
high retention counts are retained in the cache longer. This
value is used only if cache_writes is set to one.
search_percentage
The percentage of the cache to be searched each time a buffer
or data block is required. This percentage of the cache is
searched looking for a clean buffer with the lowest retention
count. If no clean buffer is found, a dirty buffer with the
lowest retention value is flushed and reused. On the next
search, the search will begin where the last search left off,
so eventually the entire cache will be searched. If set to
DG_VDMCACHE_POLICY_MIN_SEARCH_PERCENTAGE, the first available
buffer is used. If set to
DG_VDMCACHE_POLICY_MAX_SEARCH_PERCENTAGE, the entire cache is
searched.
flusher_type
The type of flusher to be used with the cache. Set it to
either DG_VDMCACHE_POLICY_NO_FLUSHER or
DG_VDMCACHE_POLICY_CYCLE_FLUSHER.
dg_vdmcache_get_child_packet
The Cache Subdriver ioctl(2) packet for getting child role
information about the children of a cache instance. Put a pointer to
this packet into the child instance sub-packet for the
DG_VDM_GET_CHILD_INSTANCE command.
struct dg_vdmcache_get_child_packet
{
int version;
int child_type;
};
version
The version of the packet.
child_type
The returned type of the child. This identifies the role the
child plays for the cache. Will be either
DG_VDMCACHE_BED_CHILD or DG_VDMCACHE_FED_CHILD.
dg_vdmcache_fed_stats_packet
The Cache Subdriver ioctl(2) packet for returning the statistics for
a cache Front-End Device. Put a pointer to this packet into the
argument field of the packet for the DG_VDM_IOCTL_SUBDRIVER command,
and set the command field to DG_VDMCACHE_GET_FED_STATS. Put the
device number of the cache instance to query in the instance device
number field.
struct dg_vdmcache_fed_stats_packet
{
int version;
unsigned int flags;
dev_t fed_device_number;
unsigned int read_count;
unsigned int write_count;
unsigned int total_buffers_count;
unsigned int total_data_blocks_count;
unsigned int allocated_buffer_nodes_count;
unsigned int allocated_data_blocks_count;
unsigned int dirty_buffers_count;
unsigned int buffer_nodes_awaiting_recovery_count;
unsigned int data_blocks_awaiting_recovery_count;
};
version
The version of the packet.
flags An input flag indicating whether the statistics are to be
returned (DG_VDMCACHE_RETURN_STATS), or whether the statistics
are to be returned and then reset to zero
(DG_VDMCACHE_RESET_STATS).
fed_device_number
The device number of the Front-End Device to return statistics
for.
read_count
The returned total number of times a buffer was read from the
Front-End Device.
write_count
The returned total number of times a buffer was written to the
Front-End Device.
total_buffers_count
The returned total number of buffers on the Front-End Device.
total_data_blocks_count
The returned total number of data blocks on the Front-End
Device.
allocated_buffers_count
The returned number of allocated buffers on the Front-End
Device.
allocated_data_blocks_count
The returned number of allocated data blocks on the Front-End
Device.
dirty_buffers_count
The returned number of dirty buffers on the Front-End Device.
buffers_awaiting_recovery_count
The returned number of buffers requiring recovery on the
Front-End Device. If a Front-End Device is shared by several
caches, and one or more of the Back-End Devices it supports
fails to reconnect to its cache during disk registration, the
Front-End Device will hold the buffers belonging to that Back-
End Device in the Front-End Device awaiting recovery. The
recovery will complete when the Back-End Device reconnects.
Buffers awaiting recovery are not usable for any other
purpose.
data_blocks_awaiting_recovery_count
The returned number of data blocks requiring recovery on the
Front-End Device. If a Front-End Device is shared by several
caches, and one or more of the Back-End Devices it supports
fails to reconnect to its cache during disk registration, the
Front-End Device will hold the data blocks belonging to that
Back-End Device in the Front-End Device awaiting recovery.
The recovery will complete when the Back-End Device
reconnects. Data blocks awaiting recovery are not usable for
any other purpose.
EXAMPLES
The following code fragments illustrate how to use the packets and
commands described in this man page. The fragments are not complete
and are not intended to be compilable or executable. In particular,
initialization code, instance locating code, and error handling are
left out for brevity.
Common includes and variables
#include <errno.h>
#include <sys/dg_sysctl.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <stdio.h>
#include <unistd.h>
int status; /* status from system call */
int vdm_channel; /* channel to /dev/vdm node */
Create a cache
dev_t cache_instance;
dev_t bed_instance;
struct dg_vdm_create_instance_packet create_pkt;
struct dg_vdmcache_create_packet cache_create_pkt;
create_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
create_pkt.subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
create_pkt.persistent = 1;
create_pkt.enable_disk_updates = 1;
strcpy (create_pkt.instance_name, "inventory_db");
create_pkt.subdriver_attributes_packet_ptr = &cache_create_pkt;
cache_create_pkt.version = DG_VDMCACHE_IOCTL_PACKET_VERSION_0;
cache_create_pkt.child_device_number = bed_instance;
status = ioctl (vdm_channel,
DG_VDM_CREATE_INSTANCE,
&create_pkt);
cache_instance = create_pkt.device_number;
printf ("Cache device number = 0x%08x\n", cache_instance);
Update a cache's flusher type
dev_t update_instance;
struct dg_vdm_get_attributes_packet get_attrs_pkt;
struct dg_vdm_update_attributes_packet update_attrs_pkt;
struct dg_vdmcache_get_attributes_packet cache_get_pkt;
struct dg_vdmcache_update_packet cache_update_pkt;
/* Get the cache's current attributes. */
get_attrs_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
get_attrs_pkt.instance_device_number = update_instance;
get_attrs_pkt.subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
get_attrs_pkt.subdriver_attributes_packet_ptr = &cache_get_pkt;
cache_get_pkt.version = DG_VDMCACHE_IOCTL_PACKET_VERSION_0;
cache_get_pkt.flags = DG_VDMCACHE_RETURN_STATS;
status = ioctl (vdm_channel,
DG_VDM_GET_ATTRIBUTES,
&get_attrs_pkt);
/* Copy the current attributes into the update packet. */
cache_update_pkt.cache_reads =
cache_get_pkt.cache_reads;
cache_update_pkt.cache_writes =
cache_get_pkt.cache_writes;
cache_update_pkt.cache_only_meta_data =
cache_get_pkt.cache_only_meta_data;
cache_update_pkt.async_flush_on_first_write =
cache_get_pkt.async_flush_on_first_write;
cache_update_pkt.async_flush_on_all_writes =
cache_get_pkt.async_flush_on_all_writes;
cache_update_pkt.disable_dma_to_vme =
cache_get_pkt.disable_dma_to_vme;
cache_update_pkt.read_retention =
cache_get_pkt.read_retention;
cache_update_pkt.write_retention =
cache_get_pkt.write_retention;
cache_update_pkt.search_percentage =
cache_get_pkt.search_percentage;
cache_update_pkt.flusher_type =
cache_get_pkt.flusher_type;
/* Update the cache's flusher type. */
update_attrs_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
update_attrs_pkt.instance_device_number = update_instance;
update_attrs_pkt.subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
update_attrs_pkt.subdriver_attributes_packet_ptr = &cache_update_pkt;
cache_update_pkt.version = DG_VDMCACHE_IOCTL_PACKET_VERSION_0;
cache_update_pkt.flusher_type = DG_VDMCACHE_POLICY_CYCLE_FLUSHER;
status = ioctl (vdm_channel,
DG_VDM_UPDATE_ATTRIBUTES,
&update_attrs_pkt);
Get a cache's attributes
char *flusher_name_ptr;
char *state_name_ptr;
unsigned int read_hit_miss_total;
unsigned int write_hit_miss_total;
double read_hit_rate;
double write_hit_rate;
dev_t get_instance;
struct dg_vdm_get_attributes_packet get_attrs_pkt;
struct dg_vdmcache_get_attributes_packet cache_get_pkt;
get_attrs_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
get_attrs_pkt.instance_device_number = get_instance;
get_attrs_pkt.subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
get_attrs_pkt.subdriver_attributes_packet_ptr = &cache_get_pkt;
cache_get_pkt.version = DG_VDMCACHE_IOCTL_PACKET_VERSION_0;
cache_get_pkt.flags = DG_VDMCACHE_RETURN_STATS;
status = ioctl (vdm_channel,
DG_VDM_GET_ATTRIBUTES,
&get_attrs_pkt);
switch (cache_get_pkt.state)
{
case DG_VDMCACHE_STATE_UP:
state_name_ptr = "up";
break;
case DG_VDMCACHE_STATE_DOWN:
state_name_ptr = "down";
break;
case DG_VDMCACHE_STATE_CHANGING_POLICY:
state_name_ptr = "changing policy";
break;
case DG_VDMCACHE_STATE_RECOVERING:
state_name_ptr = "recovering";
break;
case DG_VDMCACHE_STATE_SEALED_BED_FAILED:
state_name_ptr = "sealed - bed failed";
break;
case DG_VDMCACHE_STATE_SEALED_FED_FAILED:
state_name_ptr = "sealed - fed failed";
break;
case DG_VDMCACHE_STATE_SEALED_CACHE_FAILED:
state_name_ptr = "sealed - cache failed";
break;
}
switch (cache_get_pkt.flusher_type)
{
case DG_VDMCACHE_POLICY_NO_FLUSHER:
flusher_name_ptr = "none";
break;
case DG_VDMCACHE_POLICY_CYCLE_FLUSHER:
flusher_name_ptr = "cycle";
break;
}
printf ("\nState = %u (%s)\n",
cache_get_pkt.state,
state_name_ptr);
printf ("\nPolicy:\n\n");
printf ("Cache reads = %-5s Async flush on first write = %s\n",
(cache_get_pkt.cache_reads) ? "true" : "false",
(cache_get_pkt.async_flush_on_first_write) ?
"true" : "false");
printf ("Cache writes = %-5s Async flush on all writes = %s\n",
(cache_get_pkt.cache_writes) ? "true" : "false",
(cache_get_pkt.async_flush_on_all_writes) ?
"true" : "false");
printf ("Cache only metadata = %-5s Disable DMA-to-VME = %s\n",
(cache_get_pkt.cache_only_meta_data) ?
"true" : "false",
(cache_get_pkt.disable_dma_to_vme) ?
"true" : "false");
printf ("Read retention = %-10d Search Percentage = %d%%\n",
cache_get_pkt.read_retention,
cache_get_pkt.search_percentage);
printf ("Write retention = %-10d Flusher type = %d (%s)\n",
cache_get_pkt.write_retention,
cache_get_pkt.flusher_type,
flusher_name_ptr);
printf ("\nStatistics:\n\n");
printf ("Read count = %-10u Write count = %u\n",
cache_get_pkt.read_count,
cache_get_pkt.write_count);
printf ("Read hits = %-10u Write hits = %u\n",
cache_get_pkt.read_hit_count,
cache_get_pkt.write_hit_count);
printf ("Read misses = %-10u Write misses = %u\n",
cache_get_pkt.read_miss_count,
cache_get_pkt.write_miss_count);
printf ("Read purges = %-10u Write purges = %u\n",
cache_get_pkt.read_purge_count,
cache_get_pkt.write_purge_count);
read_hit_miss_total =
(cache_get_pkt.read_hit_count) +
(cache_get_pkt.read_miss_count);
if (read_hit_miss_total == 0)
{
read_hit_rate = 0.0;
}
else
{
read_hit_rate =
(((double) (cache_get_pkt.read_hit_count)) * 100.0) /
((double) (read_hit_miss_total));
}
write_hit_miss_total =
(cache_get_pkt.write_hit_count) +
(cache_get_pkt.write_miss_count);
if (write_hit_miss_total == 0)
{
write_hit_rate = 0.0;
}
else
{
write_hit_rate =
(((double) (cache_get_pkt.write_hit_count)) * 100.0) /
((double) (write_hit_miss_total));
}
printf ("Read hit rate = %-3.1f%% Write hit rate = %3.1f%%\n",
read_hit_rate, write_hit_rate);
printf ("BED read count = %-10u BED write count = %u\n",
cache_get_pkt.bed_read_count,
cache_get_pkt.bed_write_count);
printf ("Allocated buffers = %-10u Total buffers = %u\n",
cache_get_pkt.allocated_buffers_count,
cache_get_pkt.total_buffers_count);
printf ("Allocated data blocks = %-10u Total data blocks = %u\n",
cache_get_pkt.allocated_data_blocks_count,
cache_get_pkt.total_data_blocks_count);
printf ("Dirty buffers = %-10u State stalls = %u\n",
cache_get_pkt.dirty_buffers_count,
cache_get_pkt.state_stall_count);
printf ("Allocate purges = %-10u Flush stalls = %u\n",
cache_get_pkt.allocate_purge_count,
cache_get_pkt.flush_stall_count);
Insert a cache
dev_t bed_instance;
dev_t cache_instance;
dev_t dummy_instance;
dev_t insert_instance;
struct dg_vdm_create_instance_packet create_pkt;
struct dg_vdm_insert_instance_packet insert_pkt;
struct dg_vdmcache_create_packet cache_create_pkt;
/* Create a dummy instance. */
create_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
create_pkt.subdriver_id = DG_VDMDUMMY_SUBDRIVER_ID;
create_pkt.persistent = 1;
create_pkt.enable_disk_updates = 1;
create_pkt.instance_name [0] = '\0';
create_pkt.subdriver_attributes_packet_ptr = NULL;
status = ioctl (vdm_channel,
DG_VDM_CREATE_INSTANCE,
&create_pkt);
dummy_instance = create_pkt.device_number;
/* Insert a cache. */
insert_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
insert_pkt.instance_to_change = insert_instance;
insert_pkt.dummy_instance = dummy_instance;
insert_pkt.subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
insert_pkt.attributes_packet_ptr = &cache_create_pkt;
cache_create_pkt.version = DG_VDMCACHE_IOCTL_PACKET_VERSION_0;
cache_create_pkt.child_device_number = dummy_instance;
status = ioctl (vdm_channel,
DG_VDM_INSERT_INSTANCE,
&insert_pkt);
cache_instance = insert_instance;
bed_instance = dummy_instance;
Extract a cache
dev_t cache_instance;
dev_t child_instance;
dev_t dummy_instance;
dev_t extract_instance;
struct dg_vdm_create_instance_packet create_pkt;
struct dg_vdm_delete_instance_packet delete_pkt;
struct dg_vdm_extract_instance_packet extract_pkt;
/* Create a dummy instance. */
create_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
create_pkt.subdriver_id = DG_VDMDUMMY_SUBDRIVER_ID;
create_pkt.persistent = 1;
create_pkt.enable_disk_updates = 1;
create_pkt.instance_name [0] = '\0';
create_pkt.subdriver_attributes_packet_ptr = NULL;
status = ioctl (vdm_channel,
DG_VDM_CREATE_INSTANCE,
&create_pkt);
dummy_instance = create_pkt.device_number;
/* Extract the cache. */
extract_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
extract_pkt.parent_device_number = extract_instance;
extract_pkt.new_parent_device_number = dummy_instance;
status = ioctl (vdm_channel,
DG_VDM_EXTRACT_INSTANCE,
&extract_pkt);
cache_instance = dummy_instance;
dummy_instance = child_instance;
/* Delete the leftover instance and the child dummy instance. */
delete_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
delete_pkt.device_number = cache_instance;
status = ioctl (vdm_channel,
DG_VDM_DELETE_INSTANCE,
&delete_pkt);
delete_pkt.device_number = dummy_instance;
status = ioctl (vdm_channel,
DG_VDM_DELETE_INSTANCE,
&delete_pkt);
Link a formatted FED to a cache
dev_t link_instance;
dev_t new_fed_instance;
struct dg_vdm_link_child_instance_packet link_pkt;
link_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
link_pkt.parent_device_number = link_instance;
link_pkt.parent_subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
link_pkt.child_device_number = new_fed_instance;
link_pkt.child_packet_ptr = NULL;
status = ioctl (vdm_channel,
DG_VDM_LINK_CHILD_INSTANCE,
&link_pkt);
Get the children of a cache
dev_t cache_instance;
dev_t child_instance;
struct dg_vdm_child_instance_packet child_pkt;
struct dg_vdm_get_child_instance_packet get_child_pkt;
struct dg_vdmcache_get_child_packet cache_get_child_pkt;
get_child_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
get_child_pkt.key = 0;
get_child_pkt.parent_device_number = cache_instance;
get_child_pkt.parent_subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
get_child_pkt.available = 0;
get_child_pkt.child_instance_array_ptr = NULL;
status = ioctl (vdm_channel,
DG_VDM_GET_CHILD_INSTANCE,
&get_child_pkt);
printf ("Number of children = %d\n",
get_child_pkt.number_of_children);
get_child_pkt.key = 0;
get_child_pkt.available = 1;
get_child_pkt.child_instance_array_ptr = &child_pkt;
child_pkt.child_role_packet_ptr = &cache_get_child_pkt;
cache_get_child_pkt.version = DG_VDMCACHE_IOCTL_PACKET_VERSION_0;
while (1)
{
status = ioctl (vdm_channel,
DG_VDM_GET_CHILD_INSTANCE,
&get_child_pkt);
if ((status == -1) && (errno == ENOENT))
{
break;
}
switch (cache_get_child_pkt.child_type)
{
case DG_VDMCACHE_BED_CHILD:
printf ("Back-End Device\n");
break;
case DG_VDMCACHE_FED_CHILD:
printf ("Front-End Device\n");
break;
}
if (child_pkt.child_instance.device_number_used)
{
child_instance = child_instance_pkt.child_instance.
specifier_value.device_number;
printf ("Child device number = 0x%08x\n", child_instance);
if (child_pkt.child_subdriver_id ==
DG_VDM_SUBDRIVER_ID_OF_A_NON_VDM_DEVICE)
{
printf (" non-vdm child\n");
}
else
{
printf (" child subdriver 0x%08x\n",
child_pkt.child_subdriver_id);
}
}
else
{
printf (" missing child ID %08x,%08x\n",
child_pkt.child_instance.specifier_value.id.generation,
child_pkt.child_instance.specifier_value.id.system_id);
}
printf ("\n");
}
Unlink a FED from a cache
dev_t unlink_instance;
dev_t child_instance;
struct dg_vdm_unlink_child_instance_packet unlink_pkt;
unlink_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
unlink_pkt.parent_device_number = unlink_instance;
unlink_pkt.child_instance.device_number_used = 1;
unlink_pkt.child_instance.specifier_value.device_number =
child_instance;
status = ioctl (vdm_channel,
DG_VDM_UNLINK_CHILD_INSTANCE,
&unlink_pkt);
Get the statistics for a FED
dev_t ioctl_instance;
struct dg_vdm_ioctl_subdriver_packet ioctl_pkt;
struct dg_vdmcache_fed_stats_packet fed_get_pkt;
ioctl_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
ioctl_pkt.subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
ioctl_pkt.instance_device_number = ioctl_instance;
ioctl_pkt.command = DG_VDMCACHE_GET_FED_STATS;
ioctl_pkt.argument = (int) &fed_get_pkt;
fed_get_pkt.version = DG_VDMCACHE_IOCTL_PACKET_VERSION_0;
fed_get_pkt.flags = DG_VDMCACHE_RETURN_STATS;
status = ioctl (vdm_channel,
DG_VDM_IOCTL_SUBDRIVER,
&ioctl_pkt);
printf ("Reads = %-10u Writes = %u\n",
fed_get_pkt.read_count,
fed_get_pkt.write_count);
printf ("Total buffers = %-10u Total data blocks = %u\n",
fed_get_pkt.total_buffers_count,
fed_get_pkt.total_data_blocks_count);
printf ("Allocated buffer nodes = %-10u Allocated data blocks = %u\n",
fed_get_pkt.allocated_buffer_nodes_count,
fed_get_pkt.allocated_data_blocks_count);
printf ("Recovery buffer nodes = %-10u Recovery data blocks = %u\n",
fed_get_pkt.buffer_nodes_awaiting_recovery_count,
fed_get_pkt.data_blocks_awaiting_recovery_count);
printf ("Dirty buffers = %u\n",
fed_get_pkt.dirty_buffers_count);
Get the size of a cache
dev_t ioctl_instance;
struct dg_vdm_ioctl_subdriver_packet ioctl_pkt;
struct dskget dskget_pkt;
ioctl_pkt.version = DG_VDM_IOCTL_PACKET_VERSION_0;
ioctl_pkt.subdriver_id = DG_VDMCACHE_SUBDRIVER_ID;
ioctl_pkt.instance_device_number = ioctl_instance;
ioctl_pkt.command = DSKIOCGET;
ioctl_pkt.argument = (int) &dskget_pkt;
status = ioctl (vdm_channel,
DG_VDM_IOCTL_SUBDRIVER,
&ioctl_pkt);
printf ("Total blocks = %u\n",
dskget_pkt.total_sectors *
(dskget_pkt.bytes_per_sector / 512));
FILES
/dev/dsk
Directory for the block special device nodes for virtual
disks. These are the buffered access device nodes.
/dev/rdsk
Directory for the character special device nodes for virtual
disks. These are the unbuffered (or raw) access device nodes.
/dev/vdm
Device node for issuing all ioctl(2) commands.
/usr/include/sys/ioctl.h
ioctl(2) definitions.
ACCESS CONTROL
Access control for the standard VDM framework commands is controlled
by the VDM itself. See the vdm(7) man page for a listing of these
controls.
Access control for the Cache Subdriver commands is controlled by the
subdriver itself. Below is the listing of these commands.
All users may execute these commands: DG_VDM_GET_ATTRIBUTES (to
return cache statistics), DG_VDM_GET_CHILD_INSTANCE,
DG_VDMCACHE_GET_FED_STATS (to return FED statistics), DSKIOCGET,
DSKIOCUSAGE.
Only users with appropriate privilege may execute these commands:
DG_VDM_CREATE_INSTANCE, DG_VDM_GET_ATTRIBUTES (to return and reset
cache statistics), DG_VDM_LINK_CHILD_INSTANCE,
DG_VDM_UNLINK_CHILD_INSTANCE, DG_VDM_UPDATE_ATTRIBUTES,
DG_VDMCACHE_GET_FED_STATS (to return and reset FED statistics).
On a generic DG/UX system, appropriate privilege is granted by having
an effective UID of 0 (root). See the appropriate_privilege(5) man
page for more information.
On a system with DG/UX information security, appropriate privilege is
granted by having one or more specific capabilities enabled in the
effective capability set of the user. See the cap_defaults(5) man
page for the default capabilities for these commands.
RETURN VALUE
The return value from the ioctl(2) system call used to issue the
command will be one of the following:
0 The command was successful.
-1 An error occurred. errno is set to indicate the error.
DIAGNOSTICS
There are descriptive extended error message strings available for
most of the error codes. They can be retrieved with the
dg_ext_errno(2), extended_strerror(3C), and extended_perror(3C)
routines.
Errno may be set to one of the DG/UX standard error codes. See
/usr/include/sys/errno.h for a listing of those error codes.
Errno may be set to one of the standard error codes provided by the
VDM framework. See vdm(7) for a listing of those error codes.
Errno may also be set to one of the following error codes that are
specific to this subdriver:
EINVAL The front-end device is not formatted for use with the cache.
EINVAL The buffer size is too large for the cache front-end device.
EINVAL The cache front-end device type is not valid.
EINVAL The cache policy option is not valid.
EINVAL No need to purge a cache buffer, there are now free buffers.
EINVAL The flags field in the cache packet is not valid.
EIO The cache virtual disk is in an error state and cannot be
used.
EIO The cache device is sealed. An internal buffer state is not
valid.
ENOSPC There are no more buffers available on the cache front-end
device.
ENXIO The virtual disk is not a front-end device for the cache.
SEE ALSO
ioctl(2), dsk(7), nvrd(7), rdsk(7), vdm(7).
Licensed material--property of copyright holder(s)