| Subject: |
HP-UX: Asynchronous i/o |
| |
Doc ID : |
Note:139272.1 |
Type: |
BULLETIN |
| |
Last Revision Date: |
21-JUL-2004 |
Status: |
PUBLISHED |
PURPOSE
——-
The purpose of this document is to discuss the implementation of
asynchronous i/o (aio) on HP-UX, specifically to enable aio for the Oracle
RDBMS Server.
SCOPE & APPLICATION
——————-
This note is intended for DBAs and Support Personnel.
1.0 Introduction
================
On HP-UX, when the Oracle datafiles reside on a filesystem, then the DBWR
process(es) make synchronous writes to these datafiles. This means that each
write must return with a ’succesful completion’ before the next write is
issued. This serial operation can lead to a i/o bottleneck. There are two ways
to counteract this:
a. configure multiple DBWR processes
b. use asynchronous i/o
Before deciding on one of these two options, it should be noted that on HP-UX,
aio is *only* possible on a raw device. Put in another way, aio *cannot* be used
on a filesystem.
Multiple DBWRs can be used on a filesystem.
It is not recommended to use both multiple DBWRs and aio.
2.0 Configuring asynchronous i/o
================================
aio requires configuration of both Oracle and HP-UX
2.1 HP-UX Configuration
———————–
a. create the /dev/async character device
% /sbin/mknod /dev/async c 101 0×0
% chown oracle:dba /dev/async
% chmod 660 /dev/async
If the device does not exist, then ENODEV (Err #19) is returned when the device
is accessed:
Async driver not configured : errno=19
If the permissions are incorrect, then EACCES (Err #13) is returned:
Async driver not configured : errno=13
b. configure the async driver in the kernel
Using SAM
-> Kernel Configuration
-> Drivers
-> the driver is called ‘asyncdsk’
Generate new kernel
Reboot
c. set max_async_ports
Using SAM
-> Kernel Configuration
-> Configurable Parameters
-> max_async_ports
max_async_ports limits the maximum number of processes that can concurrently
use /dev/async. Set this parameter to the sum of
‘processes’ from init.ora + number of background processes
The background processes started at instance startup will open /dev/async
twice.
If max_async_ports is reached, subsequent processes will use synchronous i/o.
d. set aio_max_ops
Using SAM
-> Kernel Configuration
-> Configurable Parameters
-> aio_max_ops
aio_max_ops limits the maximum number of asynchronous i/o operations that
can be queued at any time. Set this parameter to the default value (2048),
and monitor over time (use glance).
e. if HP-UX patch PHKL_22126 (or one that supersedes it) is installed, then
see Section 3.0
2.2 Oracle Configuration
————————
The following init.ora parameters are required.
a. pre 7.3.0
no init.ora parameters required
b. 7.3.X
use_async_io = TRUE
c. 8.X
disk_asynch_io = TRUE
3.0 Implications of HP-UX Patch PHKL_22126
==========================================
This patch is called:
PHKL_22126: s700_800 11.00 VM,async,hyperfabric
The HP-UX patch description says:
The effective user ID of the process calling async driver,
typically called by a process for database applications such
as Oracle, must be a superuser or the user must
be a member of a group that has the MLOCK privilege.
In essence, aio will not work for Oracle, unless the dba group has the MLOCK
privilege.
To check if a group has the MLOCK privilege, execute:
% /usr/bin/getprivgrp <group-name>
If the dba group is not listed, then set it:
% /usr/sbin/setprivgrp <group-name> MLOCK
Note, the next reboot will clear this privilege. To automate this at startup,
create /etc/privgroup with the entry
dba MLOCK
This can be tested with
% /usr/sbin/setprivgrp -f /etc/privgroup
See ‘man 1m setprivgrp’ for more information.
If the MLOCK privilege is not granted to the dba group, then instance startup
will exhibit the following:
a. prior to 8.1.7, each background and shadow process will dump a trace file
with the following:
Ioctl ASYNC_CONFIG error, errno = 1
errno 1 is EPERM (Not super-user).
Such trace files may be found in
$ORACLE_HOME/rdbms/log
background_dump_dest
user_dump_dest
Additionally, a tusc of instance startup will show the following for
each background process:
[16044] open("/dev/async", O_RDWR, 01760) ……………. = 14
…
[16044] ioctl(14, 0×80106101, 0×7f7f51b0) ……………. ERR#1 EPERM
Similar output will be given for a tusc of a shadow process.
The instance does start, but i/o is synchronous
b. in 8.1.7, the instance will not start, and will error with:
SVRMGR> startup
ORA-00445: background process "PMON" did not start after 120 seconds
See Note 133007.1
for the alert on this.
c. in 9.0.1, the behaviour is similar to pre-8.1.7, in that trace files with
the following are dumped:
Ioctl ASYNC_CONFIG error, errno = 1
Additionally, the following is also seen:
Ioctl ASYNC_ADDSEG error, errno = 1
The instance does start, but i/o is synchronous.
4.0 Checking that Asynchronous i/o is being used
================================================
Async i/o is being used if both the following are true:
a. /dev/async is open by DBWR
b. the relevant init.ora partameter is set (see Section 2.2)
c. the datafiles are on raw devices
To check if /dev/async is open by DBWR, do one of:
i. % fuser /dev/async
ii. % lsof -p <dbwr pid>
lsof can be downloaded from ftp://vic.cc.purdue.edu/pub/tools/unix/lsof
iii. use glance/gpm to check files opened by DBWR
NB - Oracle server processes (background and foreground) will attempt to open
/dev/async if the async driver is enabled in the HP-UX kernel, regardless
of init.ora settings.
If (a), (b)and (c) are true, and i/o is still perceived to be synchronous, attach
to DBWR with tusc, and check that write calls to /dev/async are not returning an
error. The file descriptor for /dev/async will be needed, which can be retrieved
using lsof.
5.0 The fs_async kernel parameter
=================================
The kernel parameter fs_async can be set to allow asynchronous writes to file
systems. However, write calls can return without the data being physically
written to disk (the write sits in the UNIX buffer cache). The data in question
is file-system metadata such as free space lists, blocks and inodes.
A system crash would potentially lose this data, and leave the filesystem in an
inconsistent state, causing database corruption.
Oracle still opens files with the O_DSYNC flag (see ‘tusc’ snipet
below), which insists that writes are physically written:
open("/oracle/datafiles/system01.dbf", O_RDWR|O_LARGEFILE|O_DSYNC, 0) = 19
In summary, fs_async is ignored for datafiles(due to open() with O_DSYNC).
However, filesystem metadata may be lost, potentially causing datafile
corruption.
Oracle does not recommend setting fs_async to ‘1′.
Settings:
fs_async=0 Do not use async writes to file systems
fs_async=1 Do async writes to file systems
RELATED DOCUMENTS
—————–
Note 174487.1
- ALERT:HP-UX: RDBMS May Not Start if Async Disk Driver is
Configured
Note 133007.1
- ALERT: HP/UX: 8.1.7 RDBMS will not start if the async disk
driver is configured