From: Shane Dowdy (shane.dowdy@unetworks.com)
Date: Wed Oct 23 2002 - 10:27:57 EDT
Here is my hardware and problem description:
Sun 420r
Solaris 8 - new install, no patches
Emulex LP8000 Fibre-Channel adapter
MTI Vivant 20 - 24 disks - config: 2 striped & mirrored sets plus 4 spares
I have installed the driver, configured the MTI from it's management
station, and tried everything I know how to get the system to map and use
the array on the fibre channel. I read and followed all the Emulex
documentation to the letter, tried many combinations of edits to the config
files. I have used fsck, format, luxadm, and a mass of other programs I have
read about in file system management documentation I have found, but nothing
seems to be able to use these volumes across the fibre. There is a
connection somewhere I must not be putting together. I have included some
information below I hope will help you help me. I have never ran into
anything like this, mind-numbing amounts of documentation has not helped,
and I'm out of time. I throw myself at the mercy of my peers.
The Emulex driver comes with a utility (lputil) that is able to see the
nodes and set up persistent binding.
$ cd /dev/dsk
$ ls -l
total 80
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s0 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:a
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s1 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:b
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s2 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:c
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s3 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:d
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s4 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:e
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s5 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:f
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s6 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:g
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t0d0s7 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:h
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s0 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:a
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s1 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:b
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s2 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:c
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s3 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:d
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s4 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:e
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s5 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:f
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s6 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:g
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t1d0s7 ->
../../devices/pci@1f,4000/scsi@3/sd@1,0:h
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s0 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:a
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s1 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:b
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s2 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:c
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s3 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:d
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s4 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:e
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s5 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:f
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s6 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:g
lrwxrwxrwx 1 root root 41 Sep 26 16:07 c0t6d0s7 ->
../../devices/pci@1f,4000/scsi@3/sd@6,0:h
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t20d1s0 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:a
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t20d1s1 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:b
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t20d1s2 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:c
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t20d1s3 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:d
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t20d1s4 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:e
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t20d1s5 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:f
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t20d1s6 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:g
lrwxrwxrwx 1 root other 51 Oct 3 18:18 c2t20d1s7 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@14,1:h
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s0 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:a
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s1 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:b
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s2 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:c
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s3 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:d
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s4 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:e
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s5 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:f
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s6 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:g
lrwxrwxrwx 1 root other 51 Oct 3 18:17 c2t22d2s7 ->
../../devices/pci@1f,4000/fibre-channel@2/sd@16,2:h
Solaris Configs:
############################################################################
#################################################################
######################################################################[lpfc.
conf]############################################################
----------snip----------
# Verbosity: only turn this flag on if you are willing to risk being
# deluged with LOTS of information.
# You can set a bit mask to record specific types of verbose messages:
#
# 0x1 ELS events
# 0x2 Device Discovery events
# 0x4 Mailbox Command events
# 0x8 Miscellaneous events
# 0x10 Link Attention events
# 0x20 IP events
# 0x40 FCP events
# 0x80 Node table events
# 0x1000 FCP Check Condition events
log-verbose=0;
# Setting log-only to 0 causes log messages to be printed on the
# console and to be logged to syslog (which may send them to the
# console again if it's configured to do so).
# Setting log-only to 1 causes log messages to go to syslog only.
log-only=1;
#
# +++ Variables relating to FCP (SCSI) support. +++
#
# Setup FCP persistent bindings,
# fcp-bind-WWPN binds a specific WorldWide PortName to a target id,
# fcp-bind-WWNN binds a specific WorldWide NodeName to a target id,
# fcp-bind-DID binds a specific DID to a target id.
# Only one binding method can be used.
# WWNN, WWPN and DID are hexadecimal values.
# WWNN must be 16 digits with leading 0s.
# WWPN must be 16 digits with leading 0s.
# DID must be 6 digits with leading 0s.
# The SCSI ID to bind to consists of two parts, the lpfc interface
# to bind to, and the target number for that interface.
# Thus lpfc0t2 specifies target 2 on interface lpfc0.
# NOTE: Target ids, with all luns supported, must also be in sd.conf.
# scan-down must be set to 0 or 1, not 2 which is the default!!
#
# Here are some examples:
# WWNN SCSI ID
# fcp-bind-WWNN="2000123456789abc:lpfc1t0",
# "20000020370c27f7:lpfc0t2";
#
# WWPN SCSI ID
# fcp-bind-WWPN="2100123456789abc:lpfc0t0",
# "21000020370c2855:lpfc0t1",
# "2100122222222222:lpfc2t2";
#
# DID SCSI ID
# fcp-bind-DID="0000ef:lpfc0t3";
# BEGIN: LPUTIL-managed Persistent Bindings
fcp-bind-WWNN="200000a0f1000e24:lpfc0t22",
"200000a0f1000e23:lpfc0t20";
# If automap is set, SCSI IDs for all FCP nodes without
# persistent bindings will be automatically generated.
# If new FCP devices are added to the network when the system is down,
# there is no guarantee that these SCSI IDs will remain the same
# when the system is booted again.
# If one of the above fcp binding methods is specified, then automap
# devices will use the same mapping method to preserve
# SCSI IDs between link down and link up.
# If no bindings are specified above, a value of 1 will force WWNN
# binding, 2 for WWPN binding, and 3 for DID binding.
# If automap is 0, only devices with persistent bindings will be
# recognized by the system.
automap=0;
# fcp-on: true (1) if FCP access is enabled, false (0) if not.
fcp-on=1;
# lun-queue-depth: the default value lpfc will use to limit
# the number of outstanding commands per FCP LUN. This value is
# global, affecting each LUN recognized by the driver, but may be
# overridden on a per-LUN basis (see below). RAID arrays may want
# to be configured using the per-LUN tunable throttles.
lun-queue-depth=30;
# tgt-queue-depth: the default value lpfc will use to limit
# the number of outstanding commands per FCP target. This value is
# global, affecting each target recognized by the driver, but may be
# overridden on a per-target basis (see below). RAID arrays may want
# to be configured using the per-target tunable throttles. A value
# of 0 means don't throttle the target.
tgt-queue-depth=0;
# lpfcNtM-lun-throttle: the maximum number of outstanding commands to
# permit for each LUN of an FCP target that supports multiple LUNs.
# The default throttle for the number of commands outstanding to a single
# LUN of a multiple-LUN target is lun-queue-depth. For a target that
# can support multiple LUNs, it may be useful to specify a LUN throttle
# that differs from the default.
# Example: lpfc0t17-lun-throttle=48;
# says that each LUN on target 17, interface lpfc0 should be allowed
# up to 48 simultaneously outstanding commands.
#lpfc1t39-lun-throttle=10;
#lpfc0t40-lun-throttle=30;
# lpfcNtM-tgt-throttle: the maximum number of outstanding commands to
# permit for a FCP target.
# By default, target throttle is diabled.
# Example: lpfc0t17-tgt-throttle=48;
# says that target 17, interface lpfc0 should be allowed
# up to 48 simultaneously outstanding commands.
#lpfc1t39-tgt-throttle=10;
#lpfc0t40-tgt-throttle=30;
# no-device-delay [0 to 30] - determines the length of
# the interval between deciding to fail back an I/O because there is no way
# to communicate with its particular device (e.g., due to device failure)
and
# the actual fail back. A value of zero implies no delay whatsoever.
# Cautions: (1) This value is in seconds.
# (2) Setting a long delay value may permit I/O to build up,
# each with a pending timeout, which could result in the exhaustion of
# critical Solaris kernel resources. In this case, you may see a fatal
# message such as
# PANIC: Timeout table overflow
#
# Note that this value can have an impact on the speed with which a
# system can shut down with I/Os pending and with the HBA not able to
# communicate with the loop or fabric, e.g., with a cable pulled.
no-device-delay=1;
#
# +++ Variables relating to IP networking support. +++
#
# network-on: true (1) if networking is enabled, false (0) if not
# This variable will be set during the installation of the driver
# via pkgadd.
network-on=0;
# xmt-que-size: size of the transmit queue for mbufs (128 - 10240)
xmt-que-size=256;
#
# +++ Variables common to both SCSI (FCP) and IP networking support. +++
#
# Some disk devices have a "select ID" or "select Target" capability.
# From a protocol standpoint "select ID" usually means select the
# Fibre channel "ALPA". In the FC-AL Profile there is an "informative
# annex" which contains a table that maps a "select ID" (a number
# between 0 and 7F) to an ALPA. If scan-down is set to a value of 0,
# the lpfc driver assigns target ids by scanning its ALPA map
# from low ALPA to high ALPA.
#
# Turning on the scan-down variable (on = 1,2, off = 0) will
# cause the lpfc driver to use an inverted ALPA map, effectively
# scanning ALPAs from high to low as specified in the FC-AL annex.
# A value of 2, will also cause target assignment in a private loop
# environment to be based on the ALPA (hard addressed).
#
# Note: This "select ID" functionality is a PRIVATE LOOP ONLY
# characteristic and will not work across a fabric.
scan-down=1;
# Determine how long the driver will wait to begin linkdown processing
# when a cable has been pulled or the link has otherwise become
# inaccessible, 1 - 255 secs. Linkdown processing includes failing back
# cmds to the target driver that have been waiting around for the link
# to come back up. There's a tradeoff here: small values of the timer
# cause the link to appear to "bounce", while large values of the
# timer can delay failover in a fault tolerant environment. Units are in
# seconds. A value of 0 means never failback cmds until the link comes up.
linkdown-tmo=30;
# If set, nodev-holdio will hold all I/O errors on devices that disappear
# until they come back. Default is 0, return errors with no-device-delay.
nodev-holdio=0;
# If set, nodev-tmo will hold all I/O errors on devices that disappear
# until the timer expires. Default is 0, return errors with no-device-delay.
nodev-tmo=0;
# Use no-device-delay to delay FCP RSP errors and certain check conditions.
delay-rsp-err=0;
# Treat certain check conditions as an FCP error.
check-cond-err=0;
# num-iocbs: number of iocb buffers to allocate (128 to 10240)
num-iocbs=1024;
# num-bufs: number of ELS buffers to allocate (128 to 4096)
# ELS buffers are needed to support Fibre channel Extended Link Services.
# Also used for SLI-2 FCP buffers, one per FCP command, and Mailbox
commands.
num-bufs=1024;
# topology: link topology for initializing the Fibre Channel connection.
# 0 = attempt loop mode, if it fails attempt point-to-point mode
# 2 = attempt point-to-point mode only
# 4 = attempt loop mode only
# 6 = attempt point-to-point mode, if it fails attempt loop mode
# Set point-to-point mode if you want to run as an N_Port.
# Set loop mode if you want to run as an NL_Port.
topology=0;
# Set a preferred ALPA for the adapter, only valid if topology is loop.
lpfc0-assign-alpa=2; Request ALPA 2 for lpfc0
# ip-class: FC class (2 or 3) to use for the IP protocol.
ip-class=3;
# fcp-class: FC class (2 or 3) to use for the FCP protocol.
fcp-class=3;
# Use ADISC for FCP rediscovery instead of PLOGI.
use-adisc=1;
# Extra FCP timeout for fabrics (in seconds).
fcpfabric-tmo=0;
# Number of 4k STREAMS buffers to post to IP ring.
post-ip-buf=128;
# Set to 1 to decrement lun throttle on a queue full condition.
dqfull-throttle=1;
#Use dqfull-throttle-up-time to specify when to increment the current Q
depth.
# This variable is in seconds.
dqfull-throttle-up-time=1;
# Increment the current Q depth by dqfull-throttle-up-inc
dqfull-throttle-up-inc=1;
# Use ACK0, instead of ACK1 for class 2 acknowledgement.
ack0=0;
# cr-delay: Coalesce Response Delay
# This value specifies a count of milliseconds after which an interrupt
response
# is generated if cr-count has not been satisfied. This value is set to 0
# to disable the Coalesce Response feature as default.
cr-delay=0;
# cr-count: Coalesce Response Count
# This value specifies a count of I/O completions after which an interrupt
respo
nse
# is generated. This feature is disabled if cr-delay is set to 0.
cr-count=0;
# Used only by i386 FCP (SCSI)
flow_control="duplx" queue="qfifo" disk="scdk" tape="sctp";
# Solaris/x86 only: select allocation of memory for DMA. THIS VARIABLE
# CAN AFFECT WHETHER LPFC RUNS CORRECTLY ON AN X86 PLATFORM. The Solaris
# DDI specification mandates the use of ddi_dma_mem_alloc when allocating
# memory suitable for DMA. This memory comes from a pool reserved at
# boot-time and sized by a variable called "lomempages"; this variable
# may be set in /etc/system. The variable defaults to a small value, e.g.,
# 36 pages, which isn't nearly enough for LPFC when running IP. Typically,
# we've cranked the value up to 1100 pages or so. But this pool represents
# precious "low memory" on a PC -- memory below the 16M address boundary.
# This memory is also needed by the OS and other drivers.
#
# On some machines, we can get away with using kmem_zalloc instead of
# ddi_dma_mem_alloc, thus avoiding the requirement to use lomempages.
# However, this trick is NOT portable! Some x86 systems absolutely need
# to use lomempages for their DMA.
#
# So... if you think your x86 system is one of those that requires the
# use of lomempages, set this variable to one. Be sure to pick a suitable
# value for lomempages in /etc/system; the value depends on how many of
# the various kinds of buffers you allocate for IP and SCSI. Otherwise,
# set this variable to zero and relax, as then lpfc can allocate the
# memory it needs without further input from you.
use-lomempages=0;
# Old Open Boot Prom (SPARC): if your SPARC doesn't have a sufficiently
# recent version of OBP, it may be unable to probe and identify a
# LightPulse adapter. You will need to use the following workaround.
# Important note: you can't just use the following three lines "as is"!
# Refer to the Solaris LightPulse Device Driver documentation for details.
#reg = 0x00801000, 0x00000000, 0x00000000, 0x00000000, 0x00000000, # PCI
# 0x02801010, 0x00000000, 0x00000000, 0x00000000, 0x00001000, # SLIM
# 0x02801018, 0x00000000, 0x00000000, 0x00000000, 0x00000100; # CSRs
# link-speed: link speed selection for initializing the Fibre Channel
connectio
n.
# 0 = auto select (default)
# 1 = 1 Gigabaud
# 2 = 2 Gigabaud
link-speed=0;
# MultiPulse configuration
#
# multipulse-fcp="lpfcXtYdZ:tr1: ... :lpfcXtYdZ:trN:route_flags";
# Where X, Y, and Z are the devices lpfc DDI interface, target, and LUN
numbers.
# tr1 thru trN, when N is up to 4, are the paths component of the traffic
ratio
# route_flags MUST be a 4 digit hex number for the following flags:
# Load balancing flags (1 means cold standby)
# MPL_PC_TYPE_FAILOVER 0x0001 /* cold standby */
# MPL_PC_TYPE_TRAFFIC_RATIO 0x0002 /* paths balanced by traffic ratio
*/
# MPL_PC_TYPE_DYNAMIC_LUNQ 0x0003 /* Dynamic balancing algrithnm used
*/
# MPL_PC_TYPE_DYNAMIC_TGTQ 0x0004 /* Dynamic balancing algrithnm used
*/
# MPL_PC_TYPE_DYNAMIC_HBAQ 0x0005 /* Dynamic balancing algrithnm used
*/
#
# General flags
# MPL_PC_CFG_AUTO_FAILBACK 0x0010 /* failback to primary path is auto
*/
# MPL_PC_CFG_VALIDATE_INQUIRY 0x0020 /* Vaildate all paths with inquiry
*/
# MPL_PC_CFG_INQUIRY_HEARTBEAT 0x0040 /* Validate path inquiry heartbeat
*/
# MPL_PC_CFG_FAIL_ABORT_TASK_SET 0x0080 /* Send ABORT_TASK_SET on failover
*/
# MPL_PC_CFG_FAIL_LUN_RESET 0x0100 /* Send LUN_RESET on failover */
# MPL_PC_CFG_FAIL_TARGET_RESET 0x0200 /* Send TARGET_RESET on failover */
#
# The following example uses cold standby as path control and automatic
failback
#
# Here is a sample configuration to setup lpfc0 target 0 lun 1 failover
# to the MultiPulse device, lpfc1 target 0 lun 1 and lpfc0 target 0 lun 2
# failover to the MultiPulse device, lpfc1 target 1 lun 4.
# multipulse-fcp="lpfc0t0d1:0:lpfc1t0d1:0:0010",
# "lpfc0t0d2:0:lpfc1t1d4:0:0010";
#
# BEGIN: MultiPulse managed entries
#
############################################################################
#################################################################
##################################################################[sd.conf]#
#################################################################
#
# Copyright (c) 1992, by Sun Microsystems, Inc.
#
#ident "@(#)sd.conf 1.9 98/01/11 SMI"
name="sd" class="scsi" class_prop="atapi"
target=0 lun=0;
name="sd" class="scsi" class_prop="atapi"
target=1 lun=0;
name="sd" class="scsi" class_prop="atapi"
target=2 lun=0;
name="sd" class="scsi" class_prop="atapi"
target=3 lun=0;
# Start lpfc auto-generated configuration -- do NOT alter or delete this
line
# WARNING: anything you put within this auto-generated section will
# be DELETED if you execute pkgrm to remove the lpfc driver package.
# You may need to add additional lines to probe for additional LUNs
# or targets. You SHOULD delete any lines that represent lpfc targets
# or LUNs that are not used.
# You should add any new entries between this line
# and the End lpfc auto generated configuration line
# name="sd" parent="lpfc" target=16 lun=0;
# name="sd" parent="lpfc" target=17 lun=0;
# A small number of LUNs for a RAID array
name="sd" parent="lpfc" target=22 lun=2;
name="sd" parent="lpfc" target=20 lun=1;
# name="sd" parent="lpfc" target=17 lun=3;
# End lpfc auto-generated configuration -- do NOT alter or delete this line
(END)
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:25:09 EDT