Summary DMP and 3510FC troubles

---------- Forwarded message ----------
From: Kristopher Briscoe <kris.briscoe@gmail.com>
Date: Oct 4, 2006 11:25 AM
Subject: Summary DMP and 3510FC troubles
To:

All:

Well after a lot of work the problem has finally been resolved.  I must say
I was very surprised at the lack of responses from the group.  A very
special thanks to Dana Hudes.

1)  Installed latest Solaris 9 Cluster

2)  Installed latest Veritas Storage Cluster 4.1 patches

3)  Installed latest JFCA driver
     114878-10

4)  Modified the /kernel/drv/jfca.conf
    -- Turned off topology auto-negotiation and forced 2GB
         FcLinkSpeed = 2
    -- Forced private loop
         FcLoopEnabled   = 1;
    -- Turned off the EngHeartBeat check.
        FcEngHeartbeatInterval = 0;
    -- Reduced the Port Discovery time from 1sec to 500ms
        FcLinkUpRecoveryTime = 500;

5)  Removed the vxfencoorddg, vxdiskunsetup on all the fencing disks

6)  modified the vxfenmode to use scsi-3
     scsi3_disk_policy=dmp
     vxfen_mode=scsi3

7)  modified the main.cf  in the cluster <clustname> declaration
       UseFence = SCSI3

8) Recreated the vxfencoorddg with the three disks

9) vxdg -t deport vxfencoorddg

10) restarted the cluster and now things are working like a charm.  There
still appears a little latency with the jfca driver passing upstream that
there is a pathway problem, but it's about 1-2 secs then dmp disables the
pathway.  For now it works.

If anyone has any thoughts on the latency of the jfca driver reporting the
path failure upstream I would love to hear about it.

Thanks,
Kris--

-----Original Message-----
From: sunmanagers-bounces@sunmanagers.org
[mailto:sunmanagers-bounces@sunmanagers.org] On Behalf Of Kristopher
Briscoe
Sent: Tuesday, September 26, 2006 12:59 PM
To: sunmanagers
Subject: dmp and 3510fc troubles

All,

Having a problem with DMP not disabling a controller once the pathway is
gone.  To simulate a controller failures I merely unplug the cable
providing
the secondary pathway to the array.  I see errors from the sd and jfca
driver, but I never see the "dmp disable " message until AFTER I plug
the
cable back in.   While the cable is unplugged the following return
"positive" results

vxdisk path --> shows all pathways enabled
vxdmpadm getsubpaths dmpnodename=<one of the dmp devices> --> shows all
pathways "ENABLED".

Now, after plugging the cable back in I immediately see the "dmp
disabled"
for the pathway message, and if i run the two commands again I now see
the
pathways "DISABLED".  After a few seconds dmp reenables the paths and
all is
fine.


Sun V440
2 dual channel JNI/JFCA cards
   Patch: 114878-09

Sun StorEdge 3510FC
  1 logical volume with 3 partitions
    2 for IO Fencing
    1 for Oracle data

Solaris 9  Generic_117171-07
All VERITAS components are 4.0
  VERITAS Storage Foundation for Oracle RAC
  VERITAS Volume Manager
  VERITAS Storage Foundation for Oracle
  VERITAS File System
  VERITAS Volume Manager
  VERITAS Cluster Server
  VERITAS Mapping Services


An interesting note here.  When NODE(1) is master and I unplug the cable
on
it vxconfigd dies thus causing the cluster monitoring to die.  vxconfigd
isn't started again until I reconnect the cable.

Anyone out there have any similar experiences?   My thought is that the
jfca
driver isn't letting dmp know that the pathway is gone...it's as if the
driver gets locked up.  I'm going to install rev 10 of the JNI/JFCA
patch
and see if that helps.

Please shout if you have any suggestions.

Kris--


--
Kristopher T. Briscoe
Sr. UNIX Engineer
404.353.3223

Unix is very simple, but it takes a genius to understand the simplicity.
(Dennis Ritchie)
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers


-- 
Kristopher T. Briscoe
Sr. UNIX Engineer
404.353.3223

Unix is very simple, but it takes a genius to understand the simplicity.
(Dennis Ritchie)

-- 
Kristopher T. Briscoe
Sr. UNIX Engineer
404.353.3223

Unix is very simple, but it takes a genius to understand the simplicity.
(Dennis Ritchie)
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers