# Quantum QXS FC Passthrough — Handoff Notes

**Goal:** Decommission a Quantum QXS storage array (12 drives) and its Fibre Channel cards.
Step 1 (in progress): identify which FC card connects to the QXS, pass it through to a
Fedora VM, and verify/test the drives before tearing everything out.

**Date started:** 2026-06-11

---

## Systems

| Role | Host | Notes |
|------|------|-------|
| Proxmox host | `honeybun` | Debian trixie, Dell box, Intel CPU (VT-d), Proxmox 6.x kernel |
| Test VM | Fedora 44 (`localhost`, `192.168.3.191`) | login user `wfmz`, cockpit on :9090 |
| Storage | Quantum QXS, firmware `G22x` | Seagate/Dot Hill AssuredSAN-based, dual-controller, 12 drives |

---

## Hardware identified

Two **QLogic QLE2562** dual-port 8Gb FC HBAs (chipset ISP2532, PCI ID `1077:2532`,
subsystem `1077:015d`), driver `qla2xxx`. **4 FC ports total.**

On the Proxmox host (`lspci`):

| PCI addr | Card | Host-side fc_host | WWPN | State |
|----------|------|-------------------|------|-------|
| `09:00.0` | Card #1 | host7 | `21:00:00:24:ff:4e:a1:c2` | Linkdown (not connected) |
| `09:00.1` | Card #1 | host8 | `21:00:00:24:ff:4e:a1:c3` | Linkdown (not connected) |
| `0a:00.0` | **Card #2** | host9 | `21:00:00:24:ff:50:36:b4` | **Online, 8 Gbit, Point-to-Point** |
| `0a:00.1` | **Card #2** | host10 | `21:00:00:24:ff:50:36:b5` | **Online, 8 Gbit, Point-to-Point** |

### Conclusion: the card connected to the QXS is `0a:00` (Card #2)
Both its ports are Online and **direct point-to-point** (no FC switch — `fabric_name = 0xffff…`),
one port to each QXS controller. Card #1 (`09:00`) is connected to nothing.

IOMMU group for the passthrough card:
- `0a:00.0` and `0a:00.1` → **IOMMU group 26** (clean, both functions together)
- IOMMU (Intel VT-d) is **already active** by default — no kernel cmdline edit was needed
  (the `/sys/.../iommu_group` symlinks exist and are populated).

---

## Passthrough config (Proxmox)

VM must be **q35** machine type for PCIe passthrough (i440fx has no PCIe slots).

In `/etc/pve/qemu-server/<vmid>.conf`:
```
machine: q35
hostpci0: 0000:0a:00,pcie=1
```
(Omitting the function number passes the whole card / both functions. GUI equivalent:
Hardware → Add → PCI Device → `0000:0a:00.0`, tick **All Functions** + **PCI-Express**.)

When the VM starts, Proxmox binds the card to `vfio-pci` automatically (pulls it off the
host's `qla2xxx`). To confirm on the host while the VM runs:
```bash
lspci -nnk -s 0a:00     # should show "Kernel driver in use: vfio-pci"
```

---

## Status inside the Fedora VM — CONFIRMED WORKING

- Card visible: `01:00.0` / `01:00.1` (`1077:2532`), `qla2xxx` bound.
- **Both ports Online @ 8 Gbit**, WWPNs match the host's `0a:00` card
  (`…5036b4`, `…5036b5`). fc_host = host7 (pg0/ctrl A) and host8 (pg1/ctrl B).
- Array enumerates on each path:
  - LUN 0 = **Enclosure** (SES, type 13) → `/dev/sg2` (path A), `/dev/sg6` (path B)
  - LUN 2, 3, 4 = **3 Direct-Access volumes**, each seen via both controllers (multipath)
- **ALUA active** (`supports implicit TPGS`).

### The 3 volumes (each has 2 paths)

| Volume NAA / WWN | Path A (host7, pg0) | Path B (host8, pg1) |
|------------------|---------------------|---------------------|
| `…600c0ff000291d8ad7…` | sdb (7:0:0:2) | sde (8:0:0:2) |
| `…600c0ff000291d8adb…` | sdc (7:0:0:3) | sdf (8:0:0:3) |
| `…600c0ff000291d8ae3…` | sdd (7:0:0:4) | sdg (8:0:0:4) |

---

## CURRENT BLOCKER

All volumes show **0B** in `lsblk`, and **READ CAPACITY fails with "Device not ready"**
(`sg_readcap /dev/sdb`). `sg_inq` succeeds (LUN responds; Vendor=Quantum, Product=QXS,
rev G22x), so the LUNs are mapped and reachable — they just won't report capacity yet.

Most likely cause (one of):
1. **ALUA non-owning path** — `sdb` (controller A) may be the non-optimized/standby path
   for that volume; the owning path (`sde`, controller B) may read fine. Fix = multipath.
2. **Volume not-ready on the array** — disk group faulted / volume offline / disabled on
   the QXS. Fix = QXS management GUI (array-side).

> Note: `sg_turs -v` printed "bad pass-through setup" — that's just sg_turs being fussy
> with `-v`; run it WITHOUT `-v`.

---

## NEXT STEPS (run in the Fedora VM)

### 1. Determine which path is live (ALUA) — this decides everything
```bash
for d in sdb sdc sdd sde sdf sdg; do
  printf "%s state=%s access=%s\n" "$d" \
    "$(cat /sys/block/$d/device/state)" \
    "$(cat /sys/block/$d/device/access_state 2>/dev/null)"
done

sudo sg_turs /dev/sdb           # no -v
sudo sg_readcap /dev/sdb        # controller A path
sudo sg_readcap /dev/sde        # controller B path, SAME volume (…d7…)
sudo sg_rtpg -d /dev/sdb        # which port group is active/optimized
```
- One path active/optimized + returns real capacity → **just ALUA**, enable multipath.
- All paths standby/unavailable + both readcaps fail → **volumes offline on the array**,
  fix in QXS management.

### 2. Enable multipath (handles ALUA routing automatically)
```bash
sudo dnf install -y device-mapper-multipath
sudo mpathconf --enable --with_multipathd y
sudo systemctl enable --now multipathd
sudo multipath -ll              # expect 3 mpath devices, 2 paths each
lsblk /dev/mapper/mpath*        # should now show real sizes
```

### 3. Rescan if needed (e.g. after fixing mapping on the array)
```bash
for h in /sys/class/scsi_host/host*/scan; do echo "- - -" | sudo tee $h; done
for d in /sys/block/sd[b-g]/device/rescan; do echo 1 | sudo tee $d; done
```

### 4. Inspect the 12 physical drives via SES enclosure
The host only sees the 3 RAID **volumes**, NOT the 12 raw disks. For per-drive status:
```bash
sudo sg_ses --page=2 /dev/sg2   # enclosure status: per-slot drive state
sudo sg_ses -p cf /dev/sg2      # element descriptors (drive slots)
```
Full drive-level testing (SMART, media scan) is done from the **QXS management GUI**
(controller management IP). **Open question: do we have admin login to the QXS web UI?**

---

## Useful reference commands

Identify FC cards on any Linux host:
```bash
lspci -nn | grep -i fibre
lspci -nnk -s <slot>                 # driver in use
ls /sys/class/fc_host/
cat /sys/class/fc_host/host*/port_state
cat /sys/class/fc_host/host*/port_name
systool -c fc_host -v                 # pkg: sysfsutils  (clean per-HBA dump)
```

---

## Open questions / TODO
- [ ] Run the ALUA access-state check (step 1) → confirm ALUA path vs. offline volume.
- [ ] Do we have admin access to the QXS web management GUI?
- [ ] After verifying/testing the 3 volumes + 12 drives → safe teardown of array + both
      FC cards.

## Security note (this session)
A pasted command attempted to add a yum repo from `pkg.claude-desktop-debian.dev`. This is
**NOT an official Anthropic domain** — Anthropic ships Claude Desktop for macOS/Windows only,
no official Linux RPM/yum repo exists. Do not add that repo. Official Linux options:
Claude Code CLI (`npm i -g @anthropic-ai/claude-code`) or claude.ai in a browser.