Gentoo, ReadyNAS, and iSCSI

I recently bought a Netgear ReadyNAS 104 while trying to recover from a failure of my old RAID-5 enclosure.  Now that it's all put together, it looks like I can pull a maximum of 50 MB/s, with typical speeds of 30-40 MB/s.  I'm not sure what the limiting factor is at this point, since it doesn't seem to be network bandwidth, or CPU on either end.  I think something in the system is latency bound, since it seems to pull a little faster when my desktop CPU is completely idle.

At first I was going to set it up using sshfs, but the little ARM CPU in that thing can only push about 3-4 MB/s when it has to do the encryption itself.  So what I finally settled on was exporting it as iSCSI, and having my desktop do the encryption with cryptfs (LUKS).

I get the feeling that iSCSI doesn't get a lot of love on Gentoo, so I figured I'd post my troubles and what I finally got to work.  This post from the Gentoo Wiki Archives was the most helpful, though I skipped all his interface setup, which I'm assuming was designed for a dedicated storage network.

iSCSI Basics:
A target is a server which is offering up a drive for clients to use.
An initiator is the client which consumes a drive to read or write it.

Init Script:
The init script for open-iscsi was kind of primitive... I had to install the unstable version (sys-block/open-iscsi-2.0.872-r2) since it hadn't been updated since modprobe stopped supporting "-l" (which still makes me sad...).  At that point, I had to go redo all my kernel setup since I compiled the iSCSI stuff into my kernel, and the init script assumes that it can manually load and unload the modules.  I eventually commented out all the do_modules calls to work around that.

I also installed net-libs/libiscsi although I never figured out if it was required or not.

I never did figure out how to get it to automatically connect to my drive, but I got the init script to stop complaining at me by setting AUTOSTARTTARGETS="no" in /etc/conf.d/iscsid.  At this point (plus commenting out the modprobes) I could start and stop the daemon cleanly.

To manually mount a drive:

#Start a session to a target
iscsiadm -m discovery -t st -p 192.168.1.100

#Open a drive named "group1"
# on the target "iqn.1994-11.com.netgear:host:a1b2c3d4f"
# (as generated by my NAS) at IP 192.168.1.100
iscsiadm -m node --targetname iqn.1994-11.com.netgear:host:a1b2c3d4f:group1 --portal 192.168.1.100 --login

#At this point /dev/sde should appear and you can mount/format it.

#Disconnect from the target to make the drive disappear 
iscsiadm -m node --targetname iqn.1994-11.com.netgear:host:a1b2c3d4f:group1 --portal 192.168.1.100 --logout  

You can add "-d 8" (with a number 1-8) for increased debug messages if things are going wrong, but I can't say I found it very helpful myself.

The drive letter it shows up at will increment as you keep connecting/disconnecting.  Which was enough to convince me I wanted to manually mount the thing so I've given up trying to get the init script to autostart it anyway.

CHAP:
CHAP is the iSCSI authentication protocol, and was the biggest pain point of the whole experience.  I couldn't get my NAS or open-iscsi to give me any kind of useful error message other than "it didn't work" (or more precisely "iscsiadm: discovery login to 192.168.1.100 rejected: initiator error (02/01), non-retryable, giving up" ).

There are two kinds of CHAP, one used to authenticate initiators (clients), and one used to authenticate the targets (servers).  Since I was already encrypting my data on my desktop, I didn't bother setting up the server authentication, but I think it works basically the same way.  Here's what finally worked:

First I had to generate an InitiatorName and tell my NAS to restrict access to only that initiator.  My NAS only had a field for one "Initiator (IQN)" entry, while the Linux settings had 3 different values.  I don't really know what the difference is supposed to be, but I had problems until I set them all to the same thing.

## -*- mode: Conf-space;-*-
##/etc/iscsi/initiatorname.iscsi
## For examples, see /etc/iscsi/initiatorname.iscsi.example

InitiatorName=iqn.2008-10.com.example.mybox:openiscsi-a1b2c3f
InitiatorAlias=iqn.2008-10.com.example.mybox:openiscsi-a1b2c3f

Uncomment the following lines in /etc/iscsi/iscsid.conf, and set them accordingly.

##/etc/iscsi/iscsid.conf 
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.2008-10.com.example.mybox:openiscsi-a1b2c3f
node.session.auth.password = ThisIsAPassword

For whatever reason, CHAP passwords have to be between 12 and 16 characters.  I spent a while failing to get it to work with an 8-character password before I figured this out.

It probably goes without saying, but make sure you type the same password into your NAS.  (And set it for your initiator, instead of what my NAS called the "bidirectional authentication" for letting clients know they're talking to the right server.)

There are also settings in iscsid.conf for using CHAP during discovery.  You might have to set those too, but my NAS didn't seem to support it, so I left them alone.

After some trial and error, I finally figured out that my NAS was picking up password changes immediately, but my client was saving some kind of a session that was making things difficult.  Eventually I determined that to actually get my new settings to take effect I had to stop my daemon, rm -rf the session folder it had created in /etc/iscsi/nodes/, start the daemon again, and then start over from the discovery phase.

And there you go!  Good luck.