39.2.2. replacement_timeout

replacement_timeout controls how long the iSCSI layer should wait for a timed-out path/session to reestablish itself before failing any commands on it. The default replacement_timeout value is 120 seconds.
To adjust replacement_timeout, open /etc/iscsi/iscsid.conf and edit the following line: 
node.session.timeo.replacement_timeout = [replacement_timeout]
The 1 queue_if_no_path option in /etc/multipath.conf sets iSCSI timers to immediately defer commands to the multipath layer (refer to Section 39.2, “iSCSI Settings With dm-multipath”). This setting prevents I/O errors from propagating to the application; because of this, you can set replacement_timeout to 15-20 seconds.
By configuring a lower replacement_timeout, I/O is quickly sent to a new path and executed (in the event of a NOP-Out timeout) while the iSCSI layer attempts to re-establish the failed path/session. If all paths time out, then the multipath and device mapper layer will internally queue I/O based on the settings in /etc/multipath.conf instead of /etc/iscsi/iscsid.conf.

Important

Whether your considerations are failover speed or security, the recommended value for replacement_timeout will depend on other factors. These factors include the network, target, and system workload. As such, it is recommended that you thoroughly test any new configurations to replacements_timeout before applying it to a mission-critical system.