Issue

• We need to pvmove some clustered volumes onto new storage, and thus need to enable cmirrord. How can we do this without restarting clvm, GFS2 Filesystem resources, and other dependents?
• If I update my clvm resource to have with_cmirrord, I get errors citing "unimplemented feature" and my resources are stopped.
• clvm seems to fail if you update its configuration on the-fly, referencing error 3

Jul 22 12:05:05 node1 pengine[19033]:  notice: Reload  clvmd:0#011(Started node2)
Jul 22 12:05:05 node1 pengine[19033]:  notice: Reload  clvmd:1#011(Started node1)
Jul 22 12:05:05 node1 crmd[19034]:  notice: Initiating action 10: reload clvmd_reload_0 on node2
Jul 22 12:05:05 node1 crmd[19034]:  notice: Initiating action 11: reload clvmd_reload_0 on node1 (local)
Jul 22 12:05:05 node1 pengine[19033]:  notice: Calculated Transition 3278: /var/lib/pacemaker/pengine/pe-input-628.bz2
Jul 22 12:05:05 node1 crmd[19034]: warning: Action 10 (clvmd_reload_0) on node2 failed (target: 0 vs. rc: 3): Error
Jul 22 12:05:05 node1 crmd[19034]:  notice: Transition aborted by clvmd_start_0 'modify' on node2: Event failed (magic=0:3;10:3278:0:f10202f4-55f3-48bb-8de6-dee0706281f6, cib=0.290.1, source=match_graph_event:381, 0)
Jul 22 12:05:05 node1 crmd[19034]: warning: Action 10 (clvmd_reload_0) on node2 failed (target: 0 vs. rc: 3): Error
Jul 22 12:05:05 node1 crmd[19034]:  notice: Transition aborted by status-2-fail-count-clvmd, fail-count-clvmd=INFINITY: Transient attribute change (create cib=0.290.2, source=abort_unless_down:319, path=/cib/status/node_state[@id='2']/transient_attributes[@id='2']/instance_attributes[@id='status-2'], 0)
Jul 22 12:05:05 node1 crmd[19034]:  notice: Operation clvmd_reload_0: unimplemented feature (node=node1, call=72, rc=3, cib-update=3351, confirmed=true)
Jul 22 12:05:05 node1 crmd[19034]:  notice: node1-clvmd_reload_0:72 [ usage: /usr/lib/ocf/resource.d/heartbeat/clvm {start|stop|monitor|validate-all|meta-data}\n\nExpects to have a fully populated OCF RA-compliant environment set.\n ]
Jul 22 12:05:05 node1 crmd[19034]: warning: Action 11 (clvmd_reload_0) on node1 failed (target: 0 vs. rc: 3): Error
Jul 22 12:05:05 node1 crmd[19034]: warning: Action 11 (clvmd_reload_0) on node1 failed (target: 0 vs. rc: 3): Error
Jul 22 12:05:05 node1 crmd[19034]:  notice: Transition 3278 (Complete=2, Pending=0, Fired=0, Skipped=0, Incomplete=2, Source=/var/lib/pacemaker/pengine/pe-input-628.bz2): Complete
Jul 22 12:05:05 node1 pengine[19033]: warning: Processing failed op start for clvmd:0 on node2: unimplemented feature (3)
Jul 22 12:05:05 node1 pengine[19033]:   error: Preventing clvmd-clone from re-starting on node2: operation start failed 'unimplemented feature' (3)
Jul 22 12:05:05 node1 pengine[19033]: warning: Processing failed op start for clvmd:0 on node2: unimplemented feature (3)
Jul 22 12:05:05 node1 pengine[19033]:   error: Preventing clvmd-clone from re-starting on node2: operation start failed 'unimplemented feature' (3)
Jul 22 12:05:05 node1 pengine[19033]: warning: Processing failed op start for clvmd:1 on node1: unimplemented feature (3)
Jul 22 12:05:05 node1 pengine[19033]:   error: Preventing clvmd-clone from re-starting on node1: operation start failed 'unimplemented feature' (3)
Jul 22 12:05:05 node1 pengine[19033]: warning: Processing failed op start for clvmd:1 on node1: unimplemented feature (3)
Jul 22 12:05:05 node1 pengine[19033]:   error: Preventing clvmd-clone from re-starting on node1: operation start failed 'unimplemented feature' (3)
Jul 22 12:05:05 node1 pengine[19033]: warning: Forcing clvmd-clone away from node1 after 1000000 failures (max=1000000)
Jul 22 12:05:05 node1 pengine[19033]: warning: Forcing clvmd-clone away from node1 after 1000000 failures (max=1000000)
Jul 22 12:05:05 node1 pengine[19033]: warning: Forcing clvmd-clone away from node2 after 1000000 failures (max=1000000)
Jul 22 12:05:05 node1 pengine[19033]: warning: Forcing clvmd-clone away from node2 after 1000000 failures (max=1000000)
Jul 22 12:05:05 node1 pengine[19033]:  notice: Stop    clvmd:0#011(node2)
Jul 22 12:05:05 node1 pengine[19033]:  notice: Stop    clvmd:1#011(node1)

# pcs status
[...]
 Clone Set: clvmd-clone [clvmd]
     Stopped: [ node1 node2 ]

Failed Actions:
* clvmd_start_0 on node2 'unimplemented feature' (3): call=72, status=complete, exitreason='none',
    last-rc-change='Fri Jul 22 12:05:06 2016', queued=0ms, exec=11ms
* clvmd_start_0 on node1 'unimplemented feature' (3): call=72, status=complete, exitreason='none',
    last-rc-change='Fri Jul 22 12:05:05 2016', queued=0ms, exec=37ms