Accessing HBase through JBoss DV results in "org.apache.hadoop.hbase.DoNotRetryIOException" with "isDelete failed"

Solution Unverified - Updated -

Issue

  • Query against HBase source results in exception similar to below:
12:08:43,203 WARN  [org.teiid.PROCESSOR] (http-0.0.0.0:8080-2) TEIID16053 Unable to process odata request due to: TEIID30504 hbase: 08000 Unexpected exception while translating results: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: ... : ... : isDelete failed: deleteBuffer=..., qualifier=..., timestamp=1517542198190, comparison result: 1
        at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:87)
        at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:53)
        at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$2.nextRaw(BaseScannerRegionObserver.java:446)
        at org.apache.phoenix.coprocessor.DelegateRegionScanner.nextRaw(DelegateRegionScanner.java:77)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2506)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32385)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
Caused by: java.lang.IllegalStateException: isDelete failed: deleteBuffer=..., qualifier=..., timestamp=1517542198190, comparison result: 1
        at org.apache.hadoop.hbase.regionserver.ScanDeleteTracker.isDeleted(ScanDeleteTracker.java:146)
        at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:386)
        at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:545)
        at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:147)
        at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:5856)
        at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:6007)
        at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:5794)
        at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$2.nextRaw(BaseScannerRegionObserver.java:416)
        ... 7 more

Environment

  • Red Hat JBoss Data Virtualization (DV) 6.3
  • Apache HBase accessed through Phoenix

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content