mirror maker - some (not all) topics being produced to in the active/source cluster are showing offsets being skipped, ie. messages 200, 202, 204.

Solution Unverified - Updated -

Issue

  • some (not all) topics being produced to in the active/source cluster are showing offsets being skipped, ie. messages 200, 202, 204. Then when mirrormaker replicates to the other cluster, it rightfully gets produced to that cluster as 200, 201, 202 ... Etc (numbers are just examples).
  • I am also keen to find out how we ensure that we have the correct data at the target site. After deletion/compaction runs at each site the physical numbers in the topics will differ physically as well as logically (offset).
  • We thought that tracking lag at the primary site was going to work, but based on todays observation this doesn't seem reliable.
  • compares the last X messages, and ensures they are the same, or calculate how much REAL lag is present

Environment

  • AMQ Streams
    • 1.2.0

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content