Skip to main content

sdac-detect-duplicates

Purpose​

Finds duplicate salary or fringe values (consecutive or overall) to highlight potential copy‑paste errors.

Tool type​

Deterministic (rule-based)

Tool ID​

sdac-detect-duplicates

Inputs​

  • report_id (string)
  • field ("salary" | "fringe" | "both")
  • consecutive_only (boolean, optional, default: true)
  • min_occurrences (number, optional, default: 2)
  • include_copy_paste_analysis (boolean, optional, default: false)
  • min_streak_length (number, optional, default: 3)

Outputs​

  • Duplicate groups with affected record IDs and row numbers
  • Summary and recommendation

Example output​

{
  "success": true,
  "report_id": "SDAC-2024-Q2-001",
  "duplicate_groups": [
    {
      "group_id": "name-dob-001",
      "record_ids": [
        501,
        502
      ],
      "row_numbers": [
        38,
        39
      ],
      "match_reason": "Same name and identical claimable costs",
      "severity": "WARNING"
    }
  ],
  "summary": {
    "total_groups": 1,
    "affected_records": 2
  },
  "recommendation": "Review duplicate groups before validating the report."
}

Functional details​

  • Detects duplicate values by exact numeric match.
  • Flags long consecutive streaks for copy/paste risk scoring.
  • Adds Excel-style references for faster reviewer follow-up.

Implementation details​

  • Queries SDAC.data_PersonnelRecords via executeSql for salary/fringe values.
  • Groups by exact numeric value and checks for consecutive rows.
  • Respects consecutive_only and min_occurrences thresholds.
  • Ignores nulls and non-positive values during grouping.

Code Location: packages/domain-sdac/src/tools/detect-duplicates.tool.ts