Issue Type Icon CAPL-4790

SQL Processing: user integration with tenant user without depending on merges (RT)

Description

PRDE - Bug default text according to the team DoR (Definition of Ready)

01 - PERSON OF CONTACT (PERSON THAT CAN ANSWER QUESTIONS ABOUT THE PROBLEM):

02 - PROBLEM (WHAT'S THE ISSUE?):

@Cindy de Araujo Soares Moore described:

There is a specific scenario that we didn’t consider when discussing this ticket. On User data model creation we should always create the mappings, even for disabled RT environments.

When we process records to mdmuser data model, we depend on RT to create the Carol Platform user.

We will need this change in EntityTemplate class: https://github.com/totvslabs/mdm/pull/3515/files#diff-c06d96dd86e76637c7cd0aa7676ecd5d308f62d7343524e287df2ec5c215a22bR713-R718

On:

CAPL-3197:Validate strategies to avoid creating the ES "schema" (mapping) on ES for sta...Done

03 - STEPS TO REPRODUCE (STEP (1...N), VIDEO, SCREENSHOTS, LOGS FOLDER, HEARTBEAT, ETC. – IF IS NOT POSSIBLE TO REPRODUCE EXPLAIN THE REASON):
04 - LINKS (ADD A LINK TO THE BUG OR TO THE TENANT):
05 - EXPECTED BEHAVIOR (LIST THE EXPECTED BEHAVIORS TO CONSIDER THIS BUG AS DONE):

  • SQL Pipeline processing user integrated with Tenant User must not depend on RT storage layer and should not depend on merges for the integration with Tenant User to happen.
    • DO NOT force userGolden to go to ES. This should respect the storage type defined on the DM and the flag sendToRealTime from pipeline.
    • REMOVE the force ES mapping creation AND REMOVE the force of realtime storage type. ( CAPL-4022 Done )
  • Carol App installing the DM mdmUser should follow the storage types defined on Carol App, DO NOT making the RT storage layer enabled by default by Carol.
  • Pipeline is responsible to return the data as unique. So, If on the golden record the source is sql_processing no EXACT_MERGE task should be created. Otherwise, the EXACT_MERGE should be triggered and the Carol user creation will follow the old flow.
  • If RT storage type is disabled for DM we MUST NOT create the MERGE TASK.
  • Please validate any hard code reference to mdmUser.
  • The password must not be store on ES or BQ storage type. In other words, all golden records must not store password.
    • If for some reason it is needed to create the user, we need to remove it right after creating the user.
    • We will handle only at golden level.
  • As aligned (Robson and Geny) we should consider the rollout of this feature through feature flag on Organization Level.

NOT ON THIS AC:

  • UNIFIED FLOW. Will be covered on this card
    CAPL-4814:SQL Processing: user integration with tenant user without depending on merges...Done

Attachments

image-20240114-...
99.8 KiB
image-20240114-...
469.7 KiB
image-20240114-...
277.0 KiB
image-20240114-...
465.0 KiB
image-20240114-...
375.6 KiB
image-20240114-...
179.8 KiB
image-20240114-...
158.7 KiB
image-20240114-...
162.3 KiB
image-20240114-...
370.3 KiB
image-20240114-...
389.4 KiB
image-20240114-...
162.1 KiB
image-20240114-...
608.4 KiB
image-20240114-...
114.0 KiB
image-20240114-...
131.3 KiB
image-20240114-...
176.5 KiB
image-20240114-...
212.5 KiB
image-20240114-...
234.0 KiB
image-20240114-...
367.8 KiB
image-20240114-...
152.4 KiB
image-20240114-...
1.1 MiB
image-20240114-...
152.0 KiB
image-20240114-...
206.0 KiB
image-20240114-...
1004.2 KiB
image-20240114-...
156.4 KiB
image-20240114-...
867.6 KiB
image-20240114-...
361.5 KiB
image-20240114-...
759.3 KiB
image-20240114-...
153.9 KiB
image-20240114-...
183.8 KiB
image-20240114-...
177.0 KiB
image-20240114-...
364.9 KiB
image-20240114-...
364.8 KiB
image-20240114-...
148.4 KiB
image-20240114-...
697.2 KiB
image-20240114-...
157.7 KiB
image-20240114-...
832.4 KiB
image-20240114-...
208.0 KiB
image-20240114-...
364.8 KiB
image-20240114-...
1.2 MiB
image-20240114-...
366.4 KiB
image-20240114-...
207.0 KiB
image-20240114-...
207.8 KiB
image-20240114-...
1.0 MiB
image-20240114-...
366.4 KiB
image-20240114-...
111.9 KiB
image-20240114-...
720.9 KiB
image-20240114-...
366.9 KiB
image-20240114-...
276.7 KiB
image-20240114-...
267.3 KiB
image-20240114-...
171.8 KiB
image-20240114-...
883.4 KiB
image-20240114-...
363.3 KiB
image-20240114-...
436.7 KiB
image-20240114-...
454.3 KiB
image-20240114-...
302.3 KiB
image-20240114-...
451.2 KiB
image-20240114-...
441.8 KiB
image-20240114-...
305.3 KiB
image-20240114-...
424.8 KiB
image-20240114-...
434.6 KiB
image-20240114-...
452.0 KiB
image-20240114-...
368.7 KiB
image-20240114-...
454.0 KiB
image-20240114-...
433.4 KiB
image-20240114-...
338.7 KiB
image-20240114-...
437.5 KiB
image-20240114-...
452.8 KiB
image-20240114-...
459.7 KiB
image-20240114-...
452.7 KiB
image-20240114-...
289.5 KiB
image-20240114-...
457.0 KiB
image-20240114-...
435.1 KiB
image-20231227-...
143.8 KiB
image-20231227-...
339.7 KiB
image-20231227-...
383.2 KiB
image-20231227-...
306.6 KiB
image-20231219-...
175.4 KiB
image-20231219-...
93.3 KiB
image-20231219-...
158.2 KiB
image-20231219-...
215.1 KiB
image-20231218-...
80.8 KiB
image-20231218-...
269.4 KiB
image-20231215-...
807.3 KiB
image-20231215-...
115.7 KiB
image-20231215-...
63.7 KiB
image-20231215-...
115.3 KiB
image-20231215-...
58.2 KiB
image-20231214-...
123.2 KiB
image-20231214-...
227.8 KiB
image-20231214-...
526.7 KiB
image-20231214-...
1.3 MiB
image-20231214-...
1.6 MiB
image-20231214-...
202.8 KiB
image-20231214-...
146.0 KiB
image-20231212-...
131.7 KiB
image-20231212-...
1.7 MiB
image-20231212-...
261.9 KiB
image-20231212-...
173.1 KiB
image-20230614-...
223.0 KiB
Done

Details

Priority

Priority: HighestHighest

Assignee

Geny Isam Hamud Herrera

Reporter

MARCOS STUMPF

Labels

CAROL_BACKLOG_REVIEWDEMOPLANNEDPROD_IMPACTREADYREVIEWED

Due Date

22 January 2024

Fix versions

CAPL_3.98

Components

SQL_PIPELINES | DATA_PROCESSING

Created

4 October 2023, 15:43

Updated

23 January 2024, 18:45

More fields

Created: 4 October 2023, 20:43 Updated: 23 January 2024, 22:45