Generic SQL Processing Efficiency stopped to work for a Gesplan tenant
Description
PRDE - Bug default text according to the team DoR (Definition of Ready)
01 - PERSON OF CONTACT (PERSON THAT CAN ANSWER QUESTIONS ABOUT THE PROBLEM):
02 - PROBLEM (WHAT'S THE ISSUE?):
The efficiency was working as expected by 3:17 pm EST on this tenant:
Last task with efficiency:
Task without efficiency:
Scroll down:
Filtering all tasks but SQL Processing tasks, no special tasks today to justify (app instalation, etc):
Unif tenant has the checkout from 11/21:
Things checked:
- No new data received (for all tasks after 11/22/2023 03:17:26 PM
- No new checkout
- Parameter on tasks are OK: carol app name defined, pipeline name defined, optimization flag enabled.
- There is the pipeline on pipelines' manifest.
- There is no error message on the taskl.
- It seems the task is stuck in a status there is new data, based on this message:
- There is new data in the staging tables to proceed with execution of pipeline contaspagarprevisto in app gesplanprotheuslink since last pipeline execution at 2023-11-21 23:26:45 (UTC) by task cf146f1ddf1c48f49f4263a68e1ad60b. Staging tables: protheus_carol: [se2, ct1, sx5, cto, sev, sa2, ctt, fk7, sed, protheus_sharing, sez]
- We have more pipelines on this tenant with this status:
03 - STEPS TO REPRODUCE (STEP (1...N), VIDEO, SCREENSHOTS, LOGS FOLDER, HEARTBEAT, ETC. – IF IS NOT POSSIBLE TO REPRODUCE EXPLAIN THE REASON):
04 - LINKS (ADD A LINK TO THE BUG OR TO THE TENANT):
05 - EXPECTED BEHAVIOR (LIST THE EXPECTED BEHAVIORS TO CONSIDER THIS BUG AS DONE):
- Understand why the generic optimization got disabled.
- It seems an error, we need to check if it is a misunderstanding on tenant’s configuration.
Activity
Show:
Details
Priority
HighestAssignee
Renan Schroeder
Reporter
MARCOS STUMPF
Labels
RED_PHONE
Due Date
14 December 2023
Fix versions
CAPL_3.96
Components
SQL_PIPELINES | DATA_PROCESSING
Created
23 November 2023, 09:22
Updated
12 December 2023, 19:41
More fields
Created: 23 November 2023, 13:22
Updated:
12 December 2023, 23:41
This issue was automatically transitioned to DONE after being deployed to production on Github.
No associated issue on JIRA TOTVS BRAZIL.
This issue was automatically transitioned to DONE after being deployed to production on Github.
https://github.com/totvslabs/mdm/releases/tag/v4.89.1
No associated issue on JIRA TOTVS BRAZIL.
This issue was automatically transitioned to WAITING DEPLOY, as its PR was just merged into master branch in Github.
Github user douglascoimbra has just approved a PR (added as Shard Assignee in this Jira issue).
fix: Generic SQL Processing Efficiency stopped to work for a Gesplan tenant
This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.
This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.
@Renan Schroeder This card has been validated by the QA team. It is pending only the code review. cc @Gabriel DAmore Marciano
ORCHESTRATOR FLOW PASSED pipeline with 1 & 2 tables for the same datamodel
Scenario encontrado voltado ao ORCHESTRATOR Flow.
• Update topico resolvido: falso. negativo :white_check_mark:
• Uma vez enviado os primeiros registros para tabela e a primeira task eh reprocessada, esta nao esta seguindo a otimizacao generica. A otimizacao generica apenas eh apicada a partir da segunda task.
•
•
Edited on Slack - platform-internal - Douglas Coimbra Lopes
@Renan Schroeder When the first task is executed from the ORCHESTRATOR trigger source, the table has new records, but the generic optimization is not working:
TEST PASSED Generic Optimization Working for 2 stagings vs 1 data model:
TEST PASSED for Unified
:rotating_light: Scenario encontrado voltado ao ORCHESTRATOR Flow.
• Uma vez enviado os primeiros registros para tabela e a primeira task eh reprocessada, esta nao esta seguindo a otimizacao generica. A otimizacao generica apenas eh apicada a partir da segunda task.
•
•
Sent by Slack - platform-internal - Douglas Coimbra Lopes
:rotating_light: Scenario identificado voltado ao
• A tabela product recebeu os dados desde as 15h21, mas nenhuma task reprocessou os dados via otimizacao generica
• sandbox
Sent by Slack - platform-internal - Douglas Coimbra Lopes
:alert: Scenario encontrado no :
• Logo apos a primeira task de ORCHESTRATOR rodar para o datamodel mdmfarhorizon com dados, novos registros foram enviados, porem as task sao criados mas nao os encontram, apresentando a msg abaixo:
• Em suma, intake record enviados dentro da janela de 2mins nao estao entrando no flow de potential records
• `There is no new data in the staging tables to proceed with execution of pipeline newfarhorizon_main in app myapp since last pipeline execution at 2023-11-23 22:14:45 (UTC) by task fc3aae22a53b493e89d11a94301bf336. Staging tables: nlp: [purchaseorder, product]`
• TenantCustomer:
Sent by Slack - platform-internal - Douglas Coimbra Lopes
This issue was automatically transitioned to REVIEW, as its PR (not DRAFT and not WIP) was just created in Github.
fix: Generic SQL Processing Efficiency stopped to work for a Gesplan tenant
This issue was automatically transitioned to REVIEW, as its PR (not DRAFT and not WIP) was just created in Github.
fix: Generic SQL Processing Efficiency stopped to work for a Gesplan tenant
@MARCOS STUMPF ,
@Geny Isam Hamud Herrera ,
This issue was planned to be delivered until 2023-12-11. You can check that by consulting the issue in the Due Date field.
Dates already planned for this issue: 2023-12-11
If External Issue Link field is filled, customer was also informed on JIRA TOTVS.
Message thread link on #red-phone channel:
https://totvscarol.slack.com/archives/C03NT4US9J9/p1700742185662449