Failed to create tasks of copy data - Unificados laketechfin

Description

PRDE - Bug default text according to the team DoR (Definition of Ready)

01 - PERSON OF CONTACT (PERSON THAT CAN ANSWER QUESTIONS ABOUT THE PROBLEM):

@Breno Zipoli Monteiro Papa , @Gabriel DAmore Marciano
02 - PROBLEM (WHAT'S THE ISSUE?):

Instalações iniciadas via script estão gerando erro Failed to create tasks of copy data e não criando tasks de copy data nos unificados.

Histórico/Resumo da thread: https://totvscarol.slack.com/archives/C0330CSV2/p1700863206548689

Foram identificados logs de Failed to create tasks of copy data no GCP na data de ontem, onde a cópia de dados para o unificado não era feita (task não era criada).
Após investigações descobrimos que esses erros são provenientes de ações de instalação de app feita via script, chamando os endpoints subscribe e install.
Após mais investigações, descobrimos que os erros são gerados pois o código de criação da task valida o connector_id utilizado na autenticação da chamada, que é diferente para a tenant que instalou o app e para a tenant unificada, gerando o erro. Via UI/Swagger é usado o connector_id global da plataforma, não gerando erro.

03 - STEPS TO REPRODUCE (STEP (1...N), VIDEO, SCREENSHOTS, LOGS FOLDER, HEARTBEAT, ETC. – IF IS NOT POSSIBLE TO REPRODUCE EXPLAIN THE REASON):

Use um connector e um token gerado na plataforma para instalar um app com unificado. Abaixo o código.

import requests, json

connector = 'xxx'
token = 'xxx'

subscribe = requests.post(
            f'https://totvstechfin.carol.ai/api/v1/tenantApps/subscribe/carolApps/760ba29e677c4a6087aaebd0669a45ed',
                headers = { "Content-Type": "application/json", "X-Auth-Key": token, "X-Auth-ConnectorId": connector })

install = requests.post(
            f'https://totvstechfin.carol.ai/api/v1/tenantApps/{json.loads(subscribe.text)["mdmId"]}/install?publish=true',
                headers = { "Content-Type": "application/json", "X-Auth-Key": token, "X-Auth-ConnectorId": connector })
print(install)

Será instalado o app normalmente, mas não será criada task de copy data. No GCP haverá o log como abaixo:

https://cloudlogging.app.goo.gl/qd7qpomNEYjcGwtA9
04 - LINKS (ADD A LINK TO THE BUG OR TO THE TENANT):
05 - EXPECTED BEHAVIOR (LIST THE EXPECTED BEHAVIORS TO CONSIDER THIS BUG AS DONE):

  • For copy data tasks. we’re going to inject the global connector for the task creation, since it is an internal task created during a carol app installation and should not follow the existing validation of connector and tenant being requested to create the task.

Activity

Automation for Jira 1 December 2023, 00:08 Jira Internal Users

This issue was automatically transitioned to DONE after being deployed to production on Github.

https://github.com/totvslabs/mdm/releases/tag/v4.89.3

Automation for Jira 1 December 2023, 00:08 Jira Internal Users

No associated issue on JIRA TOTVS BRAZIL.

Automation for Jira 30 November 2023, 23:21 Jira Internal Users

This issue was automatically transitioned to WAITING DEPLOY, as its PR was just merged into master branch in Github.

Automation for Jira 30 November 2023, 23:21 Jira Internal Users

This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.

Automation for Jira 30 November 2023, 21:51 Jira Internal Users

Github user reinaldomjr has just commited and issue was sent back to the REVIEW column.

Automation for Jira 30 November 2023, 20:38 Jira Internal Users

Github user rfschroeder has just approved a PR (added as Shard Assignee in this Jira issue).

fix: CAPL-5091 Done Set connectorId to GLOBAL on copyData Tasks

Automation for Jira 30 November 2023, 20:37 Jira Internal Users

Github user douglascoimbra has just approved a PR (added as Shard Assignee in this Jira issue).

fix: CAPL-5091 Done Set connectorId to GLOBAL on copyData Tasks

Automation for Jira 30 November 2023, 20:36 Jira Internal Users

This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.

Automation for Jira 30 November 2023, 20:35 Jira Internal Users

Github user Damore has just commited and issue was sent back to the REVIEW column.

Douglas Coimbra Lopes 30 November 2023, 20:30 Jira Internal Users

CONNECTOR TOKEN CREATED FOR Oauth

CUSTOMER SUBSCRIBABLE /SUBSCRIBE / INSTAL CAROLAPPS

APP INSTALLED ON CUSTOMER

Automation for Jira 30 November 2023, 20:21 Jira Internal Users

This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.

Automation for Jira 30 November 2023, 19:15 Jira Internal Users

This issue was automatically transitioned to REVIEW, as its PR (not DRAFT and not WIP) was just created in Github.

fix: CAPL-5091 Done Set connectorId to GLOBAL on copyData Tasks

Automation for Jira 29 November 2023, 14:26 Jira Internal Users

@MARCOS STUMPF ,

@Geny Isam Hamud Herrera ,
This issue was planned to be delivered until 2023-12-11. You can check that by consulting the issue in the Due Date field.

Dates already planned for this issue: 2023-12-11

If External Issue Link field is filled, customer was also informed on JIRA TOTVS.

Automation for Jira 29 November 2023, 14:26 Jira Internal Users

Message thread link on #red-phone channel:

https://totvscarol.slack.com/archives/C03NT4US9J9/p1701264386532029