[SQL Processing] Fail SQL Processing task when BQ insertion fails in pipeline

Description

PRDE - Story default text according to the team DoR (Definition of Ready)

01 - PERSON OF CONTACT (PERSON THAT CAN ANSWER QUESTIONS ABOUT THE PROBLEM):
02 - PROBLEM (WHAT'S THE CURRENT PROBLEM SCENARIO OR PAIN TO BE RESOLVED?):

When BQ has issues inserting records, the Platform sends a warning to the logs but it is hidden for the user. The SQL Process task is completed successfully, and nothing is being reported in the UI.

It was noticed when we had an expressive number of errors reported in

, but the behavior of the Pipeline related to the BQ insertion didn’t change recently. It aways worked like that. The tasks only fails when errors are found before BQ writes.

03 - GOAL (DESCRIBE THE PROPOSED SOLUTION):

Make sure that if BQ had issues inserting records during SQL Processing, the user will be warned that the task had errors to complete.

04 - WHO CAN USE THIS FEATURE (USER ROLES):
05 - ASSETS (FIGMA LINKS, RELEVANT DOCUMENTATION LINKS, JSON EXAMPLES, ETC):
06 - ACCEPTANCE CRITERIA:

  • SQL Task: fail it.

Activity

Automation for Jira 2 May 2024, 18:51 Jira Internal Users

Esta issue foi automaticamente movida para REGRESSION, pois o PR foi mergeado na branch QA no Github.

Automation for Jira 2 May 2024, 18:51 Jira Internal Users

Esta issue foi automaticamente movida para REGRESSION, pois o PR foi mergeado na branch QA no Github.

Automation for Jira 25 April 2024, 14:37 Jira Internal Users

Esta issue foi automaticamente movida para TESTED & MERGED, pois o PR foi mergeado na branch Develop no Github. PR aprovado por reinaldomjr,douglascoimbra.

Automation for Jira 25 April 2024, 14:36 Jira Internal Users

Github usuário douglascoimbra aprovou um PR e foi adicionado como Shared Assignee nesta issue.

feat: https://totvsideia.atlassian.net/browse/CAPL-5221#icft=CAPL-5221 Falhar a Task de processamento SQL em caso de falhas de escrita no BQ

Automation for Jira 24 April 2024, 15:14 Jira Internal Users

Github usuário cindysoares realizou um commit e issue foi retornada para a coluna REVIEW no Kanban.

Automation for Jira 19 April 2024, 14:21 Jira Internal Users

Esta issue foi automaticamente movimentada para QA REVIEW, pois o PR foi aprovado no Github.

Automation for Jira 15 April 2024, 19:37 Jira Internal Users

@MARCOS STUMPF ,
@Jonathan Willian Moraes , @Cindy de Araujo Soares Moore , @Cindy de Araujo Soares Moore , @Douglas Coimbra Lopes , @Reinaldo Oliveira Machado Junior

Este issue foi planejada para ser entregue até 2024-05-06. Você pode confirmar consultando o campo Due Date desta issue.

Data já planejadas para esta issue: 2024-04-15, 2024-05-06, 2024-03-25

Se o campo External Issue Link estiver preenchido com o link de uma issue válida no Jira Produção o cliente também será notificado no Jira Produção.

Automation for Jira 12 April 2024, 12:28 Jira Internal Users

• Card validado pelo time de QA. Pendente apenas o code review.

Mensagem enviada pelo Slack - plataforma-carol-internal - Douglas Coimbra Lopes

Douglas Coimbra Lopes 12 April 2024, 12:27 Jira Internal Users

SQL PROCESSGING

image-20240412-122454.png

CHECKING THAT ORDER SQL QUERIES ARE WORKING

image-20240412-122738.png
Automation for Jira 25 March 2024, 19:30 Jira Internal Users

@MARCOS STUMPF ,
@Jonathan Willian Moraes , @Cindy de Araujo Soares Moore , @Cindy de Araujo Soares Moore , @Douglas Coimbra Lopes

This issue was planned to be delivered until 2024-04-15. You can check that by consulting the issue in the Due Date field.

Dates already planned for this issue: 2024-04-15, 2024-03-25

If External Issue Link field is filled, customer was also informed on JIRA TOTVS.

Automation for Jira 25 March 2024, 14:00 Jira Internal Users

Card validado pelo time de QA. Pendente apenas o code review

Mensagem enviada pelo Slack - plataforma-carol-internal - Douglas Coimbra Lopes

Douglas Coimbra Lopes 25 March 2024, 13:59 Jira Internal Users

ERROR CORRECTLY BEING PRINTED WHEN DATALFOW TRUE

UNIFIED

Automation for Jira 1 March 2024, 19:18 Jira Internal Users

@MARCOS STUMPF ,
@Pedro Buzzi , @Cindy de Araujo Soares Moore

This issue was planned to be delivered until 2024-03-25. You can check that by consulting the issue in the Due Date field.

Dates already planned for this issue: 2024-03-25

If External Issue Link field is filled, customer was also informed on JIRA TOTVS.