heavy Elasticsearch config queries in tasks-sql

Description

CAPL - Bug default text according to the team DoR (Definition of Ready)

01 - PERSON OF CONTACT (PERSON THAT CAN ANSWER QUESTIONS ABOUT THE PROBLEM):
@henrique.cavarsan
02 - PROBLEM (WHAT'S THE ISSUE?):

It is necessary to optimize the Elasticsearch queries in the config; they are too heavy and affecting the ES latency times.

These queries are executed by the worker-sql in task-sql, and here is the log:
https://cloudlogging.app.goo.gl/nYDeUP8r6t5FQYs58

It would be interesting not to use config-* in these queries if possible.

ES latency in the last 30 days:

03 - STEPS TO REPRODUCE (STEP (1...N), VIDEO, SCREENSHOTS, LOGS FOLDER, HEARTBEAT, ETC. – IF IS NOT POSSIBLE TO REPRODUCE EXPLAIN THE REASON):
04 - LINKS (ADD A LINK TO THE BUG OR TO THE TENANT):
https://cloudlogging.app.goo.gl/nYDeUP8r6t5FQYs58

https://cloudlogging.app.goo.gl/WEV5EAs3h7jZc3CW6
05 - EXPECTED BEHAVIOR (LIST THE EXPECTED BEHAVIORS TO CONSIDER THIS BUG AS DONE):
We need to do this for this card:

  1. We are searching for all TenantApps that matches the CarolAppName and then for each record TenantApp that we found, we are searching the Tenant based on the TenantId of the TenantApp just to find which one of all TenantApps belongs to the Unified Tenant.
  2. We could search the CarolApp document based on the CarolAppName, and inside the CarolApp we will have the mdmUnifiedTenantId which already point us to the Unified Tenant Id.
    // When customer tenant has unified tenants mapping
        Optional<TenantApp> tenantAppOpt =
            tenant.getMdmUnifiedTenantsMapping().keySet().stream()
                .map(
                    tenantUnifiedId -> {
                      try {
                        return getTenantAppService()
                            .getByCarolAppName(
                                new UserAccessDetails(tenantUnifiedId),
                                carolAppName,
                                EntitySpaceType.PRODUCTION,
                                true);
                      } catch (RecordNotFoundException e) {
                        String msg =
                            String.format(
                                ERROR_MSG_GIT_CONFIGURATION_CAROL_APP, carolAppName, tenantUnifiedId);
                        log.warn(msg);
                        return null;
                      }
                    })
                .filter(Objects::nonNull)
                .findFirst();
    
        if (tenantAppOpt.isPresent()) {
          return tenantAppOpt.get();
        }
    
        // When customer tenant doesn't have unified tenants mapping
        List<TenantApp> tenantApps = getTenantAppService().getAllByCarolAppName(carolAppName);
        return getTenantAppService()
            .getUnifiedTenantAppFromList(tenantApps)
            .orElseThrow(
                () -> {
                  String errorMsg =
                      MessageFormat.format(
                          ERROR_MSG_NOT_FOUND_TENANT_UNIFIED_FOR_CAROL_APP, carolAppName);
                  logger.error(errorMsg);
                  return new ApplicationException(Status.BAD_REQUEST, errorMsg);
                });

PS: A busca de CarolApp por name pode ser feita através do método CarolAppServiceImpl.findByName

Activity

Automation for Jira 4 January 2024, 21:02 Jira Internal Users

@henrique.cavarsan ,
@Douglas Coimbra Lopes , @Renan Schroeder , @Douglas Coimbra Lopes , @Gabriel DAmore Marciano

Flag was removed since you have just transitioned the issue status/column.

Automation for Jira 4 January 2024, 20:31 Jira Internal Users

This issue was automatically transitioned to REGRESSION, as its PR was just merged into qa branch in Github.

Automation for Jira 4 January 2024, 20:31 Jira Internal Users

This issue was automatically transitioned to REGRESSION, as its PR was just merged into qa branch in Github.

Automation for Jira 4 January 2024, 20:31 Jira Internal Users

This issue was automatically transitioned to REGRESSION, as its PR was just merged into qa branch in Github.

Automation for Jira 4 January 2024, 20:31 Jira Internal Users

This issue was automatically transitioned to REGRESSION, as its PR was just merged into qa branch in Github.

Automation for Jira 4 January 2024, 20:24 Jira Internal Users

Github user Damore has just commited and issue was sent back to the REVIEW column.

Automation for Jira 4 January 2024, 20:23 Jira Internal Users

This issue was automatically transitioned to WAITING DEPLOY, as its PR was just merged into master branch in Github.

Douglas Coimbra Lopes 4 January 2024, 18:30 Jira Internal Users

@Jonathan Willian Moraes @wilson.souza @Chámam Diomede Caires This card must not be merged into the master branch until the product team approves its tests. It was booked a KT tomorrow with them.

cc @Gabriel DAmore Marciano @Geny Isam Hamud Herrera @Robson Thanael Poffo @MARCOS STUMPF

Automation for Jira 4 January 2024, 18:29 Jira Internal Users

Github user douglascoimbra has just approved a PR (added as Shard Assignee in this Jira issue).

fix: https://totvslabs.atlassian.net/browse/CAPL-5246#icft=CAPL-5246 heavy Elasticsearch config queries in tasks-sql

Automation for Jira 4 January 2024, 18:18 Jira Internal Users

This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.

Douglas Coimbra Lopes 4 January 2024, 17:46 Jira Internal Users

RETESTING

Automation for Jira 4 January 2024, 17:38 Jira Internal Users

This issue was automatically transitioned to REVIEW, as its PR (not DRAFT and not WIP) was just created in Github.

fix: https://totvslabs.atlassian.net/browse/CAPL-5246#icft=CAPL-5246 heavy Elasticsearch config queries in tasks-sql

Douglas Coimbra Lopes 4 January 2024, 14:49 Jira Internal Users

REGRESSION OK for this branch cc @Renan Schroeder @Gabriel DAmore Marciano @Jonathan Willian Moraes

Douglas Coimbra Lopes 4 January 2024, 00:13 Jira Internal Users

ORCHESTRATOR TASK RUNNING GENERIC OPTIMIZATION

NO ERRORS DETECTED

Automation for Jira 2 January 2024, 19:56 Jira Internal Users

@henrique.cavarsan ,
@Geny Isam Hamud Herrera , @Renan Schroeder ,

This issue was planned to be delivered until 2024-01-22. You can check that by consulting the issue in the Due Date field.

Dates already planned for this issue: 2024-01-22

If External Issue Link field is filled, customer was also informed on JIRA TOTVS.