API returns timeout when trying to enable short URL authentication
Description
Contact
- @Bruno Furtado
Problem
When an user tries to update the tenant to enable the short URL authentication, API returns timeout:
- Organization/tenant: icomon/icomon
curl -X 'PUT' \
'https://icomon.carol.ai/api/v3/admin/tenants/f2c73dce2636428688b803a3ef729331' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer TOKEN' \
-d '{ "mdmCdsShortURLAuthentication": true }'
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>502 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>
The timeout happens but the feature is enabled. Maybe that's why we are receiving several error logs.
Even disabling the flag, errors still happening.
Notes
It only happens when tenant was a lot of URLs (we tested is small environments and it works well):
- Organization/tenant: furtado/furtadoclockin
curl -X 'PUT' \
'https://furtado.carol.ai/api/v3/admin/tenants/2510054dfd434767b443e6315315b4d0' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer TOKEN' \
-d '{ "mdmCdsShortURLAuthentication": true }'
{
"success": true
}
Links
- Tenant: https://icomon.carol.ai/icomon/
- Google Logs (warn): https://cloudlogging.app.goo.gl/TgHQya6pyAZP9odN8
- Google Logs (error): https://cloudlogging.app.goo.gl/DggsjLCuLWCaQ19n6
- Slack thread: https://totvscarol.slack.com/archives/C03LA7B048G/p1706133663868609
Regression link:
Expected behavior
- When there are many URLs, the platform must enable authentication without generating errors.
Activity
Show:
A causa do problema foi identificada, sendo que as tasks `CAROL_PIPELINES_EXECUTION_SUMMARY` receberam `ACCESS DENIED` ao lançarem os jobs de BigQuery para calcular a unicidade de registros por Batch Id + Data Model.
O problema ocorreu devido a uma mudança recente de projeto GCP que seria executado a query de consolidação de Summary no BigQuery pela variável de ambiente `BIGQUERY_OBSERVABILITY_PROJECT_ID='labs-techfin-production'`, que entrou em produção no dia 18/03, junto ao deploy do card .
O time de SRE já realizou a adequação de permissão na service account do projeto GCP utilizado em produção (`labs-techfin-production`) e todas as tasks voltaram a executar de forma correta, sem necessidade de intervenção de código e também sem maiores impactos, exceto o atraso da entrega dos eventos de Summary para alguns batchs.
Mensagem enviada pelo Slack - plataforma-red-phone - Rodrigo Bechtold
Olá@Bruno Furtado bom dia, deploy realizado hoje.
Pessoal, entendo que este item deve subir no próximo deploy, correto?
Houve alguma mudança no endpoint (na forma com que chamamos ou recebemos os dados)? Apenas tentando antecipar o tema e orientar o time Clockin.
Obrigado pela atuação e desculpem o incomodo.
This issue was automatically transitioned to TESTED & MERGED, as its PR was just merged into develop branch in Github. PR Approved by glaucioscheibel,douglascoimbra.
Github user douglascoimbra has just approved a PR (added as Shard Assignee in this Jira issue).
feat: https://totvslabs.atlassian.net/browse/CAPL-5419#icft=CAPL-5419 - ShortURL Authentication processed by task with rate limit
This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.
Github user rodrigo-bechtold has just commited and issue was sent back to the REVIEW column.
Olá pessoal, temos uma previsão de data para o deploy dessa alteração?
@Rodrigo Bechtold Regression validated by the QA team.
cc @Gabriel DAmore Marciano @Douglas Coimbra Lopes
This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.
Github user rodrigo-bechtold has just commited and issue was sent back to the REVIEW column.
This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.
@MARCOS STUMPF ,
@Pedro Buzzi , @Rodrigo Bechtold , @André Pereira de Oliveira , @Lucas Noetzold
This issue was planned to be delivered until 2024-03-25. You can check that by consulting the issue in the Due Date field.
Dates already planned for this issue: 2024-03-01, 2024-03-25
If External Issue Link field is filled, customer was also informed on JIRA TOTVS.
@Rodrigo Bechtold Card validated by the QA team. It is pending only the code review.
cc @Gabriel DAmore Marciano
RETESTING FIX
NO ERRORS ON GCP:
PUBLIC ACCESS REMOVED
NEW TENANT AFTER REDEPLOY WITH URL FILES
Github user rodrigo-bechtold has just commited and issue was sent back to the REVIEW column.
@Rodrigo Bechtold The retest has been performed but now the Task is not being created for the tenant with the parameter true
SANDBOX: https://totvsdragonmaster.qarol.ai/newtenatdoug/carol-ui/explore/search/mdmproduct/rt
SANDBOX REPLOYED
GLOBAL USER RUNNING THE MIGRATION
This issue was automatically transitioned to QA REVIEW, as its PR was just approved in Github.
Adiantando um tema de parking lot voltado ao card .
• Uma tenant eh criada e tem o parametro `"mdmCdsShortURLAuthentication": true`
• Roda-se o Update Migration pontuado no card
• O Public Access eh atualizado na base como false na tabela ShortURL, porem o parametro da tenant continua como TRUE. Esse cenario eh esperado, tendo em vista que a plataforma nao suporta atualizar o parametro para false correto?
Edited on Slack - platform-internal - Douglas Coimbra Lopes
Adiantando um tema de parking lot voltado ao card .
• Uma tenant eh criada e tem o parametro `"mdmCdsShortURLAuthentication": true`
• Roda-se o Update Migration pontuado no card
Sent by Slack - platform-internal - Douglas Coimbra Lopes
:rotating_light: Cenario encontrado no .
• Ao rodar o endpoint de migration, as tenants continuam com parametero de ShortURL Auth configurado como true
• Evidencias e passos no card
Sent by Slack - platform-internal - Douglas Coimbra Lopes
@Rodrigo Bechtold
When The SHORTURLmigration endpoint is executed, the tenants are not getting their ShortURL parameter to false.
The Remove SHortURL access task is failing with NPE: https://cloudlogging.app.goo.gl/Rs7sQBWzUpkqmUe66
SANDBOX TASK: https://totvsybagems.qarol.ai/newtenantdoug/carol-ui/tasks/activity/715b896afad34f4dbc4c78aea7e017cc?p=1&ps=25&sort=dateUpdated&order=DESC&filters=%5B%7B%22hideInternal%22:%22false%22%7D%5D
Steps:
as CarolAdmin/ OrgAdmin, create a tenant
as CarolAdmin/ OrgAdmin, update the tenant to be
"mdmCdsShortURLAuthentication": true
Try to Run the Migration endpoint
1
2
3
4
5
Check the tenant that is still with the parameter as false
TENANT WITH THE IMAGE
TENANT BEING UPDATED BY CAROL ADMIN WITH PARAMETER TRUE
This issue was automatically transitioned to REVIEW, as its PR (not DRAFT and not WIP) was just created in Github.
feat: https://totvslabs.atlassian.net/browse/CAPL-5419#icft=CAPL-5419 - ShortURL Authentication processed by task with rate limit
@MARCOS STUMPF ,
@Gabriel DAmore Marciano , @Lucas Noetzold
This issue was planned to be delivered until 2024-03-04. You can check that by consulting the issue in the Due Date field.
Dates already planned for this issue: 2024-03-04
If External Issue Link field is filled, customer was also informed on JIRA TOTVS.