Add label for bigquery jobs
Description
PRDE - Story default text according to the team DoR (Definition of Ready)
01 - PERSON OF CONTACT (PERSON THAT CAN ANSWER QUESTIONS ABOUT THE PROBLEM):
02 - STORY BRIEFING (AS A < PERSONA >, I [WANT TO] < NEED >, [SO THAT] < GOAL >):
03 - PROBLEM (WHAT'S THE CURRENT PROBLEM SCENARIO OR PAIN TO BE RESOLVED?):
04 - GOAL (DESCRIBE THE PROPOSED SOLUTION):
We need to classify better the BQ jobs we have today.
Add the following labels (at least):
orgId
orgName
tenantId
tenantName
taskID (SQL Processing)
source: explore, task, internal, etc
appName: when it’s called by Carol App.
carolAppProcessName: when it’s called by Carol App.
We need to add the label for all BQ Jobs. Some I can list right now:
- Carol App: queries generated through pyCarol SA (is it possible to define labels through a SA?)
- SQL Processing tasks
- Explore
- VSCode
How we are mapping the user vs apps today in some evaluations:
when user_email like 'tmpaccess%' then 'carolApp'
when user_email like 'permanent-data-writer%' then 'sqlPipeline'
when user_email like 'carol-sql-prod-s%' then 'dataPlatform'
Endpoint called by FE (SQL Query)
https://totvs.carol.ai/api/v1/bigQuery/query
05 - WHO CAN USE THIS FEATURE (USER ROLES):
06 - ACTIVITY DIAGRAM (ACTIVITY DIAGRAM LINK AND IMAGE):
07 - STEPS (ACTIONS TO BE PERFORMED LINKING TO SCREENSHOTS):
08 - ALTERNATIVE STEPS:
09 - ASSETS (FIGMA LINKS, RELEVANT DOCUMENTATION LINKS, JSON EXAMPLES, ETC):
10 - ACCEPTANCE CRITERIA:
- All BQ jobs should have the specified labels:
- orgId
- orgName
- tenantId
- tenantName
- taskID (SQL Processing)
- source: explore, task, internal, etc
- appName: when it’s called by Carol App.
- carolAppProcessName: when it’s called by Carol App.
- BQ Jobs to be considered:
- SQL Processing tasks
- Explore
- VSCode
- Jobs from Date team (source: internal)
- User Rejected Golden Records
- Not Covered:
- Carol App: queries generated through pyCarol SA (is it possible to define labels through a SA?)