Skip to content

Feature/no answer pipeline #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
viktors264 wants to merge 7 commits into
base: main
Choose a base branch
from
+23 −11
Open

Feature/no answer pipeline #183

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8e880a6
Added noAnswer key and updated generic_pipeline aggregation
Dec 29, 2022
79a4fef
Added missing industry_selectior keys
Dec 29, 2022
8f2ef57
Added no_answer key to buckets, updated generic_pipeline.ts
Jan 18, 2023
42afe3d
Removed no_answer key from keys.yml
Jan 18, 2023
02475a5
Merge branch 'main' into feature/no-answer-pipeline
viktors264 Jan 18, 2023
28fdd21
Added unwind with preserveNullAndEmptyArrays option to fix field with…
Jan 18, 2023
90d5f8a
Merge branch 'feature/no-answer-pipeline' of github.com:viktors264/Mo…
Jan 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions api/src/compute/generic_pipeline.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,18 +37,23 @@ export const getGenericPipeline = (pipelineProps: PipelineProps) => {
$match: match
},
// { $count: 'questionRespondents' },
{
$set: {
[`${facetPath}`]: { $cond: [ { $not: [`$${facetPath}`] }, "noAnswer", `$${facetPath}` ] }
}
},
{
$unwind: {
path: `$${key}`
}
},
...(facetPath
? [
{
$unwind: {
path: `$${facetPath}`
}
}
// {
// $unwind: {
// path: `$${facetPath}`
// }
// }

@SachaG SachaG Jan 17, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we can safely remove this whole step then?

@viktors264 viktors264 Jan 18, 2023

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed comments. Changed aggregation.

]
: []),
{
Expand Down
12 changes: 7 additions & 5 deletions api/src/data/keys.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ age:
- range_more_than_65

years_of_experience:
- no_answer

@SachaG SachaG Jan 17, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because every field will need a no_answer key I don't think it makes sense to add it here, it's probably better to add it directly in generic.ts somewhere.

@SachaG SachaG Jan 18, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we actually need to add it though, or maybe only at the GraphQL level… at least I don't think we use those keys in the pipeline? I'll double check.

@viktors264 viktors264 Jan 18, 2023 •
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously

This is a good start! But it's missing a key feature, which is that the no_answer key should be added to buckets, not just facets.

Generic pipeline updated. Added "no_answer" key to the buckets section. Also, when no facets is selected, I tested and got results as you wrote before.

@viktors264 viktors264 Jan 18, 2023

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because every field will need a no_answer key I don't think it makes sense to add it here, it's probably better to add it directly in generic.ts somewhere.

Yes, I absolutely agree. If 'no_answer' key is needed for every field, better to add it in generic.ts or at the GraphQL level (if keys.yml are not used in pipeline). I have checked - don't found straight use of keys in Mongo aggregation, but you better check.

- range_less_than_1
- range_1_2
- range_2_5
Expand Down Expand Up @@ -47,6 +48,7 @@ gender:
- non_binary
- not_listed


race_ethnicity:
# - biracial
- black_african
Expand Down Expand Up @@ -99,14 +101,14 @@ feature:
- heard
- used

happiness:
happiness:
- 0
- 1
- 2
- 3
- 4

opinions:
opinions:
- 0
- 1
- 2
Expand All @@ -125,12 +127,12 @@ graphql_experience:
- range_3_5
- range_6_10

code_generation_type:
code_generation_type:
- code_first
- schema_first
- database_first

css_js_balance:
css_js_balance:
- 0
- 1
- 2
Expand All @@ -139,4 +141,4 @@ css_js_balance:
- 5
- 6
- 7
- 8
- 8