Add demand allocation scraper#4428
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #4428 +/- ##
==========================================
+ Coverage 54.52% 56.40% +1.87%
==========================================
Files 274 317 +43
Lines 6076 6975 +899
Branches 1455 1683 +228
==========================================
+ Hits 3313 3934 +621
- Misses 2763 3041 +278 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| import type { ModuleCode, Semester } from './modules'; | ||
|
|
||
| export type CourseRegStudentType = 'UG' | 'GD'; | ||
|
|
||
| export type CourseRegRound = 0 | 1 | 2 | 3; | ||
|
|
||
| export type CourseRegSlotValue = number | 'notAvailable' | 'unlimited'; | ||
|
|
||
| export type CourseRegRoundHistory = { | ||
| allocatedSlots: CourseRegSlotValue; | ||
| forecastedSlots: CourseRegSlotValue; | ||
| registered: number | null; | ||
| round: CourseRegRound; | ||
| }; | ||
|
|
||
| export type CourseRegClassHistory = { | ||
| classNo: string; | ||
| rounds: Array<CourseRegRoundHistory>; | ||
| studentType: CourseRegStudentType; | ||
| }; | ||
|
|
||
| export type CourseRegModuleSemesterHistory = { | ||
| acadYear: string; | ||
| classes: Array<CourseRegClassHistory>; | ||
| moduleCode: ModuleCode; | ||
| semester: Semester; | ||
| }; |
There was a problem hiding this comment.
CourseRegModuleSemesterHistory, CourseRegClassHistory, CourseRegRoundHistory, CourseRegSlotValue, CourseRegRound, and CourseRegStudentType are defined identically in both scrapers/demand-allocation-scraper/src/types.ts and this file. These two copies will drift independently as the format evolves. Consider exporting the types from one canonical location (e.g., the scraper package) and importing them in nus-v2, or pulling the shared contract into a shared workspace package.
Prompt To Fix With AI
This is a comment left during a code review.
Path: scrapers/nus-v2/src/types/course-reg.ts
Line: 1-27
Comment:
**Duplicate type definitions across packages**
`CourseRegModuleSemesterHistory`, `CourseRegClassHistory`, `CourseRegRoundHistory`, `CourseRegSlotValue`, `CourseRegRound`, and `CourseRegStudentType` are defined identically in both `scrapers/demand-allocation-scraper/src/types.ts` and this file. These two copies will drift independently as the format evolves. Consider exporting the types from one canonical location (e.g., the scraper package) and importing them in `nus-v2`, or pulling the shared contract into a shared workspace package.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Summary
scrapers/demand-allocation-scraperpackage for CourseReg vacancy and demand allocation PDFsscrapers/demand-allocation-scraper/requirements.txtcourseRegHistory.json/v2/{acadYear}/semesters/{semester}/courseRegHistory.jsonin Swagger and NUS v2 typesRefs #4424
CourseRekt parity
courseRegHistory.json: 0 missing rows, 0 extra rows, 0 cell mismatchesTesting
pnpm --filter nusmods-demand-allocation-scraper check/private/tmp/courserekt-venv/bin/python -m pip install --dry-run -r scrapers/demand-allocation-scraper/requirements.txt/private/tmp/courserekt-venv/bin/python -m pip checkpnpm --filter nusmods test -- src/apis/nusmods.test.js(ran full website suite; unrelated existing date/weather assertions failed under local Node v24 run)pnpm --filter nusmods exec vitest run src/apis/nusmods.test.js --coverage=falsepnpm --filter nusmods typecheckpnpm --filter nusmods lint:codepnpm --filter nus-v2 lintpnpm --filter nus-v2 typecheckpnpm --filter nus-v2 buildpython3 -c "import yaml, pathlib; yaml.safe_load(pathlib.Path('scrapers/nus-v2/swagger.yaml').read_text()); print('swagger yaml ok')"PYTHONPYCACHEPREFIX=/private/tmp/nusmods-pycache python3 -m py_compile scrapers/demand-allocation-scraper/scripts/coursereg_pdf_to_csv.py scrapers/demand-allocation-scraper/scripts/import_courserekt_pdfs.py