-
Notifications
You must be signed in to change notification settings - Fork 20
feat: implement CSVDiscoverer and catalog registry models #271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mnoserat
wants to merge
42
commits into
community-outpost:development
Choose a base branch
from
mnoserat:feature/CSV-Discoverer
base: development
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 38 commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
c9fa4be
feat: implement CSVDiscoverer and catalog registry models
mnoserat 45f8374
add missing validation for index.Entries
mnoserat 35d5c82
add CsvConstants.cs and reslove all warnings
mnoserat 6445bc3
change publisher to constant (the super hacker
mnoserat 938263b
fix: configuration section is at wrong path
mnoserat bc8706f
fix: schema mismatch CsvCatalogRegistryIndex
mnoserat 5853ab1
fix: Inconsistent FileCount type between CsvValidationCatalog and Csv…
mnoserat a272d46
feat(fix): configuration section is at wrong path
mnoserat 58b2780
feat(fix): rephrase the AI authorship comment
mnoserat 6d5d16e
feat(fix): Expand test coverage for CSVDiscoverer
mnoserat 61a6156
feat(fix): SemaphoreSlim is not disposed
mnoserat 03ef494
feat(fix): Double-checked locking pattern: consider using Volatile.Read
mnoserat ead2ac6
feat(fix): Minor documentation inaccuracy.
mnoserat 5ba833a
feat(fix): Remove unused import
mnoserat 8a41bed
feat(fix): Consider using a sentinel value for LastUpdatedAt default
mnoserat 6e1f2cd
feat(fix): Language code should be uppercase
mnoserat 1290c6f
feat(fix): TOCTOU race condition between File.Exists and File.ReadAll…
mnoserat bfda8d3
feat(fix): caching JsonSerializerOptions to avoid repeated allocation.
mnoserat 5eb7fe8
feat(fix): Update CsvConstants.ResolverId to match issue #139 specifi…
mnoserat 7f58082
feat(fix): match CsvCatalogRegistryEntry.cs with the actual JSON stru…
mnoserat b50f52c
feat(fix): adding explicit handling each valid game type
mnoserat 317e23c
remove unused constant
mnoserat abf8c8f
Clarify IndexFilePath documentation semantics.
mnoserat 2ca99cb
Switch expression now returns null for unknown GameType values inste…
mnoserat e896fdf
feat(fix): Language handling misses unified catalogs and emits non-ca…
mnoserat 265ac94
feat(fix): Cancellation is swallowed and converted into a generic fai…
mnoserat 3b55070
fix: Missing properties that exist in index.json
mnoserat e4d22fc
fix: CSVDiscovererTests.cs
mnoserat 4d3363b
fix No validation for null or invalid entries. If index.Entries conta…
mnoserat 7af0378
fix: IsActive flag is not checked when filtering entries.
mnoserat a78f88e
fix: remove unused config
mnoserat 5d78c8c
fix: add null guard for _config in constructor
mnoserat 5a5992f
fix: remove Unused using
mnoserat d1f34c8
Guard language collections before normalization to avoid discoverer-w…
mnoserat 6884e87
Build manifest identity from canonical game type, not raw source casing.
mnoserat ba9f625
Merge remote-tracking branch 'upstream/development' into feature/CSV-…
mnoserat af1942f
fix: compile Failing
mnoserat 8b00efd
fix: compile Failing wih exit code -1
mnoserat a725109
fix change branch url in index.json
mnoserat fdce2c6
refactor(csv-discovery): use index.json as single registry sourc
mnoserat 9980a68
fix(csv-discovery): load index.json from remote source with default f…
mnoserat 58d57e5
fix(csv-discovery): load CSV catalogs from unified source chain
mnoserat File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| namespace GenHub.Core.Constants; | ||
|
|
||
| /// <summary> | ||
| /// Constants specific to CSV catalog discovery and content pipeline. | ||
| /// </summary> | ||
| public static class CsvConstants | ||
| { | ||
| /// <summary> | ||
| /// Source name for the CSV catalog discoverer. | ||
| /// </summary> | ||
| public const string SourceName = "Csv Discoverer"; | ||
|
|
||
| /// <summary> | ||
| /// Description for the CSV catalog discoverer. | ||
| /// </summary> | ||
| public const string Description = "Discovers base game manifests from verified CSV catalogs."; | ||
|
|
||
| /// <summary> | ||
| /// Resolver ID for CSV catalog content. | ||
| /// </summary> | ||
| public const string ResolverId = "CSVResolver"; | ||
|
|
||
| /// <summary> | ||
| /// Metadata key for the CSV source URL. | ||
| /// </summary> | ||
| public const string CsvUrlMetadataKey = "csvUrl"; | ||
|
|
||
| /// <summary> | ||
| /// Metadata key for the game type. | ||
| /// </summary> | ||
| public const string GameTypeMetadataKey = "gameType"; | ||
|
|
||
| /// <summary> | ||
| /// Metadata key for the game version. | ||
| /// </summary> | ||
| public const string VersionMetadataKey = "version"; | ||
|
|
||
| /// <summary> | ||
| /// Metadata key for the content language. | ||
| /// </summary> | ||
| public const string LanguageMetadataKey = "language"; | ||
|
|
||
| /// <summary> | ||
| /// Metadata key for the expected file count. | ||
| /// </summary> | ||
| public const string FileCountMetadataKey = "fileCount"; | ||
|
|
||
| /// <summary> | ||
| /// String representation for Generals game type in CSV catalogs. | ||
| /// </summary> | ||
| public const string GeneralsGameType = "Generals"; | ||
|
|
||
| /// <summary> | ||
| /// String representation for Zero Hour game type in CSV catalogs. | ||
| /// </summary> | ||
| public const string ZeroHourGameType = "ZeroHour"; | ||
|
|
||
| /// <summary> | ||
| /// Special language filter value to include all languages. | ||
| /// </summary> | ||
| public const string AllLanguagesFilter = "ALL"; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| using System.Text.Json.Serialization; | ||
|
|
||
| namespace GenHub.Core.Models.Content; | ||
|
|
||
| /// <summary> | ||
| /// Represents checksum information for file integrity verification. | ||
| /// </summary> | ||
| public class Checksum | ||
| { | ||
| /// <summary> | ||
| /// Gets or sets the MD5 hash of the file. | ||
| /// </summary> | ||
| [JsonPropertyName("md5")] | ||
| public string Md5 { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the SHA-256 hash of the file. | ||
| /// </summary> | ||
| [JsonPropertyName("sha256")] | ||
| public string Sha256 { get; set; } = string.Empty; | ||
| } |
22 changes: 22 additions & 0 deletions
22
GenHub/GenHub.Core/Models/Content/CsvCatalogConfiguration.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| namespace GenHub.Core.Models.Content; | ||
|
|
||
| /// <summary> | ||
| /// Configuration for CSV Catalog discovery. | ||
| /// Binds to "CsvValidationCatalogs" and "GenHub:IndexFilePath" in configuration. | ||
| /// Refers to the <see cref="IndexFilePath"/> property. | ||
| /// </summary> | ||
|
mnoserat marked this conversation as resolved.
|
||
| public class CsvCatalogConfiguration | ||
| { | ||
| /// <summary> | ||
| /// Gets or sets the list of fallback validation catalogs defined in configuration. | ||
| /// Used when index.json cannot be reached. | ||
| /// </summary> | ||
| public List<CsvValidationCatalog> CsvValidationCatalogs { get; set; } = []; | ||
|
mnoserat marked this conversation as resolved.
Outdated
|
||
|
|
||
| /// <summary> | ||
| /// Gets or sets the path to the index.json file. | ||
| /// If a relative path is provided, it is resolved relative to the application's working directory; | ||
| /// absolute paths are used as given. The expected file is index.json. | ||
| /// </summary> | ||
| public string IndexFilePath { get; set; } = "docs/GameInstallationFilesRegistry/index.json"; | ||
|
mnoserat marked this conversation as resolved.
Outdated
mnoserat marked this conversation as resolved.
Outdated
|
||
| } | ||
76 changes: 76 additions & 0 deletions
76
GenHub/GenHub.Core/Models/Content/CsvCatalogRegistryEntry.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| using System.Collections.Generic; | ||
| using System.Text.Json.Serialization; | ||
|
|
||
| namespace GenHub.Core.Models.Content; | ||
|
|
||
| /// <summary> | ||
| /// Represents a single CSV catalog entry in the registry. | ||
| /// </summary> | ||
| public class CsvCatalogRegistryEntry | ||
| { | ||
| /// <summary> | ||
| /// Gets or sets the unique identifier for this registry entry. | ||
| /// </summary> | ||
| [JsonPropertyName("id")] | ||
| public string Id { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the URL to the CSV file. | ||
| /// </summary> | ||
| [JsonPropertyName("url")] | ||
| public string Url { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the game type (e.g., "Generals", "ZeroHour"). | ||
| /// </summary> | ||
| [JsonPropertyName("gameType")] | ||
| public string GameType { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the game version (e.g., "1.08", "1.04"). | ||
| /// </summary> | ||
| [JsonPropertyName("version")] | ||
| public string Version { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the list of supported languages in this CSV. | ||
| /// </summary> | ||
| [JsonPropertyName("languages")] | ||
| public List<string> SupportedLanguages { get; set; } = []; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the optional file count in the CSV. | ||
| /// </summary> | ||
| [JsonPropertyName("fileCount")] | ||
| public int? FileCount { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the total size of the CSV file in bytes. | ||
| /// </summary> | ||
| [JsonPropertyName("totalSizeBytes")] | ||
| public long TotalSizeBytes { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the checksum information for integrity verification. | ||
| /// </summary> | ||
| [JsonPropertyName("checksum")] | ||
| public Checksum? Checksum { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets when this registry entry was generated. | ||
| /// </summary> | ||
| [JsonPropertyName("generatedAt")] | ||
| public DateTime? GeneratedAt { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the version of the generator that created this entry. | ||
| /// </summary> | ||
| [JsonPropertyName("generatorVersion")] | ||
| public string GeneratorVersion { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets a value indicating whether this registry entry is active. | ||
| /// </summary> | ||
| [JsonPropertyName("isActive")] | ||
| public bool IsActive { get; set; } | ||
| } | ||
|
mnoserat marked this conversation as resolved.
|
||
36 changes: 36 additions & 0 deletions
36
GenHub/GenHub.Core/Models/Content/CsvCatalogRegistryIndex.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| using System; | ||
| using System.Collections.Generic; | ||
| using System.Text.Json.Serialization; | ||
|
|
||
| namespace GenHub.Core.Models.Content; | ||
|
|
||
| /// <summary> | ||
| /// Index of available CSV catalog registries. | ||
| /// Represents the structure of the docs/GameInstallationFilesRegistry/index.json file. | ||
| /// </summary> | ||
| public class CsvCatalogRegistryIndex | ||
| { | ||
| /// <summary> | ||
| /// Gets or sets the schema version. | ||
| /// </summary> | ||
| [JsonPropertyName("version")] | ||
| public string Version { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets when the index was last updated. | ||
| /// </summary> | ||
| [JsonPropertyName("lastUpdated")] | ||
| public DateTime? LastUpdatedAt { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the description of this registry index. | ||
| /// </summary> | ||
| [JsonPropertyName("description")] | ||
| public string Description { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the list of available catalog entries. | ||
| /// </summary> | ||
| [JsonPropertyName("registries")] | ||
|
mnoserat marked this conversation as resolved.
|
||
| public List<CsvCatalogRegistryEntry> Entries { get; set; } = []; | ||
|
mnoserat marked this conversation as resolved.
|
||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| using System.Collections.Generic; | ||
|
|
||
| namespace GenHub.Core.Models.Content; | ||
|
|
||
| /// <summary> | ||
| /// Represents a CSV catalog defined in appsettings.json. | ||
| /// Matches the structure of CsvCatalogRegistryEntry but optimized for config binding. | ||
| /// </summary> | ||
| public class CsvValidationCatalog | ||
| { | ||
| /// <summary> | ||
| /// Gets or sets the URL to the CSV file. | ||
| /// </summary> | ||
| public string Url { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the game type. | ||
| /// </summary> | ||
| public string GameType { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the version string. | ||
| /// </summary> | ||
| public string Version { get; set; } = string.Empty; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the supported languages. | ||
| /// </summary> | ||
| public List<string> SupportedLanguages { get; set; } = []; | ||
|
|
||
| /// <summary> | ||
| /// Gets or sets the file count. | ||
| /// </summary> | ||
| public int? FileCount { get; set; } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.