Repository navigation
Change ld_list according to #439 #445
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
da136c3
03ebd32
d1eb612
d28dc10
fb5aa64
5248853
e2c1bba
aeb88fb
007e3a3
3c9ea61
66e03df
c4aea65
7d30b16
f8d259f
2b98627
1958518
4acf8de
b751130
3c243f2
e0f0fc9
1b5656e
071630c
d4b34b4
f0f1818
bcc233d
287d37a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,56 +21,48 @@ | |
| "ld_container": lambda c, **_: c, | ||
| "json": lambda c, **_: c.compact(), | ||
| "expanded_json": lambda c, **_: c.ld_value, | ||
| }, | ||
| } | ||
| ), | ||
|
|
||
| # Wrap item from ld_dict in ld_list | ||
| (ld_list.is_ld_list, dict(ld_container=ld_list)), | ||
| ( | ||
| lambda c: isinstance(c, list) and all(isinstance(item, dict) for item in c), | ||
| dict(ld_container=lambda c, **kw: ld_list([{"@list": c}], **kw)) | ||
| ), | ||
| (ld_list.is_ld_list, {"ld_container": ld_list}), | ||
| (lambda c: isinstance(c, list), {"ld_container": lambda c, **kw: ld_list([{"@list": c}], **kw)}), | ||
|
|
||
| # pythonize items from lists (expanded set is already handled above) | ||
| (ld_container.is_json_id, dict(python=lambda c, **_: c["@id"])), | ||
| (ld_container.is_typed_json_value, dict(python=ld_container.typed_ld_to_py)), | ||
| (ld_container.is_json_value, dict(python=lambda c, **_: c["@value"])), | ||
| (ld_list.is_container, dict(ld_container=lambda c, **kw: ld_list([c], **kw))), | ||
| (ld_dict.is_json_dict, dict(ld_container=lambda c, **kw: ld_dict([c], **kw))), | ||
| (ld_container.is_json_id, {"python": lambda c, **_: c["@id"]}), | ||
| (ld_container.is_typed_json_value, {"python": lambda c, **kw: ld_container.typed_ld_to_py([c], **kw)}), | ||
| (ld_container.is_json_value, {"python": lambda c, **_: c["@value"]}), | ||
| (ld_list.is_container, {"ld_container": lambda c, **kw: ld_list([c], **kw)}), | ||
| (ld_dict.is_json_dict, {"ld_container": lambda c, **kw: ld_dict([c], **kw)}), | ||
| (lambda v: isinstance(v, str), {"python": lambda v, parent, **_: parent.ld_proc.compact_iri(parent.active_ctx, v)}), | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This rule is to generic IMHO, as it will try to compact all strings that are encountered, even those that are explicitly not meant as representing an IRI.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But shouldn't all other string should be known to be a value of some sort i.e. formed like
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see you have answers... but I'm totally not sure whether this is really true. Apart from this, except for
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ... anyways, there should most proably be a documentation for the conversion rules and the order they are applied in (maybe directly generated from the code... ;) ) |
||
|
|
||
| # Convert internal data types to expanded_json | ||
| (lambda c: ld_container.is_json_id(c) or ld_container.is_json_value(c), dict(expanded_json=lambda c, **_: [c])), | ||
| (ld_dict.is_json_dict, dict(expanded_json=lambda c, **kw: ld_dict.from_dict(c, **kw).ld_value)), | ||
| (ld_dict.is_ld_dict, dict(expanded_json=lambda c, **kw: ld_dict.from_dict(c[0], **kw).ld_value)), | ||
| (lambda c: ld_container.is_json_id(c), {"expanded_json": lambda c, **_: [c]}), | ||
| (lambda c: ld_container.is_ld_id(c), {"expanded_json": lambda c, **_: c}), | ||
| (lambda c: ld_container.is_json_value(c), {"expanded_json": lambda c, **_: [c]}), | ||
| (lambda c: ld_container.is_ld_value(c), {"expanded_json": lambda c, **_: c}), | ||
| (ld_dict.is_json_dict, {"expanded_json": lambda c, **kw: ld_dict.from_dict(c, **kw).ld_value}), | ||
| ( | ||
| ld_list.is_container, | ||
| dict( | ||
| expanded_json=lambda c, **kw: ld_list.from_list( | ||
| ld_list([c]).item_list, container=ld_list([c]).container, **kw | ||
| ).ld_value | ||
| ), | ||
| {"expanded_json": lambda c, **kw: ld_list.from_list(ld_list.get_item_list_from_container(c), **kw).ld_value} | ||
| ), | ||
| ( | ||
| ld_list.is_ld_list, | ||
| dict( | ||
| expanded_json=lambda c, **kw: ld_list.from_list( | ||
| ld_list(c).item_list, container=ld_list(c).container, **kw | ||
| ).ld_value | ||
| ), | ||
| {"expanded_json": lambda c, **kw: ld_list.from_list(ld_list.get_item_list_from_container(c[0]), **kw).ld_value} | ||
| ), | ||
| (lambda c: isinstance(c, list), dict(expanded_json=lambda c, **kw: ld_list.from_list(c, **kw).ld_value)), | ||
| (lambda v: isinstance(v, (int, float, str, bool)), dict(expanded_json=lambda v, **_: [{"@value": v}])), | ||
| (lambda c: isinstance(c, list), {"expanded_json": lambda c, **kw: ld_list.from_list(c, **kw).ld_value}), | ||
| (lambda v: isinstance(v, (int, float, str, bool)), {"expanded_json": lambda v, **_: [{"@value": v}]}), | ||
| ( | ||
| lambda v: isinstance(v, datetime), | ||
| dict(expanded_json=lambda v, **_: [{"@value": v.isoformat(), "@type": iri_map["schema:DateTime"]}]), | ||
| {"expanded_json": lambda v, **_: [{"@value": v.isoformat(), "@type": iri_map["schema:DateTime"]}]} | ||
| ), | ||
| ( | ||
| lambda v: isinstance(v, date), | ||
| dict(expanded_json=lambda v, **_: [{"@value": v.isoformat(), "@type": iri_map["schema:Date"]}]), | ||
| {"expanded_json": lambda v, **_: [{"@value": v.isoformat(), "@type": iri_map["schema:Date"]}]} | ||
| ), | ||
| ( | ||
| lambda v: isinstance(v, time), | ||
| dict(expanded_json=lambda v, **_: [{"@value": v.isoformat(), "@type": iri_map["schema:Time"]}]), | ||
| {"expanded_json": lambda v, **_: [{"@value": v.isoformat(), "@type": iri_map["schema:Time"]}]} | ||
| ), | ||
| ] | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -83,13 +83,6 @@ def ld_value(self): | |
| def _to_python(self, full_iri, ld_value): | ||
| if full_iri == "@id": | ||
| value = self.ld_proc.compact_iri(self.active_ctx, ld_value, vocab=False) | ||
| elif full_iri == "@type": | ||
| value = [ | ||
| self.ld_proc.compact_iri(self.active_ctx, ld_type) | ||
| for ld_type in ld_value | ||
| ] | ||
| if len(value) == 1: | ||
| value = value[0] | ||
|
Comment on lines
-86
to
-92
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure what by my guts say this will break stuff in an unexpected way. Not sure why (and hence there should be better documentation) but I remember removing and adding this section over and over again...
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. see hint above... this will break conversion of single type classes which is valid but which was already critizised towards me (or maybe there were even real errors... but I doubt it)
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I haven't found anything it breaks yet, e.g. no failing tests, but it took some effort to get this running. |
||
| else: | ||
| value, ld_output = self.ld_proc.apply_typemap(ld_value, "python", "ld_container", | ||
| parent=self, key=full_iri) | ||
|
|
@@ -189,3 +182,14 @@ def typed_ld_to_py(cls, data, **kwargs): | |
| ld_value = data[0]['@value'] | ||
|
|
||
| return ld_value | ||
|
|
||
| @classmethod | ||
| def are_values_equal(cls, first, second): | ||
| if "@id" in first and "@id" in second: | ||
| return first["@id"] == second["@id"] | ||
| for key in {"@value", "@type"}: | ||
| if (key in first) ^ (key in second): | ||
| return False | ||
| if key in first and key in second and first[key] != second[key]: | ||
| return False | ||
| return True | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,34 +3,20 @@ | |
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # SPDX-FileContributor: Michael Meinel | ||
| # SPDX-FileContributor: Michael Fritzsche | ||
|
|
||
| from .ld_container import ld_container | ||
|
|
||
|
|
||
| class ld_list(ld_container): | ||
| """ An JSON-LD container resembling a list. """ | ||
|
|
||
| container_types = ['@list', '@set', '@graph'] | ||
|
|
||
| def __init__(self, data, *, parent=None, key=None, index=None, context=None): | ||
| """ Create a new ld_list.py container. | ||
|
|
||
| # FIXME: #439 there is no parameter container | ||
| :param container: The container type for this list. | ||
| """ | ||
| # FIXME: #439 A set container does not contain "@set" in the expected data format (expanded json ld) | ||
| # Instead it is just a list of dicts and therefor would raise a ValueError here (and fail ld_list.is_ld_list) | ||
|
|
||
| if not (self.is_ld_list(data) and "@list" in data[0]): | ||
| raise ValueError("The given data does not represent a ld_list.") | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See my comments on the issue: Don't restrict it to And please, bring back the docstring!
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It wasn't about the name of the class but rather that it didn't work for Will do. |
||
| super().__init__(data, parent=parent, key=key, index=index, context=context) | ||
|
|
||
| # Determine container and correct item list | ||
| for container in self.container_types: | ||
| if container in self._data[0]: | ||
| self.item_list = self._data[0][container] | ||
| self.container = container | ||
| break | ||
| else: | ||
| raise ValueError(f"Unexpected dict: {data}") | ||
| self.item_list = data[0]["@list"] | ||
|
|
||
| def __getitem__(self, index): | ||
| if isinstance(index, slice): | ||
|
|
@@ -42,18 +28,6 @@ def __getitem__(self, index): | |
| return item | ||
|
|
||
| def __setitem__(self, index, value): | ||
| # FIXME: #439 what should your_ld_list[index] = [{"@type": "foo", "name": "bar"}] mean? | ||
| # set your_ld_list[index] to the dict {"@type": "foo", "name": "bar"} given in expanded form or | ||
| # set your_ld_list[index] to the list [{"@type": "foo", "name": "bar"}] given in non expanded form or | ||
| # set your_ld_list[index] to the set [{"@type": "foo", "name": "bar"}] given in expanded form | ||
| # (ld_list.fromlist([{"@type": "foo", "name": "bar"}]) defaults to container type list | ||
| # which would have the object as an expanded form whereas the expanded form of a list would be | ||
| # ["@list": [{"@type": "foo", "name": "bar"}]] | ||
| # This is relevent because nested sets get unnested when being expanded and lists not. | ||
| # Moreover a set inside a list gets automaticaly converted to a list when expanded) | ||
|
|
||
| # FIXME: #439 what happens when a ld_list is put inside another also depends on their container types | ||
|
|
||
| if not isinstance(index, slice): | ||
| self.item_list[index] = val[0] if isinstance(val := self._to_expanded_json(self.key, value), list) else val | ||
| return | ||
|
|
@@ -79,12 +53,36 @@ def __contains__(self, value): | |
| return expanded_value in self.item_list | ||
|
|
||
| def __eq__(self, other): | ||
| if isinstance(other, ld_list): | ||
| # FIXME: #439 When are ld_lists equal? | ||
| return self.item_list == other.item_list and self.container == other.container | ||
| if not (isinstance(other, (list, ld_list)) or ld_list.is_container(other)): | ||
| return NotImplemented | ||
| if isinstance(other, dict): | ||
| other = [other] | ||
| if isinstance(other, list): | ||
| return self.item_list == self._to_expanded_json(self.key, other)[0]["@list"] | ||
| return NotImplemented | ||
| if ld_list.is_ld_list(other): | ||
| other = ld_list.get_item_list_from_container(other[0]) | ||
| other = self.from_list(other, parent=self.parent, key=self.key, context=self.context) | ||
| if len(self.item_list) != len(other.item_list): | ||
| return False | ||
| if (self.key == "@type") ^ (other.key == "@type"): | ||
| return False | ||
| if self.key == other.key == "@type": | ||
| return self.item_list == other.item_list | ||
| for index, (item, other_item) in enumerate(zip(self.item_list, other.item_list)): | ||
| if ((ld_container.is_typed_json_value(item) or ld_container.is_json_value(item)) and | ||
| (ld_container.is_typed_json_value(other_item) or ld_container.is_json_value(other_item))): | ||
| if not ld_container.are_values_equal(item, other_item): | ||
| return False | ||
| continue | ||
| if "@id" in item and "@id" in other_item: | ||
| return item["@id"] == other_item["@id"] | ||
| item = self[index] | ||
| other_item = other[index] | ||
| res = item.__eq__(other_item) | ||
| if res == NotImplemented: | ||
| res = other_item.__eq__(item) | ||
| if res is False or res == NotImplemented: # res is not True | ||
| return False | ||
| return True | ||
|
|
||
| def __ne__(self, other): | ||
| x = self.__eq__(other) | ||
|
|
@@ -108,19 +106,25 @@ def to_python(self): | |
|
|
||
| @classmethod | ||
| def is_ld_list(cls, ld_value): | ||
| # FIXME: #439 every python list that contains at least one dict can be considerd a set in expanded json form | ||
| return cls.is_ld_node(ld_value) and cls.is_container(ld_value[0]) | ||
|
|
||
| @classmethod | ||
| def is_container(cls, value): | ||
| # FIXME: #439 "@set" will never be inside a dictionary of an expanded json ld object | ||
| return ( | ||
| isinstance(value, dict) | ||
| and len([1 for ct in cls.container_types if isinstance(value.get(ct, None), list)]) == 1 | ||
| and [*value.keys()] in [["@list"], ["@set"], ["@graph"]] | ||
| and any(isinstance(value.get(cont, None), list) for cont in {"@list", "@set", "@graph"}) | ||
| ) | ||
|
|
||
| @classmethod | ||
| def from_list(cls, value, *, parent=None, key=None, context=None, container=None): | ||
| new_list = cls([{container or "@list": []}], parent=parent, key=key, context=context) | ||
| def from_list(cls, value, *, parent=None, key=None, context=None): | ||
| new_list = cls([{"@list": []}], parent=parent, key=key, context=context) | ||
| new_list.extend(value) | ||
| return new_list | ||
|
|
||
| @classmethod | ||
| def get_item_list_from_container(cls, ld_value): | ||
| for cont in {"@list", "@set", "@graph"}: | ||
| if cont in ld_value: | ||
| return ld_value[cont] | ||
| raise ValueError("The given data does not represent a container.") | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # SPDX-FileCopyrightText: 2025 German Aerospace Center (DLR) | ||
| # | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| # SPDX-FileContributor: Michael Fritzsche | ||
|
|
||
| from hermes.model.types.ld_dict import ld_dict | ||
|
|
||
|
|
||
| def test_build_in_comparison(): | ||
| di = ld_dict([{}], context={"schema": "https://schema.org/"}) | ||
| assert di != 1 and di != [] and di != "" | ||
| di["@id"] = "foo" | ||
| di["schema:name"] = "bar" | ||
| assert di == {"@id": "foo"} | ||
| # Fail probably because of bug in ld_dict | ||
| # that is fixed on refactor/data-model after merge of refactor/384-test-ld_dict | ||
| assert di == {"@id": "foo", "schema:name": "bar"} | ||
| assert di == {"@id": "foo", "name": "b"} | ||
| assert di == {"schema:name": "bar"} | ||
| di = ld_dict([{}], context={"schema": "https://schema.org/"}) | ||
| di["schema:Person"] = {"schema:name": "foo"} | ||
| assert di == {"schema:Person": {"schema:name": "foo"}} | ||
| di["schema:Person"].append({"schema:name": "bar"}) | ||
| assert di == {"schema:Person": [{"schema:name": "foo"}, {"schema:name": "bar"}]} | ||
| assert di != {"schema:name": "foo"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The side condition is important to my understanding. This rule should invoked upon an expanded JSON-LD list. In fact, the rule should be even more strict ensuring the items in the list are not only dicts but valid JSON-LD values.
Or am I getting something wrong here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When getting a value from an ld_dict it should return an ld_list always. Therefor it is important to know wheter a given value comes from an ld_list or an ld_dict. If the value, where expanded JSON-LD is expected, is not inside a list it previously was inside a list.
I removed the condition because when getting "@type" from
{"@type": ["A"]}the value in the list is a string and not a dict. (I found no cases where this goes wrong because everything is valid expanded JSON-LD.)We should know that the values are valid expanded JSON-LD here because this conversion happens only when converting to the pythonized version of a value that was previously expanded.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I see... that seems to be related to the removal of the explcit handling of
@typein the conversion.Well, that's fine if you can deal with the implications...
{"@type": "foo"}(which is valid) will always be{"@type": ["foo"]}when converted usingto_python@listcontainer or individually wrapped in some@valueor@idnode. Yet, the values are subject to IRI expansion / compaction.