Be flexible with antlr version?

[satta]

I am looking to package this for Debian, and we need to depend on a given package for Python dependencies. This means that we can't always ensure a specific version, for example, for antlr4. This leads to messages like

ANTLR runtime and generated code versions disagree: 4.9.1!=4.8

all over the place with the Debian version (https://packages.debian.org/sid/python3-antlr4) vs. the one you were using. There is no way to get exactly version 4.8 there.

Is there a way to relax the requirements?

[chisholm]

ANTLR looks for exact match on both major and minor version values. If either is different, you get that warning message. Looks to me like their definition of "compatible version" is baked into the code there. Nothing we can really do about it. To make the generated parser compatible with runtime 4.9.x, it would have to be re-generated from the grammar files. And it would be outdated again when runtime 4.10 came out...

I wonder if it would make sense for a packager to just re-generate the parser to get something compatible with their distro's ANTLR runtime? We are slightly out of date though; @clenk would it make sense to update the master branch to 4.9.x?

[satta]

ANTLR looks for exact match on both major and minor version values. If either is different, you get that warning message. Looks to me like their definition of "compatible version" is baked into the code there. Nothing we can really do about it. To make the generated parser compatible with runtime 4.9.x, it would have to be re-generated from the grammar files. And it would be outdated again when runtime 4.10 came out...

OK, I see. Note that apart from the warning message, I can't see any negative effect yet. But it would be nice to sort this out to be able to package things in the correct way.

I wonder if it would make sense for a packager to just re-generate the parser to get something compatible with their distro's ANTLR runtime?

This would definitely the way to go. However, in Debian currently ANTLR and the Python runtime are already out of sync (4.7 for ANTLR vs. 4.9 for the Python runtime, see https://packages.debian.org/search?keywords=antlr4)

This would also require the package for python3-stix2-patterns to be rebuilt every time the antlr version in Debian is updated. This is doable but would require the packager to take care of this for an indefinite time, which I was hoping to avoid. But looks like there's no real way to work around it.

[clenk]

@chisholm yes, we should definitely update to the latest ANTLR version.

[vEpiphyte]

@clenk Is there anything an outside contributor can do to help updating the antlr4 version? Would a PR which has the files regenerated from an updated grammar be accepted?

[rpiazza]

@clenk - @vEpiphyte is volunteering to update antlr4! Can you follow thru with him?

[clenk]

@vEpiphyte Yes, a PR like that would be great! There's some documentation on how to do it here: https://stix2-patterns.readthedocs.io/en/latest/developers.html#updating-the-grammar and you can see the last PR that updated it here: #85

Hmm, that page on the docs uses -Dlanguage=Python2 but should be updated to use -Dlanguage=Python3 instead.

[chisholm]

You might also run unit tests on dependent libraries to ensure there were no changes which broke them. Two in the oasis-open organization include cti-python-stix2 and cti-pattern-matcher.

[vEpiphyte]

@clenk and @chisholm PR is up for this in #96