Skip to content

improve performance of spatial.import() for large data sets #385

Description

@jamesdhope

I ran an import of the London OSM data from Planet on a 2 Core machine with 4GIG RAM. OSM Nodes and Ways imported within 3-4 hours. After 30 hours Relations were still being imported. In the end I had to kill the process.

It was reassuring that the import ran without memory overflow on such a small machine (possibly using SWAP i'm not sure) but the speed at which relations were being imported was the main issue for us and would likely be too slow to maintain a large OSM dataset. Indeed, one option would be to run this on a larger machine and further testing would be required to see if this would be performant enough for our needs.

It would be good to see if further optimisation of the import routine could be made to improve performance particularly with regard to OSM Relations.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions