feat(osm): 🎉 extract and visualize OSM data.#93
Draft
jhw-db wants to merge 3 commits into
Draft
Conversation
A new tool allows to extract, clip, project and plot OSM data and is used for railway networks (strictly it is not limited to it). For extraction the user specifies a bounding box or a polygon. In its interior the specified OSM data is extracted. The Overpass API is used with a recursive query to follow e.g. railroad tracks starting in the interior, but ending outside the extraction area. Also this will take intermediate data nodes into account. Therefore, data might extend beyond the specified extraction area. Clipping functionality is provided to reduce the the amount of data extending from the extraction aread. It is tested which nodes and edges lie in the extraction area. If an edge starts in the interior, travels outside and ends in the interior again it will be kept. Also if the edge starts in the interior and ends in the exterior it is still kept. When trying to visualize the data it might be disorted due to the missing projection. It is possible to apply elliptic Mercator projection to the data which will add X and Y coordinates to the data. For visualization a plotting capability based on MatplotLib is introduced. Currently it assumes that it is used for visualization of railway data. Switches and buffer stop are separately marked by a symbol. Control of the functionality is done using a CLI. Further information is available in the provided README.md Unit tests and integration tests are provided leading to an overall coverage roughly greater than 80%. The existing project configuration (pyproject.toml) is extended to treat the OSM extractor as a separate package with a cli. During the test with PyLint it complied about the identation even ruff had formatted the files. Therefore, configuration was adopted to work consistently to ruff.
Contributor
|
@jhw-db sorry didn't find time before. Meantime you might have to rebase to main. But yes we definetly need to have this tool ! |
PopUpSim needs topology of the railway network for e.g. shunting operations and definition of yards. This data is normally provided by dedicated systems within railway companies, but this data is generally not freely available. PopUpSim will use its own data format internally to represent tracks, track sequences, switches and so on. Since currently no system providing the necessary data is available, a tool is provided allowing to extract data from openrailwaymap. The user specifies a bounding box or polygon in which railway data is extracted. By using the CLI it is possible to configure also if unused tracks are filtered out. Extraction of the data fully relies on the overpass API. Data is returned as a JSON with nodes and edges describing the topology. At nodes or edges tags maybe present for additional information like if the node is a switch. During extraction a recursive query is send to the overpass API. This is necessary since nodes outside the extraction area are missing which lead to missing edges (one node lies in the inside and one outside). The railway network might extend the defined extraction area. Therefore, in a next step a clipping is done. The extracted and clipped coordinates are still given in lat/lon. A projection using the full elliptical mercator projection is applied for visualization. A plotting tool (called plotter) is shipped with the osm2extractor to visualize the extracted data using matplotlib. The possible commandline arguments fo the cli are described in the provided README.md PopUpSim uses an own format. A provided tool called osm2popupsim will do the conversion. It reads the raw file and creates the reformatted topology file (sill node edge represnetation). Additionally a tracks file is created which virtually a sequence of edges to a track. A track can have a name, type and has an id. The user can assign functionality like parking, wretrofit and so on for each of these tracks. Visualization of the PopUpSim data is provided using the via. It is possible to visualize the nodes, edges and switches or show coloured the type of assigned functionality to the tracks. Output is done using Folium to show the data as an overlay to the OSM data. The full arguments of the osm2popupsim are described in the sipped README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A new tool allows to extract, clip, project and plot OSM data and is used for railway networks (strictly it is not limited to it).
For extraction the user specifies a bounding box or a polygon. In its interior the specified OSM data is extracted. The Overpass API is used with a recursive query to follow e.g. railroad tracks starting in the interior, but ending outside the extraction area. Also this will take intermediate data nodes into account. Therefore, data might extend beyond the specified extraction area.
Clipping functionality is provided to reduce the the amount of data extending from the extraction aread. It is tested which nodes and edges lie in the extraction area. If an edge starts in the interior, travels outside and ends in the interior again it will be kept. Also if the edge starts in the interior and ends in the exterior it is still kept.
When trying to visualize the data it might be disorted due to the missing projection. It is possible to apply elliptic Mercator projection to the data which will add X and Y coordinates to the data.
For visualization a plotting capability based on MatplotLib is introduced. Currently it assumes that it is used for visualization of railway data. Switches and buffer stop are separately marked by a symbol.
Control of the functionality is done using a CLI. Further information is available in the provided README.md
Unit tests and integration tests are provided leading to an overall coverage roughly greater than 80%.
The existing project configuration (pyproject.toml) is extended to treat the OSM extractor as a separate package with a cli. During the test with PyLint it complied about the identation even ruff had formatted the files. Therefore, configuration was adopted to work consistently to ruff.