Keyboard Shortcuts
ctrl + shift + ? :
Show all keyboard shortcuts
ctrl + g :
Navigate to a group
ctrl + shift + f :
Find
ctrl + / :
Quick actions
esc to dismiss
Likes
Search
Import Raw OSM file not working in build#204
|
For the last two weekly changeset files, I have receive a error when trying to import the update. The most recent file I attempted to import is changesets-241118.osm.bz2 - I also attempted both the changeset and full database for 241111.
The debug log shows the following:
OsmXmlSegmenter: minHeapRequired=991531972 free=8031321136
Unable to open OSM Map data file: java.lang.NoClassDefFoundError: org/apache/commons/io/input/CloseShieldInputStream ? ? at org.ka2ddo.yaac.osm.OsmXmlSegmenter.parse(OsmXmlSegmenter.java:241) ? ? at org.ka2ddo.yaac.core.provider.CoreProvider$FileOSMImport$OSMParseThread.run(CoreProvider.java:1377) ? ? at java.base/java.lang.Thread.run(Thread.java:840) Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.CloseShieldInputStream ? ? at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) ? ? at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592) ? ? at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ? ? ... 3 more Was there a change in the file format or something else that is causing this error? |
开云体育Greetings.
Regarding your import issues, a?couple of things here:
1. YAAC ignores changeset information, as all it is is documentation of the changes made to the OSM dataset (who/when/why), not the actual changes themselves. So, importing it does nothing. YAAC can't handle deltas
of the OSM data; it can only handle snapshots of the state of the data, where unchanged old data is present along with the new modifications.
2. The error you are getting implies that the BZ2 decompressor is failing. I'm not sure why that would be happening, but I don't test the XML importer that often (I usually use the PBF file format as that is significantly
faster). I did apply a maintenance upgrade to the Apache compress library in YAAC as of build#202. Were you able to import planet-241118.osm.pbf? And were you able to do it before with an older build of YAAC (pre-202)? And what version of the Java runtime
were you using? I recall some versions that used weak references to the jar file indexes, such that it could lose track of a class in the jar file and not go back to look it up again. And the heavy heap thrashing caused by the XML parser could cause "stale"
classes to be removed when they weren't actiually stale.
Let me know what you find out.
Andrew, KA2DDO
author of YAAC
From: [email protected] <[email protected]> on behalf of Mark KE8NHT Ohio via groups.io <ke8nht@...>
Sent: Monday, November 25, 2024 1:30:52 PM For the last two weekly changeset files, I have receive a error when trying to import the update. The most recent file I attempted to import is changesets-241118.osm.bz2 - I also attempted both the changeset and full database for 241111.
The debug log shows the following:
OsmXmlSegmenter: minHeapRequired=991531972 free=8031321136
Unable to open OSM Map data file: java.lang.NoClassDefFoundError: org/apache/commons/io/input/CloseShieldInputStream ? ? at org.ka2ddo.yaac.osm.OsmXmlSegmenter.parse(OsmXmlSegmenter.java:241) ? ? at org.ka2ddo.yaac.core.provider.CoreProvider$FileOSMImport$OSMParseThread.run(CoreProvider.java:1377) ? ? at java.base/java.lang.Thread.run(Thread.java:840) Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.CloseShieldInputStream ? ? at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) ? ? at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592) ? ? at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ? ? ... 3 more Was there a change in the file format or something else that is causing this error?
|
开云体育A followup to my previous reply:
I found that the upgraded commons-compress library slipped in a new unmentioned dependency on the commons-io library that YAAC wasn't using yet, hence the ClassNotFoundException you were receiving. Very easy for me to replicate. The fix was
easy (add the commons-io library to the YAAC distribution), but I'm running a full import of the osm.bz2 file to look for and fix any other accrued issues (found some excessive logging that was impacting performance significantly). This will be fixed in the
next build of YAAC.
Andrew, KA2DDO
author of YAAC
From: [email protected] <[email protected]> on behalf of Andrew P. via groups.io <andrewemt@...>
Sent: Monday, November 25, 2024 2:10:35 PM To: [email protected] <[email protected]> Subject: Re: [yaac-users] Import Raw OSM file not working in build#204 Greetings.
Regarding your import issues, a?couple of things here:
1. YAAC ignores changeset information, as all it is is documentation of the changes made to the OSM dataset (who/when/why), not the actual changes themselves. So, importing it does nothing. YAAC can't handle deltas
of the OSM data; it can only handle snapshots of the state of the data, where unchanged old data is present along with the new modifications.
2. The error you are getting implies that the BZ2 decompressor is failing. I'm not sure why that would be happening, but I don't test the XML importer that often (I usually use the PBF file format as that is significantly
faster). I did apply a maintenance upgrade to the Apache compress library in YAAC as of build#202. Were you able to import planet-241118.osm.pbf? And were you able to do it before with an older build of YAAC (pre-202)? And what version of the Java runtime
were you using? I recall some versions that used weak references to the jar file indexes, such that it could lose track of a class in the jar file and not go back to look it up again. And the heavy heap thrashing caused by the XML parser could cause "stale"
classes to be removed when they weren't actiually stale.
Let me know what you find out.
Andrew, KA2DDO
author of YAAC
From: [email protected] <[email protected]> on behalf of Mark KE8NHT Ohio via groups.io <ke8nht@...>
Sent: Monday, November 25, 2024 1:30:52 PM For the last two weekly changeset files, I have receive a error when trying to import the update. The most recent file I attempted to import is changesets-241118.osm.bz2 - I also attempted both the changeset and full database for 241111.
The debug log shows the following:
OsmXmlSegmenter: minHeapRequired=991531972 free=8031321136
Unable to open OSM Map data file: java.lang.NoClassDefFoundError: org/apache/commons/io/input/CloseShieldInputStream ? ? at org.ka2ddo.yaac.osm.OsmXmlSegmenter.parse(OsmXmlSegmenter.java:241) ? ? at org.ka2ddo.yaac.core.provider.CoreProvider$FileOSMImport$OSMParseThread.run(CoreProvider.java:1377) ? ? at java.base/java.lang.Thread.run(Thread.java:840) Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.CloseShieldInputStream ? ? at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) ? ? at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592) ? ? at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ? ? ... 3 more Was there a change in the file format or something else that is causing this error?
|
|
Thanks for the update. I am 99% through importing the current pbf. I appreciate the information abou the different file types and, especially, that the changeset files weren't actually adding any information to the local dataset. I hadn't read through the definitions carefully, just assumed that the changeset would load anything that had changed since the last release. It still takes over a day to load, but it's only needed when I want to bring the maps up to current status.
?
YAAC is a great tool. Thanks very much for your work on it! |
to navigate to use esc to dismiss