Deal of the Day

Home » Main » Manning Forums » 2009 » PostGIS in Action

Thread: [CH01] 1.5 - Error loading into geog type on cmd line

Reply to this Thread Reply to this Thread Search Forum Search Forum Back to Thread List Back to Thread List

Permlink Replies: 13 - Pages: 1 - Last Post: Jul 19, 2011 2:47 PM by: regina.leo
locnguyen

Posts: 3
From: Orange County, CA
Registered: 7/18/11
[CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 18, 2011 3:27 PM
  Click to reply to this thread Reply

I run this command in Terminal but get a malloc error from shp2pgsql:

hackint0sh:Desktop locnguyen$ shp2pgsql -G -g geog /Users/locnguyen/Desktop/data/roadtrl020.shp ch01.roads_geog | psql -h localhost -U locnguyen -p 5432 -d testgis

Result:

shp2pgsql(656) malloc: *** error for object 0x7fff5fbff570: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

I'm a total n00b to PostGIS and GIS in general so I'm a little lost as to what the problem might be.

rhurlin

Posts: 9
From: Göttingen / Germany
Registered: 6/3/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 18, 2011 3:58 PM   in response to: locnguyen in response to: locnguyen
  Click to reply to this thread Reply

I can confirm this behaviour for the command line version of shp2pgsql on FreeBSD operating system, version 9.0-CURRENT amd64 (www.freebsd.org).

When trying to convert a shapefile with shp2pgsql from database/postgis version 1.5.2 it core dumps when using the two options '-G' and '-g' at the same time. This error is reproducable with other shapefiles, too.

----------------------
shp2pgsql -G -g geog_newname world.shp
shp2pgsql: (malloc) /usr/src/lib/libc/stdlib/malloc.c:4287: Failed assertion: "arena != NULL"
Abbruch(core dumped)
----------------------

It seems there is a problem with memory allocation. This does not happen, if '-G' is used without the '-g' option and it does not happen when using the gui version shp2pgsql-gui.

Rainer

locnguyen

Posts: 3
From: Orange County, CA
Registered: 7/18/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 18, 2011 10:28 PM   in response to: rhurlin in response to: rhurlin
  Click to reply to this thread Reply

Thank you for the help. I hope this bug has been reported?

rhurlin

Posts: 9
From: Göttingen / Germany
Registered: 6/3/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 1:17 AM   in response to: locnguyen in response to: locnguyen
  Click to reply to this thread Reply

No, I did not. But you are right, we should do.

As far as I can see in the loader sources there is a maintainer address. I will give it a try in the next hours.

Rainer

regina.leo

Posts: 231
Registered: 5/6/09
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 6:00 AM   in response to: locnguyen in response to: locnguyen
  Click to reply to this thread Reply

Which version of 1.5 shp2pgsql are you running? I recall this being a fixed issue with upcoming 2.0, but don't recall it being an issue with 1.5 shp2pgsql

http://trac.osgeo.org/postgis/ticket/659

To verify which shp2pgsql you are running, just call it from command line with no args or with --h

like

shp2pgsql

Thanks,
Regina

rhurlin

Posts: 9
From: Göttingen / Germany
Registered: 6/3/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 6:05 AM   in response to: regina.leo in response to: regina.leo
  Click to reply to this thread Reply

On FreeBSD as I wrote before, PostGIS version 1.5.2

shp2pgsql: illegal option -- h
RCSID: $Id: shp2pgsql-core.h 5983 2010-09-19 11:27:05Z mcayland $ RELEASE: 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1

Rainer

regina.leo

Posts: 231
Registered: 5/6/09
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 6:37 AM   in response to: rhurlin in response to: rhurlin
  Click to reply to this thread Reply

Hmm okay. I suspect its the same issue as we had in 2.0. I hadn't thought the issue existed in 1.5, but perhaps it just didn't exhibit itself under windows. I think some other fixes were made in 1.5.3 so you might want to try that. That was released about 3 weeks ago. In the mean time I add the link to this thread on the original ticket I had posted for 2.0 that I suspect is the same issue.

rhurlin

Posts: 9
From: Göttingen / Germany
Registered: 6/3/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 6:49 AM   in response to: regina.leo in response to: regina.leo
  Click to reply to this thread Reply

I just tried PostGIS 1.5.3, see

# SELECT postgis_full_version();
POSTGIS="1.5.3" GEOS="3.3.0-CAPI-1.7.0" PROJ="Rel. 4.7.1, 23 September 2009" LIBXML="2.7.8" USE_STATS (procs from 1.5 r5976 need upgrade)

It seems there is some upgrade work to do inside the database ...


The loader gives me exact the same error as in 1.5.2. As far as I can see its even exact the same version as in 1.5.2 :-(

# shp2pgsql -h
RCSID: $Id: shp2pgsql-core.h 5983 2010-09-19 11:27:05Z mcayland $ RELEASE: 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1

Rainer

regina.leo

Posts: 231
Registered: 5/6/09
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 10:31 AM   in response to: rhurlin in response to: rhurlin
  Click to reply to this thread Reply

Okay I'll ticket this bug. The stamp only records when shp2pgsql-core.h is changed, but sadly other parts aren't captured so the timestamp is not absolutely definitive for micro releases. More a rule of thumb.

rhurlin

Posts: 9
From: Göttingen / Germany
Registered: 6/3/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 11:45 AM   in response to: regina.leo in response to: regina.leo
  Click to reply to this thread Reply

Thanks for the ticket. I have to mention that I also wrote a mail to the maintainer Paul Ramsey today. No answer so far.

I made a quick diff between loader section of 1.5.2 and 1.5.3 sources. Indeed there are some 'micro' changes:

-------------------------------------------------------------------
diff -Naur postgis-1.5.2/loader postgis-1.5.3/loader
diff -Naur postgis-1.5.2/loader/Makefile postgis-1.5.3/loader/Makefile
--- postgis-1.5.2/loader/Makefile 2010-09-27 17:19:08.000000000 +0200
+++ postgis-1.5.3/loader/Makefile 2011-06-25 18:29:50.000000000 +0200
@@ -16,8 +16,8 @@
# the INSTALL and DESTDIR variables so we can get the correct install paths.
# Hence we include the PGXS Makefile here, but ensure that we override the
# 'all' and 'install' targets with the ones we really want to use below.
-PG_CONFIG = /usr/local/pgsql/8.3/bin/pg_config
-PGXS := /usr/local/pgsql/8.3/lib/pgxs/src/makefiles/pgxs.mk
+PG_CONFIG = /usr/local/pgsql/9.1/bin/pg_config
+PGXS := /usr/local/pgsql/9.1/lib/pgxs/src/makefiles/pgxs.mk
include $(PGXS)

# Set CFLAGS afer PGXS, otherwise it will get overwritten with the PGXS
@@ -31,8 +31,8 @@
SHP2PGSQL-GUI=shp2pgsql-gui

# PostgreSQL frontend CPPFLAGS and LDFLAGS (for compiling and linking with libpq)
-PGSQL_FE_CPPFLAGS=-I/usr/local/pgsql/8.3/include
-PGSQL_FE_LDFLAGS=-L/usr/local/pgsql/8.3/lib -lpq
+PGSQL_FE_CPPFLAGS=-I/usr/local/pgsql/9.1/include
+PGSQL_FE_LDFLAGS=-L/usr/local/pgsql/9.1/lib -lpq

# iconv flags
ICONV_LDFLAGS= -liconv
diff -Naur postgis-1.5.2/loader/shp2pgsql-cli.c postgis-1.5.3/loader/shp2pgsql-cli.c
--- postgis-1.5.2/loader/shp2pgsql-cli.c 2010-03-22 20:38:40.000000000 +0100
+++ postgis-1.5.3/loader/shp2pgsql-cli.c 2010-12-14 05:57:59.000000000 +0100
@@ -1,5 +1,5 @@
/**********************************************************************
- * $Id: shp2pgsql-cli.c 5451 2010-03-22 19:38:40Z pramsey $
+ * $Id: shp2pgsql-cli.c 6373 2010-12-14 04:57:59Z pramsey $
*
* PostGIS - Spatial Types for PostgreSQL
* http://postgis.refractions.net
@@ -31,17 +31,18 @@
printf(" -c Creates a new table and populates it, this is the\n");
printf(" default if you do not specify any options.\n");
printf(" -p Prepare mode, only creates the table.\n");
- printf(" -g <geocolumn> Specify the name of the geometry/geography column\n");
- printf(" (mostly useful in append mode).\n");
- printf(" -D Use postgresql dump format (defaults to SQL insert statments.\n");
+ printf(" -g <geocolumn> Specify the name of the geometry/geography column.\n");
+ printf(" (mostly useful in append mode).\n");
+ printf(" -D Use postgresql dump format (defaults to SQL insert statments).\n");
printf(" -G Use geography type (requires lon/lat data).\n");
printf(" -k Keep postgresql identifiers case.\n");
printf(" -i Use int4 type for all integer dbf fields.\n");
printf(" -I Create a spatial index on the geocolumn.\n");
printf(" -S Generate simple geometries instead of MULTI geometries.\n");
+ printf(" -w Output WKT format (drops M and introduces coordinate drifts).\n");
printf(" -W <encoding> Specify the character encoding of Shape's\n");
- printf(" attribute column. (default : \"WINDOWS-1252\")\n");
- printf(" -N <policy> NULL geometries handling policy (insert*,skip,abort)\n");
+ printf(" attribute column. (default : \"WINDOWS-1252\").\n");
+ printf(" -N <policy> NULL geometries handling policy (insert*,skip,abort).\n");
printf(" -n Only import DBF file.\n");
printf(" -? Display this help screen.\n");
}
diff -Naur postgis-1.5.2/loader/shp2pgsql-core.c postgis-1.5.3/loader/shp2pgsql-core.c
--- postgis-1.5.2/loader/shp2pgsql-core.c 2010-09-19 18:54:59.000000000 +0200
+++ postgis-1.5.3/loader/shp2pgsql-core.c 2010-12-13 21:42:47.000000000 +0100
@@ -1,5 +1,5 @@
/**********************************************************************
- * $Id: shp2pgsql-core.c 5989 2010-09-19 16:54:59Z mcayland $
+ * $Id: shp2pgsql-core.c 6361 2010-12-13 20:42:47Z pramsey $
*
* PostGIS - Spatial Types for PostgreSQL
* http://postgis.refractions.net
@@ -1587,6 +1587,7 @@
char *geometry=NULL, *ret;
char *utf8str;
int res, i;
+ int rv;

/* Clear the stringbuffers */
sbwarn = stringbuffer_create();
@@ -1655,7 +1656,8 @@
{
case FTInteger:
case FTDouble:
- if (-1 == snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i)))
+ rv = snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i));
+ if (rv >= MAXVALUELEN || rv == -1)
{
vasbappend(sbwarn, "Warning: field %d name truncated\n", i);
val[MAXVALUELEN - 1] = '\0';
@@ -1676,7 +1678,8 @@
case FTString:
case FTLogical:
case FTDate:
- if (-1 == snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i)))
+ rv = snprintf(val, MAXVALUELEN, "%s", DBFReadStringAttribute(state->hDBFHandle, item, i));
+ if (rv >= MAXVALUELEN || rv == -1)
{
vasbappend(sbwarn, "Warning: field %d name truncated\n", i);
val[MAXVALUELEN - 1] = '\0';

regina.leo

Posts: 231
Registered: 5/6/09
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 12:14 PM   in response to: rhurlin in response to: rhurlin
  Click to reply to this thread Reply

I wouldn't call Paul THE maintainer. It's maintained by all of the PostGIS development crew so the proper protocol for these type of things is to submit a bug ticket via PostGIS bug tracker.

http://trac.osgeo.org/postgis/

Or a question on Postgis news group if you are not sure it's a bug.

http://www.postgis.org/support/

Paul responds more quickly to those than his own email :)

rhurlin

Posts: 9
From: Göttingen / Germany
Registered: 6/3/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 12:29 PM   in response to: regina.leo in response to: regina.leo
  Click to reply to this thread Reply

Ahh, I guessed so. OK, next time I will first ask on the mailing list.

In ticket #1132 you only refer to my report for FreeBSD. The original poster Loc Nguyen seems to be on Mac OSX or something like that ...

locnguyen

Posts: 3
From: Orange County, CA
Registered: 7/18/11
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 1:37 PM   in response to: rhurlin in response to: rhurlin
  Click to reply to this thread Reply

Indeed I'm running Mac OS X. Running shp2pgsql without arguments:

hackint0sh:~ locnguyen$ shp2pgsql
RCSID: $Id: shp2pgsql-core.h 5983 2010-09-19 11:27:05Z mcayland $ RELEASE: 1.5 USE_GEOS=1 USE_PROJ=1 USE_STATS=1


If it helps I'm running 1.5.2

POSTGIS="1.5.2" GEOS="3.2.2-CAPI-1.6.2" PROJ="Rel. 4.7.1, 23 September 2009" LIBXML="2.7.3" USE_STATS

regina.leo

Posts: 231
Registered: 5/6/09
Re: [CH01] 1.5 - Error loading into geog type on cmd line
Posted: Jul 19, 2011 2:47 PM   in response to: locnguyen in response to: locnguyen
  Click to reply to this thread Reply

Okay I've updated. I'll have to try again on windows but not near my computer with shp2pgsql

Legend
Gold: 300 + pts
Silver: 100 - 299 pts
Bronze: 25 - 99 pts
Manning Author
Manning Staff
Manning Developmental Editor