Categories
Ask Development Geography SQL

Is there an example SQLAlchemy UserDefinedType for Microsoft SQL Server Geography/Geometry Types?

How can I create a SQLAlchemy UserDefinedType that will allow me to insert into a Geography data type on SQL Server?

I am using Python 3.6 and Pandas to_sql to write into a SQL Server table that will have a column with a geography data type. I am using SQLAlchemy.create_engine to create a database connection to SQLExpress using DRIVER={ODBC Driver 13 for SQL Server}. I have polygons stored in a GeoPandas dataframe.

SQLAlchemy 1.3.10 does not directly provide support for Geography or Geometry data types; and, GeoAlchemy2 does not support MS SQL Server. I have been trying to use SQLAlchemy’s UserDefinedType to see if I can get something that results in:

GEOGRAPHY::STGeomFromText(GEOMETRY::STGeomFromText('POLYGON((-110.17315242968752 52.66767554218751,-110.18536282187502 52.66770066015627,-110.19718901640618 52.66771763203127,-110.197593865625 52.667718411718795,-110.19747227656248 52.67594785000003,-110.1732282007812 52.67592660234379,-110.17315242968752 52.66767554218751))',4269).MakeValid().STUnion(GEOMETRY::STGeomFromText('POLYGON((-110.17315242968752 52.66767554218751,-110.18536282187502 52.66770066015627,-110.19718901640618 52.66771763203127,-110.197593865625 52.667718411718795,-110.19747227656248 52.67594785000003,-110.1732282007812 52.67592660234379,-110.17315242968752 52.66767554218751))',4269).STStartPoint()).STAsText(),4269)

So far I have:

class Geography(UserDefinedType):

    def get_col_spec(self):
        return "GEOGRAPHY"

    def bind_processor(self, dialect):
        def process(value):
            if value is None:
                return None
            return 'GEOGRAPHY::STGeomFromText(GEOMETRY::STGeomFromText({0},4269).MakeValid().STUnion(GEOMETRY::STGeomFromText({0},4269).STStartPoint()).STAsText(),4269)'.format("'" + value + "'")
        return process

I am stuck at this result though (note the quotes surrounding the whole thing.:

"GEOGRAPHY::STGeomFromText(GEOMETRY::STGeomFromText('POLYGON ((-110.1731524296875 52.66767554218751, -110.185362821875 52.6677006601563, -110.19718901 ... (382 characters truncated) ...  -110.1974722765625 52.67594785, -110.1732282007812 52.67592660234379, -110.1731524296875 52.66767554218751))',4269).STStartPoint()).STAsText(),4269)"

I know that this will work is SQL:

DECLARE @g NVARCHAR(MAX)
SELECT @g = 'POLYGON((-110.17315242968752 52.66767554218751,-110.18536282187502 52.66770066015627,-110.19718901640618 52.66771763203127,-110.197593865625 52.667718411718795,-110.19747227656248 52.67594785000003,-110.1732282007812 52.67592660234379,-110.17315242968752 52.66767554218751))'
INSERT INTO dbo.[Spatial_Table] ([geometry]) 
VALUES (GEOGRAPHY::STGeomFromText(GEOMETRY::STGeomFromText(@g,4269).MakeValid().STUnion(GEOMETRY::STGeomFromText(@g,4269).STStartPoint()).STAsText(),4269))

Using pandas.DataFrame.to_sql results in an error:

DataError: (pyodbc.DataError) ('22018', '[22018] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Operand type clash: ntext is incompatible with geography (206) (SQLExecDirectW); [22018] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Statement(s) could not be prepared. (8180)')
[SQL: INSERT INTO dbo.[Spatial_Table] (geometry) VALUES (?)]
[parameters: (("GEOGRAPHY::STGeomFromText(GEOMETRY::STGeomFromText('POLYGON ((-114.4742908039062 51.94055257031255, -114.4623163671875 51.94054674921875, -114.450628 ... (3922 characters truncated) ... .4978060703125 51.9479693648438, -114.4743004117187 51.94797611093753, -114.4742908039062 51.94055257031255))',4269).STStartPoint()).STAsText(),4269)",), ("GEOGRAPHY::STGeomFromText(GEOMETRY::STGeomFromText('POLYGON ((-112.1157004828125 49.53477394218754, -112.1156657703125 49.52339953203125, -112.124386 ... (704 characters truncated) ... 1212683476562 49.53115973984376, -112.1212804195312 49.53477826953127, -112.1157004828125 49.53477394218754))',4269).STStartPoint()).STAsText(),4269)",))]
(Background on this error at: http://sqlalche.me/e/9h9h)

I am presuming the error is due to a the surrounding quotes causing a string, and not the functions being passed to SQL.

I have tried using sqlalchemy.sql.expression.text on the bind_processor return string but I get this error:

ProgrammingError: (pyodbc.ProgrammingError) ('Invalid parameter type.  param-index=0 param-type=TextClause', 'HY105')
[SQL: INSERT INTO [Spatial_Table] (geometry) VALUES (?)]
[parameters: (<sqlalchemy.sql.elements.TextClause object at 0x000002566D5A40F0>,)]
(Background on this error at: http://sqlalche.me/e/f405)

And I have looked at sqlalchemy.sql.expression.func; but, I don’t know how to use this with the complex GEOGRAPHY::STGeomFromText method.

My presumption may be incorrect though, as this will work in SQL:

INSERT INTO dbo.[Spatial_Table] ([geometry]) 
VALUES ('POLYGON((-110.17315242968752 52.66767554218751,-110.18536282187502 52.66770066015627,-110.19718901640618 52.66771763203127,-110.197593865625 52.667718411718795,-110.19747227656248 52.67594785000003,-110.1732282007812 52.67592660234379,-110.17315242968752 52.66767554218751))')

And simply using:

class Geography(sqlalchemy.types.UserDefinedType):

    def get_col_spec(self):
        return "GEOGRAPHY"

Will result in this error, even though the parameters seem to be correct. It is still getting the same operand type clash error:

DataError: (pyodbc.DataError) ('22018', '[22018] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Operand type clash: ntext is incompatible with geography (206) (SQLExecDirectW); [22018] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Statement(s) could not be prepared. (8180)')
[SQL: INSERT INTO [Spatial_Table] (geometry) VALUES (?)]
[parameters: ('POLYGON ((-114.4742908039062 51.94055257031255, -114.4623163671875 51.94054674921875, -114.4506284421875 51.94053819687502, -114.4506230351562 51.933 ... (1739 characters truncated) ... -114.49781949375 51.93693783750001, -114.4978060703125 51.9479693648438, -114.4743004117187 51.94797611093753, -114.4742908039062 51.94055257031255))',)]
(Background on this error at: http://sqlalche.me/e/9h9h)

Here is a full example:

import geopandas
import pyodbc
import urllib
import sqlalchemy

params = 'DRIVER={ODBC Driver 13 for SQL Server};' \
         'SERVER=ServerName;' \
         'PORT=1433;' \
         'DATABASE=DatabaseName;' \
         'trusted_connection=yes;'

params = urllib.parse.quote_plus(params)
db = sqlalchemy.create_engine('mssql+pyodbc:///?odbc_connect=%s' % params)


class Geography(sqlalchemy.types.UserDefinedType):

    def get_col_spec(self):
        return "GEOGRAPHY"

    def bind_processor(self, dialect):
        def process(value):
            if value is None:
                return None
            return 'GEOGRAPHY::STGeomFromText(GEOMETRY::STGeomFromText({0},4269).MakeValid().STUnion(GEOMETRY::STGeomFromText({0},4269).STStartPoint()).STAsText(),4269)'.format("'" + value + "'")
        return process


gdf = geopandas.GeoDataFrame({'geometry':['POLYGON ((-114.4742908039062 51.94055257031255, -114.4623163671875 51.94054674921875, -114.4506284421875 51.94053819687502, -114.4506230351562 51.93329010000002, -114.4506172351562 51.92605222890626, -114.4270012117187 51.92605734296876, -114.42699041875 51.91881815312502, -114.4269793164062 51.91139827421875, -114.4150635234375 51.91142824140627, -114.4150633671875 51.90417951171878, -114.41506316875 51.89693553984375, -114.403441165625 51.89697914921879, -114.3914711632812 51.89696492578128, -114.3914339117187 51.88973628046875, -114.3913953804687 51.88226671015627, -114.3677831210937 51.88229111953126, -114.3677989414062 51.87505186875001, -114.3678146921875 51.86781865390628, -114.3678270578125 51.86058223671876, -114.3678396773437 51.8531650226563, -114.3561627734375 51.85316232265626, -114.34419459375 51.85315451953124, -114.34420228125 51.8459314015625, -114.3442099304687 51.83869999843751, -114.3561678164062 51.83869954843755, -114.3561729164063 51.83145872656252, -114.3561489476562 51.82404763359375, -114.3678250734375 51.82404504843754, -114.367827496875 51.82784562734378, -114.3678343414063 51.83869510312502, -114.3797993828125 51.83869063437504, -114.3797996945312 51.84592788906252, -114.3798041078125 51.85316721640629, -114.3914338414062 51.85316111093755, -114.3914139953125 51.86780720156253, -114.4033583140625 51.86778718046878, -114.4150253679687 51.86777429218751, -114.415020278125 51.8822367257813, -114.432806471875 51.88222315703126, -114.4505780710937 51.882230434375, -114.450589053125 51.88963689218753, -114.450594271875 51.89689434218752, -114.4622547703125 51.89689832265628, -114.4622642132812 51.90414635312504, -114.4622736234375 51.91139415781254, -114.4742338164062 51.91140009453125, -114.4742472351562 51.91883031875, -114.4742673734375 51.92607473984378, -114.4858996257812 51.92606931250003, -114.4978195835938 51.92606766953128, -114.49781949375 51.93693783750001, -114.4978060703125 51.9479693648438, -114.4743004117187 51.94797611093753, -114.4742908039062 51.94055257031255))']})

gdf.to_sql('Spatial_Table',
           if_exists='replace',
           index=False,
           dtype={'geometry': Geography},
           con=db)

I would love to see direct support for Geography and Geometry for MS SQL Server in SQLAlchemy, or GeoAlchemy.

Leave a Reply

Your email address will not be published. Required fields are marked *