HANA and Harnessing the Power of Spatial – SAP + Esri Really is Better Together! PART 4
On Part 3 of this blog we gave you more technical detail on HANA Spatial, what it is, how it is used and why it is needed in the first place.
On this Part 4, we dive into data transformation and migration and look at how SAP and Esri work together. We also give you some indication on when you need to use a more advanced tools like what Esri provides instead of just using what SAP Spatial provides you for.
HANA – Transformation and Access
HANA environment has a whole lot of old and new built-in tools for moving and transforming data from databases, file systems, cloud, feeds, devices and from HANA - consider these diagrams:
On this Part 4, we dive into data transformation and migration and look at how SAP and Esri work together. We also give you some indication on when you need to use a more advanced tools like what Esri provides instead of just using what SAP Spatial provides you for.
HANA – Transformation and Access
HANA environment has a whole lot of old and new built-in tools for moving and transforming data from databases, file systems, cloud, feeds, devices and from HANA - consider these diagrams:
We can move data between databases (typically to HANA from other databases) using all the traditional SAP transformation tools like:
And of course, we now also have:
|
The top diagram is how data transformation used to be done in SAP, the bottom diagram is the new provisioning tool SAP has built for transforming data in/out HANA. With this SAP HANA Data Provisioning services tool there is no need to license additional products (like those on first diagram).
Smart Data Quality tool includes Directories that users can purchase like geocoding, reverse geocoding and (address) validation services, cost for a country (like NZ) is typically a minimal annual fee. The New Zealand version is NZ Post certified and enables your organisation for bulk mail discounting SOA (6 cents per letter).
There are also other ways for you to get help to transform your spatial data into HANA:
For HANA to enable Spatial, you will need either HANA Enterprise Edition or HANA Standard Edition with Spatial & Graph option. HANA can be managed either on-prem or cloud.
Fundamentally there are two ways to use Esri with HANA; as a GeoDB or embedding the Esri processing within HANA. This means SAP products can access spatial in several different ways;
SAP can access and use spatial capability with or without maps:
There is already a wide variety of SAP products using spatial, to name a few core products:
You can also integrate GIS capability with SAP’s Geographical Enablement Framework (GEF - engineered services) against most of the SAP products, to name a few core products:
HANA Spatial Services is a new type of access method (and an application) that provides a series of real-time streaming web services and API feeds for SAP customers. These services provide the best value when streamed on top of a map but can also be used purely to give back an answer.
They enable users to collect, prepare, process and consume spatial data and services. There are new services provided by SAP all the time, the current ones fitting into the following categories:
Weather especially is becoming a big business and real-time services like these can provide a huge additional benefit for customers dealing with location-based markets and industries.
The following diagram collects all the defined data access and transformation concepts together into one coordinated whole:
Smart Data Quality tool includes Directories that users can purchase like geocoding, reverse geocoding and (address) validation services, cost for a country (like NZ) is typically a minimal annual fee. The New Zealand version is NZ Post certified and enables your organisation for bulk mail discounting SOA (6 cents per letter).
There are also other ways for you to get help to transform your spatial data into HANA:
- SAP Database Migration Factory Program is a global program that provides a certified process and partner to migrate customer’s spatial data into SAP HANA Esri GeoDB. Note that Zag is a brand new certified spatial member of this program, one of only four in the whole world and the only one in Asia-Pacific.
- Location Intelligence Quick-Start Experience that provides you everything you need to get started and then continuous weekly webinars with SAP & Esri experts answering all your spatial questions.
- Hackathons provided by SAP and Esri together to showcase the best from both worlds.
For HANA to enable Spatial, you will need either HANA Enterprise Edition or HANA Standard Edition with Spatial & Graph option. HANA can be managed either on-prem or cloud.
Fundamentally there are two ways to use Esri with HANA; as a GeoDB or embedding the Esri processing within HANA. This means SAP products can access spatial in several different ways;
SAP can access and use spatial capability with or without maps:
- via Smart Data Access (SDA).
- with SDI Data Provisioning Agent + ABAP connector.
- using HANA Spatial Services (HSS).
- via Fiori Launchpad.
- on SAPUI5 with Esri JavaScript API (via Web IDE).
There is already a wide variety of SAP products using spatial, to name a few core products:
- SAP Work Manager.
- Service Manager.
- Multi-resource Scheduling.
- SAP Lumira.
You can also integrate GIS capability with SAP’s Geographical Enablement Framework (GEF - engineered services) against most of the SAP products, to name a few core products:
- EAM (Enterprise Asset Management).
- Real Estate (RE/FX).
- CRM (Customer Relationship Management).
HANA Spatial Services is a new type of access method (and an application) that provides a series of real-time streaming web services and API feeds for SAP customers. These services provide the best value when streamed on top of a map but can also be used purely to give back an answer.
They enable users to collect, prepare, process and consume spatial data and services. There are new services provided by SAP all the time, the current ones fitting into the following categories:
- Extreme Weather; REST services for surface index information, weather forecast up to 48 hours and wildfire hazard index for next 10 days.
- Detection & Observation; REST services for object detection from aerial/satellite imagery, Earth observation and mapping APIs.
- Places & Locations; REST services for points of Interest within specific areas, pushing customer logic to GIS business data and CRUD processes for additional custom data layers (via GeoJSON).
- Processing & Management; REST services for post processing web coverage data and image tiles for base maps and GIS customer data process management (like drone image pyramids).
- Modelling & Labelling; APIs for (Band) modelling and labelling new service interfaces that can then be published and enabled as REST services for customers.
Weather especially is becoming a big business and real-time services like these can provide a huge additional benefit for customers dealing with location-based markets and industries.
The following diagram collects all the defined data access and transformation concepts together into one coordinated whole:
On top of all that has already been discussed, we can see that SAP HANA also supports spatial standards (OGC – Open Geospatial Consortium), de-facto vendors like Esri and routing providers HERE and TomTom, open source toolkits GeoTools and GeoServer and even provides additional spatial data globally.
We also have all those real-time spatial services, data and streaming products, powerful analytics in the cloud & enterprise reporting, business applications & frameworks, connected apps, AI & IoT, whole bunch of SAP products already spatialized and wide variety of ecosystems.
So why do we need Esri? Well, a lot of the SAP customers are also Esri customers (in our case almost 70% of our SAP customer are also Esri customers), and these customers use Esri’s powerful super user tools to maintain their core spatial data.
But mostly it is about getting the best from both/all worlds, so let’s look at this by investigating the following three diagrams:
We also have all those real-time spatial services, data and streaming products, powerful analytics in the cloud & enterprise reporting, business applications & frameworks, connected apps, AI & IoT, whole bunch of SAP products already spatialized and wide variety of ecosystems.
So why do we need Esri? Well, a lot of the SAP customers are also Esri customers (in our case almost 70% of our SAP customer are also Esri customers), and these customers use Esri’s powerful super user tools to maintain their core spatial data.
But mostly it is about getting the best from both/all worlds, so let’s look at this by investigating the following three diagrams:
|
The three diagrams all look at the same environment from different perspectives; the top diagram looks at the world from SAP overview perspective, the bottom diagram from Business and Esri perspective and the middle diagram from the communication level perspective. All perspectives showcase how well these two different product suites can be integrated into one uniform system; with one source of truth for data, services, apps and UI.
It is good to understand that SAP does not have to have Esri to enable spatial; SAP HANA is natively spatial and stores the spatial entities using spatial datatypes (based on WKB and derived from Esri shapefile definition). There are also additional tools like HANA Spatial Services (HSS), apps like Geometry Explorer and Geometry Editor with APIs and frameworks to integrate like GEF (Geographical Enablement Framework). Search and navigate are provided within HSS as are HERE-based Maps API for your base maps and routing functionality. Or you can access base maps and search services with any available REST services – Open Street Maps provides some free as do most of the spatial vendors like Google, MapBox, BING and even Esri.
Also consider the following SAP HANA diagram showcasing support for open source:
It is good to understand that SAP does not have to have Esri to enable spatial; SAP HANA is natively spatial and stores the spatial entities using spatial datatypes (based on WKB and derived from Esri shapefile definition). There are also additional tools like HANA Spatial Services (HSS), apps like Geometry Explorer and Geometry Editor with APIs and frameworks to integrate like GEF (Geographical Enablement Framework). Search and navigate are provided within HSS as are HERE-based Maps API for your base maps and routing functionality. Or you can access base maps and search services with any available REST services – Open Street Maps provides some free as do most of the spatial vendors like Google, MapBox, BING and even Esri.
Also consider the following SAP HANA diagram showcasing support for open source:
|
So if you need to boost up your GIS capability in HANA, you are not forced to use Esri – you actually have multiple choices including open source (more on this on Part 5), but Esri is by far the best (easiest) integrated spatial product suite for the SAP environment.
SAP & Esri
So, let’s then look at how SAP and Esri work together in a little bit more detail, starting with some of the functionality that SAP does not support (yet).
The following 7 examples showcase some of the more advanced functionality provided by Esri:
SAP & Esri
So, let’s then look at how SAP and Esri work together in a little bit more detail, starting with some of the functionality that SAP does not support (yet).
The following 7 examples showcase some of the more advanced functionality provided by Esri:
Trends & Insights
Monitoring & Dashboards
Targeted Apps
Network Models & Routing
Topologies & Editing
3D/4D Advanced Functionality
Mixed Reality Enablement
|
These are some of the examples on what Esri can do that SAP currently does not. If you need advanced GIS functionality, then you need to use GIS Desktop tools - in Esri world there are two choices; the legacy product ArcMap and its (eventual) replacement product ArcGIS Professional, see diagram below for what extensions these products support:
ArcMap and ArcGIS Professional are desktop GIS analyst tools; but note that even if customer has no requirement for advanced GIS in their organisation, you will always need at least one ArcGIS professional as it is needed for managing the organisations spatial platform. It is used by the GIS administrators to setup (ETL) and publish Esri Map (image/raster), Feature (2D vector) and Scene (3D vector) services.
As mentioned, ArcMap is a legacy product and will be discontinued at some stage. However even though ArcGIS Professional includes more 3D capability, it is lacking with the 2D extensions - which are client end extensions. It is important to understand that most of them also have server end extensions and when there is no server end extension, then the data is manipulated directly against the data storage rather than through map and feature services. This is relevant as only features served via map/feature services will be available to web/mobile tools and to custom development.
Red text extensions in the left are currently missing from ArcGIS Pro (and some like ArcReader are legacy and will never get converted). Green text extensions in the right are ArcGIS Pro only extensions (new ones). Underlined text client extensions have existing server extensions too.
The following diagram showcases the Esri Capability/Functionality architecture:
As mentioned, ArcMap is a legacy product and will be discontinued at some stage. However even though ArcGIS Professional includes more 3D capability, it is lacking with the 2D extensions - which are client end extensions. It is important to understand that most of them also have server end extensions and when there is no server end extension, then the data is manipulated directly against the data storage rather than through map and feature services. This is relevant as only features served via map/feature services will be available to web/mobile tools and to custom development.
Red text extensions in the left are currently missing from ArcGIS Pro (and some like ArcReader are legacy and will never get converted). Green text extensions in the right are ArcGIS Pro only extensions (new ones). Underlined text client extensions have existing server extensions too.
The following diagram showcases the Esri Capability/Functionality architecture:
The text under the diagram pretty much explains the diagram, but here are a couple of other notes:
Let’s look at some of the ways/formats Esri and SAP use to integrate:
- Desktop apps are for the specialist GIS analysts and include a lot of tools and extensions.
- Web apps can be built by non-specialists using a WYSIWYG (what You See Is What You Get) tool called Web AppBuilder which deploys HTML5 compliant web apps/maps.
- Mobile apps can either be
- built with AppStudio – a cross-platform developer tool that compiles to all mobile formats, or be
- configured on any of the 8 mobile apps provided by Esri.
- There are other web services provided (within the second tier) that should be mentioned:
- Vector Tile Services – reasonably new service providing vector data in a highly compressed and fast way.
- Metadata services – Esri provides metadata services, but there is also an OGC standard called CS-W (Catalogue Services) that is supported by Esri and ALL other GIS vendors.
- OGC (Open Geospatial Consortium) web services are standardised web services that are supported by ALL GIS products. The most relevant are:
- WMS (Web Map Service) – provides raster imagery (like Esri Map Service).
- WMTS (Web Map Tile Service) – provides tiled map images.
- WFS (Web Feature Service) – provides vector data (like Esri Feature Service).
- WCS (Web Coverage Service) – provides 3D land contour.
- 3DPS (3D Portrayal Service) – brand new service (like Esri Scene Service) not supported by vendors yet.
- Your own portal can also store data in Data Store – which is a PostgreSQL PostGIS database embedded within the Esri Portal. It is equal to Esri GeoDB and a very fast way to access data.
- Note that currently HANA does not have any spatial indices in use yet – the database is fast enough without indices, so even though spatial indices are going to be introduced in near future, don’t expect them to make much notable difference to the user experience (0.5 second response time will drop to 0.05 second response time).
- There are a lot of SAP migration tools that support migrating data from most of the spatial databases to HANA, but not the other way around.
Let’s look at some of the ways/formats Esri and SAP use to integrate:
The recommended way for Esri and SAP to integrate is via the Esri RESTful web services; Map Service, Feature Service and Scene Service as these are enabled across all 100+ Esri products.
Better Together
We have already talked about this a bit on previous blogs – but with the “SAP + Esri – Better Together” alliance the idea is to merge these two worlds and provide users from both product suites access to a unified experience; one source of truth for data, services, apps and tools.
In a nutshell the alliance means the following:
Better Together
We have already talked about this a bit on previous blogs – but with the “SAP + Esri – Better Together” alliance the idea is to merge these two worlds and provide users from both product suites access to a unified experience; one source of truth for data, services, apps and tools.
In a nutshell the alliance means the following:
Furthermore, the alliance supports:
|
These benefits can better be explained with this diagram:
Full-on spatial tools provided by Esri on left, embedded spatial provided by SAP on the right, some specific native spatial tools provided by SAP in the middle. Both environments providing a wide variety of web service feeds that can be accessed by both systems. These services accessing data stored in SAP HANA either in separate DB instances linked together via synchronisation/Virtual Data Access or just in one database. Regardless of which storage mechanism is used, all data is seamlessly accessed from one place; as a one source of truth, sharing same branding, look-and-feel, symbology, labelling, colors, styles and fonts.
View an example of a SAP+Esri shared Story Map for 311 call analysis here.
This example embeds a lot of information into a one coherent whole, but you can also look at any of the specific map analysis results separately using the following links:
Other Stuff
There is a little bit more we want to go through; first let’s look at some of the noteworthy server products provided in the Esri suite:
View an example of a SAP+Esri shared Story Map for 311 call analysis here.
This example embeds a lot of information into a one coherent whole, but you can also look at any of the specific map analysis results separately using the following links:
- Noise Complaints - Bivariate Comparison of Seasonal Noise Complaints.
- Calls by Area – Calls by postal Zip code by Season.
- Calls Growth – Proportional Change in Call Volume Annually.
- Seasonal Impact – Predominant Volume of Calls by Season.
- Time of Day Impact – Predominant Volume of Calls by Morning and Afternoon.
- Calls History – Highest Annual Call Volumes.
- Total Calls – Total Call Volumes by Borough and Zip Code.
- Total Summer – Total Summer Call Volumes.
- Total Winter – Total Winter Call Volumes.
Other Stuff
There is a little bit more we want to go through; first let’s look at some of the noteworthy server products provided in the Esri suite:
All the server extensions (on the left-side list) require a client extension – you cannot use them independently. The red text depicts ArcMap (the legacy Esri desktop tool) extensions only, potentially on their way out. Green text depicts ArcGIS Professional client extensions.
There are two server technology stacks; ArcGIS Enterprise and ArcGIS Online, both are architecturally the same. ArcGIS Enterprise gets new versions as per Esri versioning rules – 3 times a year, one major and 2 minor versions. ArcGIS Online gets versions faster and more regularly than Enterprise. Platform architecture can be Hybrid; ArcGIS Enterprise on-prem, ArcGIS Online on-cloud, or ArcGIS Enterprise on Azure or AWS.
Note that included within the ArcGIS Enterprise/Online product is also a Portal that enables data, service and app sharing. Optionally ArcGIS Server/Portal can also include a Data Store that allows storing all vector and raster data within embedded PostgreSQL PostGIS database.
Some major products have been listed also in the screenshot, for example:
Some tools are new – take for example Artificial Intelligence and Machine Learning provided by Esri to help with clustering, classification and predictions, see diagram below:
There are two server technology stacks; ArcGIS Enterprise and ArcGIS Online, both are architecturally the same. ArcGIS Enterprise gets new versions as per Esri versioning rules – 3 times a year, one major and 2 minor versions. ArcGIS Online gets versions faster and more regularly than Enterprise. Platform architecture can be Hybrid; ArcGIS Enterprise on-prem, ArcGIS Online on-cloud, or ArcGIS Enterprise on Azure or AWS.
Note that included within the ArcGIS Enterprise/Online product is also a Portal that enables data, service and app sharing. Optionally ArcGIS Server/Portal can also include a Data Store that allows storing all vector and raster data within embedded PostgreSQL PostGIS database.
Some major products have been listed also in the screenshot, for example:
- GeoEvent Server is about capturing and managing real time big data, ArcGIS Notebook Server provides backend for Jupyter Notebook processes.
- ArcGIS for INSPIRE is the backend for the EU INSPIRE program, ArcGIS Utility Network Management (aka UNME) is network management engine for electricity, gas and 3-waters line utility companies.
- ArcGIS Living Atlas (example and link provided in the end of this blog) provides global set of value-add data downloadable or accessible as streaming services.
Some tools are new – take for example Artificial Intelligence and Machine Learning provided by Esri to help with clustering, classification and predictions, see diagram below:
These machine Learning tools have now been enabled and here is a link to the latest conference video on AI/ML.
Last part we want to share with you is some examples on data that Esri is sharing – these 6 data viewers showcase some of the value-add data Esri and its partners have provided freely for rest of the world to use and access:
Last part we want to share with you is some examples on data that Esri is sharing – these 6 data viewers showcase some of the value-add data Esri and its partners have provided freely for rest of the world to use and access:
Earth Systems Monitor with multiple web maps.
Elevation Profiles with Many Interesting Places.
Esri Ocean Layers – with many analytics layers.
|
Earth Observations with multiple web maps.
World Map through Multiple Lenses.
ArcGIS Living Atlas of the World – many layers.
|
There are links to access the viewers and datasets below the images. All except Esri ocean Layers are freely accessible, and do not require Esri logon. All except the two middle viewers – Elevation Profiles and Multiple Lenses – provide datasets, middle ones have a specific purpose:
I hope this has given you a perspective why we believe Esri and SAP really are BETTER TOGETHER!
- Elevation Profiles – allows you to navigate by picking a place from a thumbnail list on top or zoom/pan within globe map to your preferred area. Then you need to define the area within map by drawing a polygon, a polyline, a rectangle or a circle. System then defines an elevation profile for your selected measurement area and allows you to hover over the profile to highlight corresponding areas in the map.
- Multiple Lenses – showcases how a lens can be used within the mapping interface. There are four lenses to play with; the upper-right map and lower-left map showcase different type of topographical maps, but the main feature is they use native language labelling. For example, if you hover over these lenses on top of Asian countries, you will seen the landmarks written in that country’s language. Upper-felt lens is only useful when zoomed close – it brings additional labelling and slope/contour details for close views. The lower-right map is aerial imagery view.
I hope this has given you a perspective why we believe Esri and SAP really are BETTER TOGETHER!