SAP HANA In-Memory Database\u202f<\/strong><\/span><\/h2>\nWhat is SAP HANA?<\/span><\/h3>\nSAP HANA (High-performance Analytical Appliance) is an in-memory database platform developed by\u00a0SAP SE<\/a>. It uses in-memory computing to store data primarily in RAM, unlike traditional relational databases that need to retrieve data from disk-based storage solutions. With this technological innovation, SAP HANA can access in-memory data 10,000 times faster than data stored on standard disks. The result is that companies can now rapidly analyze large amounts of data and process transactions in seconds rather than hours. SAP HANA is the platform for SAP\u2019s Enterprise Resource Planning software (S\/4HANA) as well as other business applications and can run on-premises, in the cloud or in a hybrid configuration. Enterprises that have migrated over to SAP HANA have been able to realize\u00a0accelerated business processes, improved data insights, and simplified IT environments. Since SAP HANA can run both OLTP and OLAP workloads, it eliminates the need to move data to another database to run analytical applications. This removes the burden of maintaining separate legacy systems, data silos and data warehouses. The result is less data redundancy, a smaller hardware footprint and less data management costs as well.\u00a0SAP HANA analyzes live data for real-time business decisions and analytics, using advanced data processing engines for business, text, spatial, graph and series data. The initial version of the SAP HANA database was released to select customers in late 2010. SAP later debuted the general release at SAPPHIRE NOW, SAP\u2019s annual technology conference in Orlando, FL in June 2011. SAP HANA made history as the first in-memory database in the world. It was an extremely popular product release, quickly becoming SAP\u2019s fastest-adopted solution. Today there are over 30,000 customers utilizing SAP HANA.<\/p>\n![\"Overview](\"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-content\/uploads\/sites\/342\/2025\/07\/sap_hana1.jpg\")
<\/p>\n
SAP HANA Platform<\/span><\/h3>\n![\"Overview](\"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-content\/uploads\/sites\/342\/2025\/07\/SAP-HANA.jpeg\")
<\/p>\n
While SAP HANA is mostly recognized for its ability to process large amounts of data at record speeds, it is far more than just an in-memory database. SAP HANA offers both column and row-based storage, making it capable of handling both OLTP and OLAP workloads. Tables that are organized in columns are optimized for high performing read operations while still providing good performance for write operations. In addition, column store offers extremely efficient data compression, which in turn saves memory and speeds up searches and calculations. Typical compression rates are 7x or greater when compared with a traditional RDBMS. Other column store features include table partitioning (which can further improve performance by utilizing partition pruning), and a delta store to optimize write operations. Another technology built into the platform includes dynamic tiering, allowing for a temperature based (hot\/warm\/cold) data aging strategy. The purpose of this feature is to extend SAP HANA memory with a disk-centric columnar store for less frequently used data. In addition, SAP HANA offers data integration services, high availability and disaster recovery capabilities, and a complete development platform known as SAP HANA XSA (SAP HANA extended application services, advanced model) that can be used to code with Java and Node.js. It includes OData support and is the successor to the now-deprecated XS classic model. Apps that were created with SAP HANA XS can be ported over to SAP HANA XSA. The SAP Web IDE for SAP HANA is a browser-based integrated development environment used to create web-based and mobile user interfaces, business logic, and advanced data models. It provides a handful of developer tools, including syntax-aware code editors, inspection tools, debugging tools, and CDS modeling capability.<\/p>\n
SAP HANA and Machine Learning<\/span><\/h3>\n\nPython Machine Learning Client for SAP HANA<\/span><\/h4>\n<\/div>\n\nWelcome to Python machine learning client for SAP HANA (hana-ml)! This package enables Python data scientists to access SAP HANA data and build various machine learning models using the data directly in SAP HANA. This page provides an overview of hana-ml. Python machine learning client for SAP HANA consists of two main parts:<\/p>\n
\n- SAP HANA DataFrame, which provides a set of methods for accessing and querying data in SAP HANA without bringing the data to the client.<\/li>\n
- A set of machine learning APIs for developing machine learning models.<\/li>\n<\/ul>\n
Specifically, machine learning APIs are composed of two packages:<\/p>\n
\n- PAL packagePAL package consists of a set of Python algorithms and functions which provide access to machine learning capabilities in SAP HANA Predictive Analysis Library(PAL). SAP HANA PAL functions cover a variety of machine learning algorithms for training a model and then the trained model is used for scoring.<\/li>\n
- APL packageAutomated Predictive Library (APL) package exposes the data mining capabilities of the Automated Analytics engine in SAP HANA through a set of functions. These functions develop a predictive modeling process that analysts can use to answer simple questions on their customer datasets stored in SAP HANA.<\/li>\n<\/ul>\n
hana-ml uses SAP HANA Python driver (hdbcli) to connect to and access SAP HANA. A figure of architecture is shown below: ![\"\"](\"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-content\/uploads\/sites\/342\/2025\/07\/SAP-HANA-3-300x169.jpg\")
<\/p>\n<\/div>\n
For more information, see the\u00a0Python Machine Learning Client for SAP HANA website<\/a><\/p>\n\u00a0<\/h2>\nSecure cloud infrastructure\u202f<\/strong><\/span><\/h2>\nThe team at HPI.MS worked closely with engineers at\u00a0Microsoft<\/a>, Mount Sinai IT and\u00a0Data4Life\u00a0<\/a>to establish secure Microsoft Azure cloud infrastructure to run AIR.MS.\u00a0The following architecture diagram depicts the AIR.MS infrastructure running in the Mount Sinai Azure cloud. ![\"\"](\"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-content\/uploads\/sites\/342\/2025\/07\/Secure-cloud-infrastructure-e1753886997712-300x228.jpg\")
<\/p>\n\u00a0<\/h2>\nPython Libraries\u202f<\/strong><\/span><\/h2>\nSAP HANA ML (Machine Learning) is one of the Python libraries that we provide to researchers to query AIR.MS. SAP HANA ML is a component of SAP HANA, an in-memory database and analytics platform designed for handling massive amounts of data in real-time. HANA ML provides integrated machine learning capabilities to build, deploy, and execute machine learning models directly within the SAP HANA environment. It provides a wide range of machine learning algorithms, such as regression, classification, clustering, and time-series forecasting, that can be used for various use cases like patient dataset preparation, notes analysis, etc. It offers a set of libraries and APIs to develop custom machine learning models, or users can use pre-built models from the SAP HANA Model Store. The biggest advantage of HANA ML is In-Database Processing. HANA ML leverages the power of SAP HANA\u2019s in-memory processing capabilities, allowing data to be analyzed and processed directly within the database. This eliminates the need for data movement, resulting in faster model training and predictions. HANA ML also supports model versioning and model management, allowing researchers to track and control the lifecycle of their machine learning models. Additionally, it offers model explainability features, providing insights into how the models arrive at their predictions, which is crucial for ensuring transparency and compliance with regulations. More details can be found in\u00a0the SAP Documentation<\/a>.<\/p>\n\u00a0<\/h2>\nBenchmarking\u202f<\/strong><\/span><\/h2>\n\n\n\n\n\n\n\n\n\n\nAIR.MS in-memory full-text search performance examples<\/span><\/h2>\n<\/div>\nNumber of indexed notes: 86,893,767<\/pre>\n\nExample 1:<\/strong>\u00a0Show\u00a0count\u00a0<\/strong>for notes containing an\u00a0EXACT\u00a0<\/strong>search string ‘acute kidney disease<\/strong>‘, broken down by\u00a0gender<\/strong><\/span><\/span><\/h4>\n<\/div>\n\n\n\n\n\n\n\n\n![\"\"](\"https:\/\/mtsinai.sharepoint.com\/sites\/airms\/SiteAssets\/SitePages\/Benchmarking\/2344342433-example-1.GIF\")
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\u00a0<\/h2>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\u00a0<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n0.31\u00a0second runtime!<\/strong><\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\nExample\u00a02:<\/strong>\u00a0Show\u00a0count\u00a0<\/strong>for\u00a0notes\u00a0containing\u00a0FUZZY\u00a0<\/strong>search\u00a0string\u00a0‘acute\u00a0kidney\u00a0disease<\/strong>‘, broken\u00a0down\u00a0by\u00a0gender<\/strong><\/span><\/span><\/h4>\n<\/div>\n\n\n\n\n\n\n\n\n![\"\"](\"https:\/\/mtsinai.sharepoint.com\/sites\/airms\/SiteAssets\/SitePages\/Benchmarking\/2818629564-example-2.GIF\")
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\u00a0<\/h3>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\u00a0<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n0.39\u00a0second runtime!<\/strong><\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\nExample 3:<\/strong>\u00a0Show\u00a0results\u00a0<\/strong>for\u00a0FUZZY\u00a0<\/strong>search\u00a0containing\u00a0term\u00a0‘ulcerative\u00a0colitis<\/strong>‘, and\u00a0group\u00a0by\u00a0Note\u00a0Type<\/strong><\/span><\/span><\/h4>\n<\/div>\n\n\n\n\n\n\n\n\n![\"\"](\"https:\/\/mtsinai.sharepoint.com\/sites\/airms\/SiteAssets\/SitePages\/Benchmarking\/2765645631-example-3.GIF\")
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\u00a0<\/h2>\nAIR\u00b7MS Data Modalities\u202f<\/strong><\/span><\/h2>\n\n\n\n\n\n\n\n\n\nAIR\u00b7MS data<\/span><\/h3>\nThe AIR\u00b7MS platform proudly features the following Mount Sinai and public datasets. Our team is dedicated to continuously expanding our database with additional data modalities<\/a>, striving to build a comprehensive, multi-modal research resource. To help you get started with the data, we offer Quick Start guides available\u00a0here<\/a>.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\n\n\u00a0<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n![\"\"](\"https:\/\/media.akamai.odsp.cdn.office.net\/eastus1-mediap.svc.ms\/transform\/thumbnail?provider=url&inputFormat=jpg&docid=https%3A%2F%2Fcdn.hubblecontent.osi.office.net%2Fm365content%2Fpublish%2F953d441a-7f1c-473c-92ea-de20493a9397%2F936160696.jpg&w=400\")
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\u00a0<\/h3>\nMount Sinai Data Warehouse (MSDW)\u00a0in OMOP format<\/span><\/h3>\nThe MSDW\u00a0<\/a>dataset leverages the\u00a0OMOP Common Data Model<\/a>. The data is comprised of\u00a0clinical data extracted from Mount Sinai\u2019s Epic Caboodle database and other ancillary systems. \u200b\u200b\u200b\u200b\u200b\u200b\u200b<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n![\"\"](\"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-content\/uploads\/sites\/342\/2025\/07\/ohdsi-omop-cdm-v5-4-300x182.png\")
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n\n\n\n\n\n\n\n\u00a0<\/h3>\nWe offer both an identifiable and a de-identified version of the MSDW dataset in AIR\u00b7MS<\/h3>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
SAP HANA (High-performance Analytical Appliance) is an in-memory database platform developed by\u00a0SAP SE<\/a>. It uses in-memory computing to store data primarily in RAM, unlike traditional relational databases that need to retrieve data from disk-based storage solutions. With this technological innovation, SAP HANA can access in-memory data 10,000 times faster than data stored on standard disks. The result is that companies can now rapidly analyze large amounts of data and process transactions in seconds rather than hours. SAP HANA is the platform for SAP\u2019s Enterprise Resource Planning software (S\/4HANA) as well as other business applications and can run on-premises, in the cloud or in a hybrid configuration. Enterprises that have migrated over to SAP HANA have been able to realize\u00a0accelerated business processes, improved data insights, and simplified IT environments. Since SAP HANA can run both OLTP and OLAP workloads, it eliminates the need to move data to another database to run analytical applications. This removes the burden of maintaining separate legacy systems, data silos and data warehouses. The result is less data redundancy, a smaller hardware footprint and less data management costs as well.\u00a0SAP HANA analyzes live data for real-time business decisions and analytics, using advanced data processing engines for business, text, spatial, graph and series data. The initial version of the SAP HANA database was released to select customers in late 2010. SAP later debuted the general release at SAPPHIRE NOW, SAP\u2019s annual technology conference in Orlando, FL in June 2011. SAP HANA made history as the first in-memory database in the world. It was an extremely popular product release, quickly becoming SAP\u2019s fastest-adopted solution. Today there are over 30,000 customers utilizing SAP HANA.<\/p>\n While SAP HANA is mostly recognized for its ability to process large amounts of data at record speeds, it is far more than just an in-memory database. SAP HANA offers both column and row-based storage, making it capable of handling both OLTP and OLAP workloads. Tables that are organized in columns are optimized for high performing read operations while still providing good performance for write operations. In addition, column store offers extremely efficient data compression, which in turn saves memory and speeds up searches and calculations. Typical compression rates are 7x or greater when compared with a traditional RDBMS. Other column store features include table partitioning (which can further improve performance by utilizing partition pruning), and a delta store to optimize write operations. Another technology built into the platform includes dynamic tiering, allowing for a temperature based (hot\/warm\/cold) data aging strategy. The purpose of this feature is to extend SAP HANA memory with a disk-centric columnar store for less frequently used data. In addition, SAP HANA offers data integration services, high availability and disaster recovery capabilities, and a complete development platform known as SAP HANA XSA (SAP HANA extended application services, advanced model) that can be used to code with Java and Node.js. It includes OData support and is the successor to the now-deprecated XS classic model. Apps that were created with SAP HANA XS can be ported over to SAP HANA XSA. The SAP Web IDE for SAP HANA is a browser-based integrated development environment used to create web-based and mobile user interfaces, business logic, and advanced data models. It provides a handful of developer tools, including syntax-aware code editors, inspection tools, debugging tools, and CDS modeling capability.<\/p>\n Welcome to Python machine learning client for SAP HANA (hana-ml)! This package enables Python data scientists to access SAP HANA data and build various machine learning models using the data directly in SAP HANA. This page provides an overview of hana-ml. Python machine learning client for SAP HANA consists of two main parts:<\/p>\n Specifically, machine learning APIs are composed of two packages:<\/p>\n hana-ml uses SAP HANA Python driver (hdbcli) to connect to and access SAP HANA. A figure of architecture is shown below: For more information, see the\u00a0Python Machine Learning Client for SAP HANA website<\/a><\/p>\n The team at HPI.MS worked closely with engineers at\u00a0Microsoft<\/a>, Mount Sinai IT and\u00a0Data4Life\u00a0<\/a>to establish secure Microsoft Azure cloud infrastructure to run AIR.MS.\u00a0The following architecture diagram depicts the AIR.MS infrastructure running in the Mount Sinai Azure cloud. SAP HANA ML (Machine Learning) is one of the Python libraries that we provide to researchers to query AIR.MS. SAP HANA ML is a component of SAP HANA, an in-memory database and analytics platform designed for handling massive amounts of data in real-time. HANA ML provides integrated machine learning capabilities to build, deploy, and execute machine learning models directly within the SAP HANA environment. It provides a wide range of machine learning algorithms, such as regression, classification, clustering, and time-series forecasting, that can be used for various use cases like patient dataset preparation, notes analysis, etc. It offers a set of libraries and APIs to develop custom machine learning models, or users can use pre-built models from the SAP HANA Model Store. The biggest advantage of HANA ML is In-Database Processing. HANA ML leverages the power of SAP HANA\u2019s in-memory processing capabilities, allowing data to be analyzed and processed directly within the database. This eliminates the need for data movement, resulting in faster model training and predictions. HANA ML also supports model versioning and model management, allowing researchers to track and control the lifecycle of their machine learning models. Additionally, it offers model explainability features, providing insights into how the models arrive at their predictions, which is crucial for ensuring transparency and compliance with regulations. More details can be found in\u00a0the SAP Documentation<\/a>.<\/p>\n The AIR\u00b7MS platform proudly features the following Mount Sinai and public datasets. Our team is dedicated to continuously expanding our database with additional data modalities<\/a>, striving to build a comprehensive, multi-modal research resource. To help you get started with the data, we offer Quick Start guides available\u00a0here<\/a>.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n The MSDW\u00a0<\/a>dataset leverages the\u00a0OMOP Common Data Model<\/a>. The data is comprised of\u00a0clinical data extracted from Mount Sinai\u2019s Epic Caboodle database and other ancillary systems. \u200b\u200b\u200b\u200b\u200b\u200b\u200b<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/p>\nSAP HANA Platform<\/span><\/h3>\n
<\/p>\nSAP HANA and Machine Learning<\/span><\/h3>\n
Python Machine Learning Client for SAP HANA<\/span><\/h4>\n<\/div>\n
\n
\n
<\/p>\n<\/div>\n\u00a0<\/h2>\n
Secure cloud infrastructure\u202f<\/strong><\/span><\/h2>\n
<\/p>\n\u00a0<\/h2>\n
Python Libraries\u202f<\/strong><\/span><\/h2>\n
\u00a0<\/h2>\n
Benchmarking\u202f<\/strong><\/span><\/h2>\n
AIR.MS in-memory full-text search performance examples<\/span><\/h2>\n<\/div>\n
Number of indexed notes: 86,893,767<\/pre>\n
Example 1:<\/strong>\u00a0Show\u00a0count\u00a0<\/strong>for notes containing an\u00a0EXACT\u00a0<\/strong>search string ‘acute kidney disease<\/strong>‘, broken down by\u00a0gender<\/strong><\/span><\/span><\/h4>\n<\/div>\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\u00a0<\/h2>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\nExample\u00a02:<\/strong>\u00a0Show\u00a0count\u00a0<\/strong>for\u00a0notes\u00a0containing\u00a0FUZZY\u00a0<\/strong>search\u00a0string\u00a0‘acute\u00a0kidney\u00a0disease<\/strong>‘, broken\u00a0down\u00a0by\u00a0gender<\/strong><\/span><\/span><\/h4>\n<\/div>\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\u00a0<\/h3>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\nExample 3:<\/strong>\u00a0Show\u00a0results\u00a0<\/strong>for\u00a0FUZZY\u00a0<\/strong>search\u00a0containing\u00a0term\u00a0‘ulcerative\u00a0colitis<\/strong>‘, and\u00a0group\u00a0by\u00a0Note\u00a0Type<\/strong><\/span><\/span><\/h4>\n<\/div>\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\u00a0<\/h2>\n
AIR\u00b7MS Data Modalities\u202f<\/strong><\/span><\/h2>\n
AIR\u00b7MS data<\/span><\/h3>\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\u00a0<\/h3>\n
Mount Sinai Data Warehouse (MSDW)\u00a0in OMOP format<\/span><\/h3>\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n\u00a0<\/h3>\n
We offer both an identifiable and a de-identified version of the MSDW dataset in AIR\u00b7MS<\/h3>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n