SAP Migration to AWS - Considerations from Infrastructure side

[Sayan]

Introduction: This Blog explains few key areas in Infrastructure layer to be considered when we are planning to migrate our SAP Landscape to AWS.

Expected Readers:

1. Basis Consultant with basic AWS knowledge
2. SAP Basis Migration Tech Arch 
3. SAP Solution Architect
4. SAP pre-sales team in Basis area
5. Infra architect for any SAP migration assignment

Scenario:

This blog will cover below scenarios:

•   SAP Brownfield Migration in AWS
•   SAP Greenfield Implementation in AWS

Details:

Below are the major categories which we would need to focus on in the planning phase and have to be ready before we start with our POC/Baseline Migration. Each category provides a few options (not in every case/scenario) with pros/cons. This will help the design/architecture team to determine the right architecture for their client:

       1. VPC Architecture:

                        This will be the 1st thing that we should check while starting our designing session as this has high impact on Architecture / Cost. Here are few options based on my experience:

                     A) Complete Segregation of Landscape:   This scenario explains SAP Dev, QA, PROD are in private separate VPCs. Web-facing systems like web dispatcher, SAP routers will be in web-facing public VPCs & DR DB nodes/applications EC2 will be in another DR VPC. We can use Development, QA, and PROD account for building the same.

                  Pros:

•          We can have complete segregation of firewall, security policy which can help to determine the cost of different landscape / VPCs. 
•          Separating Web VPCs will allow us further safeguarding our Core SAP systems.
•          Before going for PROD migration , we can test all internal & external connectivity in Dev , QA and Non PROD Web VPCs.
•           Lesser operational overhead for managing Security Groups as we can define common SGs through out VPC and will get assign while provisioning EC2.

                  Cons:  This might have a bit of impact on cost. As we know we have transit gateway between every VPCs and any transit via transit gateway will have cost impact.

               B) Shared VPCs between Non PROD and PROD : This means we can share our non-production systems in one VPC, production in another VPC & web facing applications like SAP router, web dispatcher in one common VPCs. In this option, we can put our DR DB nodes in non-PROD VPCs only.

                 Pros :

•          We can have complete segregation of firewall, security policy between Production & Non Production SAP systems.
•           Separating Web VPCs will allow us further safeguarding our Core SAP systems.   
•           Before going for PROD migration , we can test all internal & external connectivity in Non PROD VPCs.
•           Cost effective comparing to Option A.

             Cons :

            Will not been able to make complete segregation between Development and QA landscape w.r.t firewall / security policy.

            Overhead on maintaining separate policy on SGs to be assign in Dev landscape Vs QA Landscape.

NOTE : We need to select our CIDR range based on the number of Ips we are anticipating in each option.

       2. External Traffic and Load balancing : As per my experience, this topic was somehow not fully discussed/aligned when we start our migration and become a challenge as we proceeded further with migration. Hence, strongly recommending discussing and closing it during planning session. Let me try and explain a few scenarios here based on my experience:

         A. Using same On prem F5 for managing External traffic & load balancing : In this scenario we need to update the existing F5 iRule with the AWS server details to navigate the traffic towards SAP. In most of the cases I see, the backend destination for F5 will be the SAP web dispatcher which will stay in web facing VPC (for some scenarios where we don’t have a web dispatcher like BOBJ, connections will go directly to BOBJ Tomcat server).

Considerations / Assumption / High level steps  :

1. No migration of iRules and no change of public IP since we are using the same F5.
2. No extra licensing cost.
3. We still need to have operational cost for maintaining the infra of F5 in on-prem.
4. Steps for allowing incoming connection from on-prem F5 to web facing F5.

    B. Using separate F5 for managing External traffic & load balancing : In this scenario , client wanted to completely decommission their on premise data center but wanted to continue with F5.

Considerations / Assumption / High level steps  :

              1. Complete removal of on-prem data center cost; however, there will be implementation effort for migration of the iRules from on-prem F5 to new F5 and updating the backend destination to AWS servers.

              2. New Public IP will need to be generated which will need to be shared with all external vendors for whitelisting.

              3. Steps for allowing incoming connection from on-prem F5 to web facing F5.

              4. We should be very clear on SSL termination, and in case SSL is getting terminated at the F5 level, we have to import the necessary certificate from the old F5. 

          C. Using AWS Inhouse functionality to manage External traffic & load balancing : Well, this is a very interesting option where we can use AWS native tools like NLB / ALB for managing external traffic and the same can be used for load balancing as well.      

Considerations / Assumption / High level steps:            

1. Complete removal of on-prem cost or F5 license cost where all the iRules of F5 need to be analyzed and implemented in NLB / ALB.
2. This needs to be implemented in the web-facing public VPC with the backend as SAP web dispatcher.
3. We should be very clear on SSL termination and in case SSL needs to be terminated in ALB, we have to import necessary certificates.
4. Less cost-effective compared to other options.
5. I strongly recommend exploring all the existing iRules with the infra-architect. In my experience, if we have a very complete path/port-based routing in F5, that might not be supported in AWS NLB/ALB.

3. Encryption to be Used for EBS: This item looks to be very simple and standard however if its not finalized and freeze , we might face challenges in later stage. In general AWS provided 3 option on encryption of EBS i.e

•       AWS Managed KMS key
•       Customer Managed KMS Key

The infra-architect should discuss this during the design session and close it, as once we provision EC2/start our migration, it would be difficult to change the options at the later stage.

4. Default Idle Timeout of  AWS Gateway Load Balancer: To explain this , I am assuming below most common  scenarios :

•        Users are connecting via VDI / Jumpbox which is part of one Public VPC
•        Users are connecting via VDI or Jumpbox or directly via direct connect which is part of inhouse corporate network
•        Users are connecting via VDI or Jumpbox from different Cloud Vendor like AWS

By default, all AWS VPC have a default Gateway Load Balancer idle timeout which is 350 seconds, and this is hard-coded. So, when connections are flowing across VPCs, SAP users will face very frequent session hang scenarios if the SAP work process idle time out is more than 350. In order to avoid this, I would suggest checking and reducing SAP parameter “rdisp/keepalive” to less than 350 seconds.

Another experience is when you have your BW landscape in one VPC but the source system in another VPC/on-prem. There will be an impact on full/delta load because of the setting. Please explore the following SAP Note: 1410736 - TCP/IP: setting keepalive interval, which resolve my issue.

Below Linux level keep alive parameter needs to be explored:

net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 75
net.ipv4.tcp_keepalive_probes = 9

       Ref : Best practices for deploying Gateway Load Balancer | Networking & Content Delivery (amazon.com)

4. DNS server and its Domain Controller impact on Windows HA: It’s another item which we should finalize and agree with client. Below are a few options:

1. Primary DNS in on-prem and AWS VPCs are having domain controller which will get sync with the primary DNS.
2. Primary DNS is with the different cloud vendor (like Azure) whereas AWS VPCs are having domain controller which will get sync with the primary DNS.
3. Primary DNS is with AWS

So, to sync any primary DNS with domain controller or vice versa, there will be a replication time, which is normally 10+ minutes. Now, if we have HA setup for windows, windows server failover cluster (popularly WSFC) will always update the ASCS / SCS node IP details directly in the DNS which is assigned to it.

For scenario 1 + 2, we should update the primary DNS details in WFSC setup as users mostly access primary detail. If we update AWS domain controller details in WSFC and a failover takes place, there might be a downtime of 10+ minutes for the users, as ASCS / SCS host details might not be synced with the primary DNS. Below picture might help to explain it in better / easy way –

5. Backup and Retention policy: There are no specific scenarios to discuss here, considering this is very generic. However, please ensure the following topics are covered and closed in the design/planning session:

1. EC2 backup and retention policy and corresponding TAG details to be assigned during EC2 provisioning.
2. Media to retain the backup and archiving strategy. In most cases I’ve come across, we take backups in S3 standard or S3 IA, and after a defined period we move this to Glacier for archiving.   
3. Depending on your database, you can decide on media, but again, in most of the cases, it will be S3. Please explore the option of using AWS native backup functionality depending on the region. Please refer this for SAP HANA blog

@ https://aws.amazon.com/about-aws/whats-new/2023/02/aws-systems-manager-sap-generally-available/

5. Time Zone sync with central NTP Server: As we aware, SAP applications always need to be in sync with server time zone. So, when we are planning to provision our EC2, we need to ensure it always gets synced with the same NTP server that on-prem servers are getting synced with.

6. Physical / Virtual Host name assignment to NIC : We mostly have both physical and virtual hostnames in our landscape. Below are the possible scenarios:

1.              One NIC: With this setup, we can have only one NIC where we assign the physical host to the physical IP and virtual host to virtual IP. In case we have DHCP or SSSD configuration, we will see physical hostname A record/PTR with both physical & virtual IPs.
2.             Multiple NIC: With this setup, we will assign one NIC to the primary hostname/IP and other NICs to other virtual hostnames/IPs. We will have a 1:1 DNS A record/PTR with this scenario.

Scenario 2 is a little more expensive than scenario 1 because of multiple NIC.